A dual role of prestimulus spontaneous neural activity in visual object recognition

Quality space computations for consciousness

The quality space hypothesis about conscious experience proposes that conscious sensory states are experienced in relation to other possible sensory states. For instance, the colour red is experienced as being more like orange, and less like green or blue. Recent empirical findings suggest that subjective similarity space can be explained in terms of similarities in neural activation patterns. Here, we consider how localist, workspace, and higher-order theories of consciousness can accommodate claims about the qualitative character of experience and functionally support a quality space. We review existing empirical evidence for each of these positions, and highlight novel experimental tools, such as altering local activation spaces via brain stimulation or behavioural training, that can distinguish these accounts.

Network mechanisms of ongoing brain activity's influence on conscious visual perception

Sensory inputs enter a constantly active brain, whose state is always changing from one moment to the next. Currently, little is known about how ongoing, spontaneous brain activity participates in online task processing. We employed 7 Tesla fMRI and a threshold-level visual perception task to probe the effects of prestimulus ongoing brain activity on perceptual decision-making and conscious recognition. Prestimulus activity originating from distributed brain regions, including visual cortices and regions of the default-mode and cingulo-opercular networks, exerted a diverse set of effects on the sensitivity and criterion of conscious recognition, and categorization performance. We further elucidate the mechanisms underlying these behavioral effects, revealing how prestimulus activity modulates multiple aspects of stimulus processing in highly specific and network-dependent manners. These findings reveal heretofore unknown network mechanisms underlying ongoing brain activity's influence on conscious perception, and may hold implications for understanding the precise roles of spontaneous activity in other brain functions.

Beyond task response-Pre-stimulus activity modulates contents of consciousness

The current discussion on the neural correlates of the contents of consciousness (NCCc) focuses mainly on the post-stimulus period of task-related activity. This neglects the substantial impact of the spontaneous or ongoing activity of the brain as manifest in pre-stimulus activity. Does the interaction of pre- and post-stimulus activity shape the contents of consciousness? Addressing this gap in our knowledge, we review and converge two recent lines of findings, that is, pre-stimulus alpha power and pre- and post-stimulus alpha trial-to-trial variability (TTV). The data show that pre-stimulus alpha power modulates post-stimulus activity including specifically the subjective features of conscious contents like confidence and vividness. At the same time, alpha pre-stimulus variability shapes post-stimulus TTV reduction including the associated contents of consciousness. We propose that non-additive rather than merely additive interaction of the internal pre-stimulus activity with the external stimulus in the alpha band is key for contents to become conscious. This is mediated by mechanisms on different levels including neurophysiological, neurocomputational, neurodynamic, neuropsychological and neurophenomenal levels. Overall, considering the interplay of pre-stimulus intrinsic and post-stimulus extrinsic activity across wider timescales, not just evoked responses in the post-stimulus period, is critical for identifying neural correlates of consciousness. This is well in line with both processing and especially the Temporo-spatial theory of consciousness (TTC).

An integrative view of the role of prefrontal cortex in consciousness

The involvement of the prefrontal cortex (PFC) in consciousness is an ongoing focus of intense investigation. An important question is whether representations of conscious contents and experiences in the PFC are confounded by post-perceptual processes related to cognitive functions. Here, I review recent findings suggesting that neuronal representations of consciously perceived contents-in the absence of post-perceptual processes-can indeed be observed in the PFC. Slower ongoing fluctuations in the electrophysiological state of the PFC seem to control the stability and updates of these prefrontal representations of conscious awareness. In addition to conscious perception, the PFC has been shown to play a critical role in controlling the levels of consciousness as observed during anesthesia, while prefrontal lesions can result in severe loss of perceptual awareness. Together, the convergence of these processes in the PFC suggests its integrative role in consciousness and highlights the complex nature of consciousness itself.

Double Dissociation of Spontaneous Alpha-Band Activity and Pupil-Linked Arousal on Additive and Multiplicative Perceptual Gain

Perception is a probabilistic process dependent on external stimulus properties and one's internal state. However, which internal states influence perception and via what mechanisms remain debated. We studied how spontaneous alpha-band activity (8-13 Hz) and pupil fluctuations impact visual detection and confidence across stimulus contrast levels (i.e., the contrast response function, CRF). In human subjects of both sexes, we found that low prestimulus alpha power induced an "additive" shift in the CRF, whereby stimuli were reported present more frequently at all contrast levels, including contrast of zero (i.e., false alarms). Conversely, prestimulus pupil size had a "multiplicative" effect on detection such that stimuli occurring during large pupil states (putatively corresponding to higher arousal) were perceived more frequently as contrast increased. Signal detection modeling reveals that alpha power changes detection criteria equally across the CRF but not detection sensitivity (d'), whereas pupil-linked arousal modulated sensitivity, particularly for higher contrasts. Interestingly, pupil size and alpha power were positively correlated, meaning that some of the effect of alpha on detection may be mediated by pupil fluctuations. However, pupil-independent alpha still induced an additive shift in the CRF corresponding to a criterion effect. Our data imply that low alpha boosts detection and confidence by an additive factor, rather than by a multiplicative scaling of contrast responses, a profile which captures the effect of pupil-linked arousal. We suggest that alpha power and arousal fluctuations have dissociable effects on behavior. Alpha reflects the baseline level of visual excitability, which can vary independent of arousal.

Neural Mechanisms Determining the Duration of Task-free, Self-paced Visual Perception

Humans spend hours each day spontaneously engaging with visual content, free from specific tasks and at their own pace. Currently, the brain mechanisms determining the duration of self-paced perceptual behavior remain largely unknown. Here, participants viewed naturalistic images under task-free settings and self-paced each image's viewing duration while undergoing EEG and pupillometry recordings. Across two independent data sets, we observed large inter- and intra-individual variability in viewing duration. However, beyond an image's presentation order and category, specific image content had no consistent effects on spontaneous viewing duration across participants. Overall, longer viewing durations were associated with sustained enhanced posterior positivity and anterior negativity in the ERPs. Individual-specific variations in the spontaneous viewing duration were consistently correlated with evoked EEG activity amplitudes and pupil size changes. By contrast, presentation order was selectively correlated with baseline alpha power and baseline pupil size. Critically, spontaneous viewing duration was strongly predicted by the temporal stability in neural activity patterns starting as early as 350 msec after image onset, suggesting that early neural stability is a key predictor for sustained perceptual engagement. Interestingly, neither bottom-up nor top-down predictions about image category influenced spontaneous viewing duration. Overall, these results suggest that individual-specific factors can influence perceptual processing at a surprisingly early time point and influence the multifaceted ebb and flow of spontaneous human perceptual behavior in naturalistic settings.

Identifying content-invariant neural signatures of perceptual vividness

Some conscious experiences are more vivid than others. Although perceptual vividness is a key component of human consciousness, how variation in this magnitude property is registered by the human brain is unknown. A striking feature of neural codes for magnitude in other psychological domains, such as number or reward, is that the magnitude property is represented independently of its sensory features. To test whether perceptual vividness also covaries with neural codes that are invariant to sensory content, we reanalyzed existing magnetoencephalography and functional MRI data from two distinct studies which quantified perceptual vividness via subjective ratings of awareness and visibility. Using representational similarity and decoding analyses, we find evidence for content-invariant neural signatures of perceptual vividness distributed across visual, parietal, and frontal cortices. Our findings indicate that the neural correlates of subjective vividness may share similar properties to magnitude codes in other cognitive domains.

Expectation Cues and False Percepts Generate Stimulus-Specific Activity in Distinct Layers of the Early Visual Cortex

Perception has been proposed to result from the integration of feedforward sensory signals with internally generated feedback signals. Feedback signals are believed to play an important role in driving false percepts, that is, seeing things that are not actually there. Feedforward and feedback influences on perception can be studied using layer-specific fMRI, which we used here to interrogate neural activity underlying high-confidence false percepts while healthy human participants (N = 25, male and female) performed a perceptual orientation discrimination task. Auditory cues implicitly signaled the most likely upcoming orientation (referred to here as expectations). These expectations induced orientation-specific templates in the deep and superficial layers of V2, without affecting perception. In contrast, the orientation of falsely perceived stimuli with high confidence was reflected in the middle input layers of V2, suggesting a feedforward signal contributing to false percepts. The prevalence of high-confidence false percepts was related to everyday hallucination severity in a separate online sample (N = 100), suggesting a possible link with abnormal perceptual experiences. These results reveal a potential feedforward mechanism underlying false percepts, reflected by spontaneous stimulus-like activity in the input layers of the visual cortex, independent of top-down signals reflecting cued orientations.SIGNIFICANCE STATEMENT False percepts have been suggested to arise through excessive feedback signals. However, feedforward contributions to false percepts have remained largely understudied. Laminar fMRI has been shown to be useful in distinguishing feedforward from feedback activity as it allows the imaging of different cortical layers. In the present study we demonstrate that although cued orientations are encoded in the feedback layers of the visual cortex, the content of the false percepts are encoded in the feedforward layers and did not rely on these cued orientations. This shows that false percepts can in principle emerge from random feedforward signals in the visual cortex, with possible implications for disorders hallmarked by hallucinations like schizophrenia and Parkinson's disease.

Functional neuroimaging as a catalyst for integrated neuroscience

Functional magnetic resonance imaging (fMRI) enables non-invasive access to the awake, behaving human brain. By tracking whole-brain signals across a diverse range of cognitive and behavioural states or mapping differences associated with specific traits or clinical conditions, fMRI has advanced our understanding of brain function and its links to both normal and atypical behaviour. Despite this headway, progress in human cognitive neuroscience that uses fMRI has been relatively isolated from rapid advances in other subdomains of neuroscience, which themselves are also somewhat siloed from one another. In this Perspective, we argue that fMRI is well-placed to integrate the diverse subfields of systems, cognitive, computational and clinical neuroscience. We first summarize the strengths and weaknesses of fMRI as an imaging tool, then highlight examples of studies that have successfully used fMRI in each subdomain of neuroscience. We then provide a roadmap for the future advances that will be needed to realize this integrative vision. In this way, we hope to demonstrate how fMRI can help usher in a new era of interdisciplinary coherence in neuroscience.

Temporospatial Nestedness in Consciousness: An Updated Perspective on the Temporospatial Theory of Consciousness

Time and space are fundamental elements that permeate the fabric of nature, and their significance in relation to neural activity and consciousness remains a compelling yet unexplored area of research. The Temporospatial Theory of Consciousness (TTC) provides a framework that links time, space, neural activity, and consciousness, shedding light on the intricate relationships among these dimensions. In this review, I revisit the fundamental concepts and mechanisms proposed by the TTC, with a particular focus on the central concept of temporospatial nestedness. I propose an extension of temporospatial nestedness by incorporating the nested relationship between the temporal circuit and functional geometry of the brain. To further unravel the complexities of temporospatial nestedness, future research directions should emphasize the characterization of functional geometry and the temporal circuit across multiple spatial and temporal scales. Investigating the links between these scales will yield a more comprehensive understanding of how spatial organization and temporal dynamics contribute to conscious states. This integrative approach holds the potential to uncover novel insights into the neural basis of consciousness and reshape our understanding of the world-brain dynamic.

Spatiotemporal neural dynamics of object recognition under uncertainty in humans

While there is a wealth of knowledge about core object recognition-our ability to recognize clear, high-contrast object images-how the brain accomplishes object recognition tasks under increased uncertainty remains poorly understood. We investigated the spatiotemporal neural dynamics underlying object recognition under increased uncertainty by combining MEG and 7 Tesla (7T) fMRI in humans during a threshold-level object recognition task. We observed an early, parallel rise of recognition-related signals across ventral visual and frontoparietal regions that preceded the emergence of category-related information. Recognition-related signals in ventral visual regions were best explained by a two-state representational format whereby brain activity bifurcated for recognized and unrecognized images. By contrast, recognition-related signals in frontoparietal regions exhibited a reduced representational space for recognized images, yet with sharper category information. These results provide a spatiotemporally resolved view of neural activity supporting object recognition under uncertainty, revealing a pattern distinct from that underlying core object recognition.

Towards a pluralistic neurobiological understanding of consciousness

Theories of consciousness are often based on the assumption that a single, unified neurobiological account will explain different types of conscious awareness. However, recent findings show that, even within a single modality such as conscious visual perception, the anatomical location, timing, and information flow of neural activity related to conscious awareness vary depending on both external and internal factors. This suggests that the search for generic neural correlates of consciousness may not be fruitful. I argue that consciousness science requires a more pluralistic approach and propose a new framework: joint determinant theory (JDT). This theory may be capable of accommodating different brain circuit mechanisms for conscious contents as varied as percepts, wills, memories, emotions, and thoughts, as well as their integrated experience.

Cognition is entangled with metabolism: relevance for resting-state EEG-fMRI

The brain is a living organ with distinct metabolic constraints. However, these constraints are typically considered as secondary or supportive of information processing which is primarily performed by neurons. The default operational definition of neural information processing is that (1) it is ultimately encoded as a change in individual neuronal firing rate as this correlates with the presentation of a peripheral stimulus, motor action or cognitive task. Two additional assumptions are associated with this default interpretation: (2) that the incessant background firing activity against which changes in activity are measured plays no role in assigning significance to the extrinsically evoked change in neural firing, and (3) that the metabolic energy that sustains this background activity and which correlates with differences in neuronal firing rate is merely a response to an evoked change in neuronal activity. These assumptions underlie the design, implementation, and interpretation of neuroimaging studies, particularly fMRI, which relies on changes in blood oxygen as an indirect measure of neural activity. In this article we reconsider all three of these assumptions in light of recent evidence. We suggest that by combining EEG with fMRI, new experimental work can reconcile emerging controversies in neurovascular coupling and the significance of ongoing, background activity during resting-state paradigms. A new conceptual framework for neuroimaging paradigms is developed to investigate how ongoing neural activity is "entangled" with metabolism. That is, in addition to being recruited to support locally evoked neuronal activity (the traditional hemodynamic response), changes in metabolic support may be independently "invoked" by non-local brain regions, yielding flexible neurovascular coupling dynamics that inform the cognitive context. This framework demonstrates how multimodal neuroimaging is necessary to probe the neurometabolic foundations of cognition, with implications for the study of neuropsychiatric disorders.

Bistability of prefrontal states gates access to consciousness

Access of sensory information to consciousness has been linked to the ignition of content-specific representations in association cortices. How does ignition interact with intrinsic cortical state fluctuations to give rise to conscious perception? We addressed this question in the prefrontal cortex (PFC) by combining multi-electrode recordings with a binocular rivalry (BR) paradigm inducing spontaneously driven changes in the content of consciousness, inferred from the reflexive optokinetic nystagmus (OKN) pattern. We find that fluctuations between low-frequency (LF, 1-9 Hz) and beta (∼20-40 Hz) local field potentials (LFPs) reflect competition between spontaneous updates and stability of conscious contents, respectively. Both LF and beta events were locally modulated. The phase of the former locked differentially to the competing populations just before a spontaneous transition while the latter synchronized the neuronal ensemble coding the consciously perceived content. These results suggest that prefrontal state fluctuations gate conscious perception by mediating internal states that facilitate perceptual update and stability.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

Cortex-wide spontaneous activity non-linearly steers propagating sensory-evoked activity in awake mice

The brain responds highly variably to identical sensory inputs, but there is no consensus on the nature of this variability. We explore this question using cortex-wide optical voltage imaging and whisker stimulation in awake mice. Clustering analysis reveals that the sensory-evoked activity propagates over the cortex via distinct pathways associated with distinct behavioral states. The pathway taken by each trial is independent of the level of primary sensory-evoked activation but is partially predictable by the spatiotemporal features of the preceding cortical spontaneous activity patterns. The sensory inputs reduce trial-to-trial variability in brain activity and alter temporal autocorrelation in spatial activity pattern evolutions, suggesting non-linear interactions between evoked activities and spontaneous activities. Further, evoked activities and spontaneous activities occupy different positions in the state space, suggesting that sensory inputs can intricately interact with the internal state to generate large-scale evoked activity patterns not frequented by spontaneous brain states.

Cortical response variability is driven by local excitability changes with somatotopic organization

Identical sensory stimuli can lead to different neural responses depending on the instantaneous brain state. Specifically, neural excitability in sensory areas may shape the brain´s response already from earliest cortical processing onwards. However, whether these dynamics affect a given sensory domain as a whole or occur on a spatially local level is largely unknown. We studied this in the somatosensory domain of 38 human participants with EEG, presenting stimuli to the median and tibial nerves alternatingly, and testing the co-variation of initial cortical responses in hand and foot areas, as well as their relation to pre-stimulus oscillatory states. We found that amplitude fluctuations of initial cortical responses to hand and foot stimulation - the N20 and P40 components of the somatosensory evoked potential (SEP), respectively - were not related, indicating local excitability changes in primary sensory regions. In addition, effects of pre-stimulus alpha (8-13 Hz) and beta (18-23 Hz) band amplitude on hand-related responses showed a robust somatotopic organization, thus further strengthening the notion of local excitability fluctuations. However, for foot-related responses, the spatial specificity of pre-stimulus effects was less consistent across frequency bands, with beta appearing to be more foot-specific than alpha. Connectivity analyses in source space suggested this to be due to a somatosensory alpha rhythm that is primarily driven by activity in hand regions while beta frequencies may operate in a more hand-region-independent manner. Altogether, our findings suggest spatially distinct excitability dynamics within the primary somatosensory cortex, yet with the caveat that frequency-specific processes in one sub-region may not readily generalize to other sub-regions.

Spatiotemporally resolved multivariate pattern analysis for M/EEG

An emerging goal in neuroscience is tracking what information is represented in brain activity over time as a participant completes some task. While electroencephalography (EEG) and magnetoencephalography (MEG) offer millisecond temporal resolution of how activity patterns emerge and evolve, standard decoding methods present significant barriers to interpretability as they obscure the underlying spatial and temporal activity patterns. We instead propose the use of a generative encoding model framework that simultaneously infers the multivariate spatial patterns of activity and the variable timing at which these patterns emerge on individual trials. An encoding model inversion maps from these parameters to the equivalent decoding model, allowing predictions to be made about unseen test data in the same way as in standard decoding methodology. These SpatioTemporally Resolved MVPA (STRM) models can be flexibly applied to a wide variety of experimental paradigms, including classification and regression tasks. We show that these models provide insightful maps of the activity driving predictive accuracy metrics; demonstrate behaviourally meaningful variation in the timing of pattern emergence on individual trials; and achieve predictive accuracies that are either equivalent or surpass those achieved by more widely used methods. This provides a new avenue for investigating the brain's representational dynamics and could ultimately support more flexible experimental designs in the future.

Arousal state affects perceptual decision-making by modulating hierarchical sensory processing in a large-scale visual system model

Arousal levels strongly affect task performance. Yet, what arousal level is optimal for a task depends on its difficulty. Easy task performance peaks at higher arousal levels, whereas performance on difficult tasks displays an inverted U-shape relationship with arousal, peaking at medium arousal levels, an observation first made by Yerkes and Dodson in 1908. It is commonly proposed that the noradrenergic locus coeruleus system regulates these effects on performance through a widespread release of noradrenaline resulting in changes of cortical gain. This account, however, does not explain why performance decays with high arousal levels only in difficult, but not in simple tasks. Here, we present a mechanistic model that revisits the Yerkes-Dodson effect from a sensory perspective: a deep convolutional neural network augmented with a global gain mechanism reproduced the same interaction between arousal state and task difficulty in its performance. Investigating this model revealed that global gain states differentially modulated sensory information encoding across the processing hierarchy, which explained their differential effects on performance on simple versus difficult tasks. These findings offer a novel hierarchical sensory processing account of how, and why, arousal state affects task performance.

Connectomics of human electrophysiology

We present both a scientific overview and conceptual positions concerning the challenges and assets of electrophysiological measurements in the search for the nature and functions of the human connectome. We discuss how the field has been inspired by findings and approaches from functional magnetic resonance imaging (fMRI) and informed by a small number of significant multimodal empirical studies, which show that the canonical networks that are commonplace in fMRI are in fact rooted in electrophysiological processes. This review is also an opportunity to produce a brief, up-to-date critical survey of current data modalities and analytical methods available for deriving both static and dynamic connectomes from electrophysiology. We review hurdles that challenge the significance and impact of current electrophysiology connectome research. We then encourage the field to take a leap of faith and embrace the wealth of electrophysiological signals, despite their apparent, disconcerting complexity. Our position is that electrophysiology connectomics is poised to inform testable mechanistic models of information integration in hierarchical brain networks, constructed from observable oscillatory and aperiodic signal components and their polyrhythmic interactions.

From Shorter to Longer Timescales: Converging Integrated Information Theory (IIT) with the Temporo-Spatial Theory of Consciousness (TTC)

Time is a key element of consciousness as it includes multiple timescales from shorter to longer ones. This is reflected in our experience of various short-term phenomenal contents at discrete points in time as part of an ongoing, more continuous, and long-term ‘stream of consciousness.’ Can Integrated Information Theory (IIT) account for this multitude of timescales of consciousness? According to the theory, the relevant spatiotemporal scale for consciousness is the one in which the system reaches the maximum cause-effect power; IIT currently predicts that experience occurs on the order of short timescales, namely, between 100 and 300 ms (theta and alpha frequency range). This can well account for the integration of single inputs into a particular phenomenal content. However, such short timescales leave open the temporal relation of specific phenomenal contents to others during the course of the ongoing time, that is, the stream of consciousness. For that purpose, we converge the IIT with the Temporo-spatial Theory of Consciousness (TTC), which, assuming a multitude of different timescales, can take into view the temporal integration of specific phenomenal contents with other phenomenal contents over time. On the neuronal side, this is detailed by considering those neuronal mechanisms driving the non-additive interaction of pre-stimulus activity with the input resulting in stimulus-related activity. Due to their non-additive interaction, the single input is not only integrated with others in the short-term timescales of 100–300 ms (alpha and theta frequencies) (as predicted by IIT) but, at the same time, also virtually expanded in its temporal (and spatial) features; this is related to the longer timescales (delta and slower frequencies) that are carried over from pre-stimulus to stimulus-related activity. Such a non-additive pre-stimulus-input interaction amounts to temporo-spatial expansion as a key mechanism of TTC for the constitution of phenomenal contents including their embedding or nesting within the ongoing temporal dynamic, i.e., the stream of consciousness. In conclusion, we propose converging the short-term integration of inputs postulated in IIT (100–300 ms as in the alpha and theta frequency range) with the longer timescales (in delta and slower frequencies) of temporo-spatial expansion in TTC.

Temporo-spatial Theory of Consciousness (TTC) - Bridging the gap of neuronal activity and phenomenal states

Consciousness and its neural mechanisms remain a mystery. Current neuroscientific theories focus predominantly on the external input/stimulus and the associated stimulus-related activity during conscious contents. Despite all progress, we encounter two gaps: (i) a gap between spontaneous and stimulus-related activity; (ii) a gap between neuronal and phenomenal features. A novel, different, and unique approach, Temporo-spatial theory of consciousness (TTC) aims to bridge both gaps. The TTC focuses on the brain's spontaneous activity and how its spatial topography and temporal dynamic shape stimulus-related activity and resurface in the corresponding spatial and temporal features of consciousness, i.e., 'common currency'. The TTC introduces four temporo-spatial mechanisms: expansion, globalization, alignment, and nestedness. These are associated with distinct dimensions of consciousness including phenomenal content, access, form/structure, and level/state, respectively. Following up on the first introduction of the TTC in 2017, we review updates, further develop these temporo-spatial mechanisms, and postulate specific neurophenomenal hypotheses. We conclude that the TTC offers a viable approach for (i) linking spontaneous and stimulus-related activity in conscious states; (ii) determining specific neuronal and neurophenomenal mechanisms for the distinct dimensions of consciousness; (iii) an integrative and unifying framework of different neuroscientific theories of consciousness; and (iv) offers novel empirically grounded conceptual assumptions about the biological and ontological nature of consciousness and its relation to the brain.

Pupil diameter is not an accurate real-time readout of locus coeruleus activity

Pupil diameter is often treated as a noninvasive readout of activity in the locus coeruleus (LC). However, how accurately it can be used to index LC activity is not known. To address this question, we established a graded relationship between pupil size changes and LC spiking activity in mice, where pupil dilation increased monotonically with the number of LC spikes. However, this relationship exists with substantial variability such that pupil diameter can only be used to accurately predict a small fraction of LC activity on a moment-by-moment basis. In addition, pupil exhibited large session-to-session fluctuations in response to identical optical stimulation in the LC. The variations in the pupil–LC relationship were strongly correlated with decision bias-related behavioral variables. Together, our data show that substantial variability exists in an overall graded relationship between pupil diameter and LC activity, and further suggest that the pupil–LC relationship is dynamically modulated by brain states, supporting and extending our previous findings (Yang et al., 2021).

State-related neural influences on fMRI connectivity estimation

The spatiotemporal structure of functional magnetic resonance imaging (fMRI) signals has provided a valuable window into the network underpinnings of human brain function and dysfunction. Although some cross-regional temporal correlation patterns (functional connectivity; FC) exhibit a high degree of stability across individuals and species, there is growing acknowledgment that measures of FC can exhibit marked changes over a range of temporal scales. Further, FC can covary with experimental task demands and ongoing neural processes linked to arousal, consciousness and perception, cognitive and affective state, and brain-body interactions. The increased recognition that such interrelated neural processes modulate FC measurements has raised both challenges and new opportunities in using FC to investigate brain function. Here, we review recent advances in the quantification of neural effects that shape fMRI FC and discuss the broad implications of these findings in the design and analysis of fMRI studies. We also discuss how a more complete understanding of the neural factors that shape FC measurements can resolve apparent inconsistencies in the literature and lead to more interpretable conclusions from fMRI studies.

Is it time to put rest to rest?

The so-called resting state, in which participants lie quietly with no particular inputs or outputs, represented a paradigm shift from conventional task-based studies in human neuroimaging. Our foray into rest was fruitful from both a scientific and methodological perspective, but at this point, how much more can we learn from rest on its own? While rest still dominates in many subfields, data from tasks have empirically demonstrated benefits, as well as the potential to provide insights about the mind in addition to the brain. I argue that we can accelerate progress in human neuroscience by de-emphasizing rest in favor of more grounded experiments, including promising integrated designs that respect the prominence of self-generated activity while offering enhanced control and interpretability.

An altered balance of integrated and segregated brain activity is a marker of cognitive deficits following sleep deprivation

Sleep deprivation (SD) leads to impairments in cognitive function. Here, we tested the hypothesis that cognitive changes in the sleep-deprived brain can be explained by information processing within and between large-scale cortical networks. We acquired functional magnetic resonance imaging (fMRI) scans of 20 healthy volunteers during attention and executive tasks following a regular night of sleep, a night of SD, and a recovery nap containing nonrapid eye movement (NREM) sleep. Overall, SD was associated with increased cortex-wide functional integration, driven by a rise of integration within cortical networks. The ratio of within versus between network integration in the cortex increased further in the recovery nap, suggesting that prolonged wakefulness drives the cortex towards a state resembling sleep. This balance of integration and segregation in the sleep-deprived state was tightly associated with deficits in cognitive performance. This was a distinct and better marker of cognitive impairment than conventional indicators of homeostatic sleep pressure, as well as the pronounced thalamocortical connectivity changes that occurs towards falling asleep. Importantly, restoration of the balance between segregation and integration of cortical activity was also related to performance recovery after the nap, demonstrating a bidirectional effect. These results demonstrate that intra- and interindividual differences in cortical network integration and segregation during task performance may play a critical role in vulnerability to cognitive impairment in the sleep-deprived state.

Dissociable Components of Information Encoding in Human Perception

Brain decoding can predict visual perception from non-invasive electrophysiological data by combining information across multiple channels. However, decoding methods typically conflate the composite and distributed neural processes underlying perception that are together present in the signal, making it unclear what specific aspects of the neural computations involved in perception are reflected in this type of macroscale data. Using MEG data recorded while participants viewed a large number of naturalistic images, we analytically decomposed the brain signal into its oscillatory and non-oscillatory components, and used this decomposition to show that there are at least three dissociable stimulus-specific aspects to the brain data: a slow, non-oscillatory component, reflecting the temporally stable aspect of the stimulus representation; a global phase shift of the oscillation, reflecting the overall speed of processing of specific stimuli; and differential patterns of phase across channels, likely reflecting stimulus-specific computations. Further, we show that common cognitive interpretations of decoding analysis, in particular about how representations generalize across time, can benefit from acknowledging the multicomponent nature of the signal in the study of perception.

Comparing stimulus-evoked and spontaneous response of the face-selective multi-units in the human posterior fusiform gyrus

The stimulus-evoked neural response is a widely explored phenomenon. Conscious awareness is associated in many cases with the corresponding selective stimulus-evoked response. For example, conscious awareness of a face stimulus is associated with or accompanied by stimulus-evoked activity in the fusiform face area (FFA). In addition to the stimulus-evoked response, spontaneous (i.e. task-unrelated) activity in the brain is also abundant. Notably, spontaneous activity is considered unconscious. For example, spontaneous activity in the FFA is not associated with conscious awareness of a face. The question is: what is the difference at the neural level between stimulus-evoked activity in a case that this activity is associated with conscious awareness of some content (e.g. activity in the FFA in response to fully visible face stimuli) and spontaneous activity in that same region of the brain? To answer this question, in the present study, we had a rare opportunity to record two face-selective multi-units in the vicinity of the FFA in a human patient. We compared multi-unit face-selective task-evoked activity with spontaneous prestimulus and a resting-state activity. We found that when activity was examined over relatively long temporal windows (e.g. 100–200 ms), face-selective stimulus-evoked firing in the recorded multi-units was much higher than the spontaneous activity. In contrast, when activity was examined over relatively short windows, we found many cases of high firing rates within the spontaneous activity that were comparable to stimulus-evoked activity. Our results thus indicate that the sustained activity is what might differentiate between stimulus-evoked activity that is associated with conscious awareness and spontaneous activity.

Spectral signature and behavioral consequence of spontaneous shifts of pupil-linked arousal in human

Arousal levels perpetually rise and fall spontaneously. How markers of arousal—pupil size and frequency content of brain activity—relate to each other and influence behavior in humans is poorly understood. We simultaneously monitored magnetoencephalography and pupil in healthy volunteers at rest and during a visual perceptual decision-making task. Spontaneously varying pupil size correlates with power of brain activity in most frequency bands across large-scale resting state cortical networks. Pupil size recorded at prestimulus baseline correlates with subsequent shifts in detection bias (c) and sensitivity (d’). When dissociated from pupil-linked state, prestimulus spectral power of resting state networks still predicts perceptual behavior. Fast spontaneous pupil constriction and dilation correlate with large-scale brain activity as well but not perceptual behavior. Our results illuminate the relation between central and peripheral arousal markers and their respective roles in human perceptual decision-making.

Spontaneous perception: a framework for task-free, self-paced perception

Flipping through social media feeds, viewing exhibitions in a museum, or walking through the botanical gardens, people consistently choose to engage with and disengage from visual content. Yet, in most laboratory settings, the visual stimuli, their presentation duration, and the task at hand are all controlled by the researcher. Such settings largely overlook the spontaneous nature of human visual experience, in which perception takes place independently from specific task constraints and its time course is determined by the observer as a self-governing agent. Currently, much remains unknown about how spontaneous perceptual experiences unfold in the brain. Are all perceptual categories extracted during spontaneous perception? Does spontaneous perception inherently involve volition? Is spontaneous perception segmented into discrete episodes? How do different neural networks interact over time during spontaneous perception? These questions are imperative to understand our conscious visual experience in daily life. In this article we propose a framework for spontaneous perception. We first define spontaneous perception as a task-free and self-paced experience. We propose that spontaneous perception is guided by four organizing principles that grant it temporal and spatial structures. These principles include coarse-to-fine processing, continuity and segmentation, agency and volition, and associative processing. We provide key suggestions illustrating how these principles may interact with one another in guiding the multifaceted experience of spontaneous perception. We point to testable predictions derived from this framework, including (but not limited to) the roles of the default-mode network and slow cortical potentials in underlying spontaneous perception. We conclude by suggesting several outstanding questions for future research, extending the relevance of this framework to consciousness and spontaneous brain activity. In conclusion, the spontaneous perception framework proposed herein integrates components in human perception and cognition, which have been traditionally studied in isolation, and opens the door to understand how visual perception unfolds in its most natural context.

Prestimulus dynamics blend with the stimulus in neural variability quenching

Neural responses to the same stimulus show significant variability over trials, with this variability typically reduced (quenched) after a stimulus is presented. This trial-to-trial variability (TTV) has been much studied, however how this neural variability quenching is influenced by the ongoing dynamics of the prestimulus period is unknown. Utilizing a human intracranial stereo-electroencephalography (sEEG) data set, we investigate how prestimulus dynamics, as operationalized by standard deviation (SD), shapes poststimulus activity through trial-to-trial variability (TTV). We first observed greater poststimulus variability quenching in those real trials exhibiting high prestimulus variability as observed in all frequency bands. Next, we found that the relative effect of the stimulus was higher in the later (300-600ms) than the earlier (0-300ms) poststimulus period. Lastly, we replicate our findings in a separate EEG dataset and extend them by finding that trials with high prestimulus variability in the theta and alpha bands had faster reaction times. Together, our results demonstrate that stimulus-related activity, including its variability, is a blend of two factors: 1) the effects of the external stimulus itself, and 2) the effects of the ongoing dynamics spilling over from the prestimulus period - the state at stimulus onset - with the second dwarfing the influence of the first.

Cortical and subcortical signatures of conscious object recognition

The neural mechanisms underlying conscious recognition remain unclear, particularly the roles played by the prefrontal cortex, deactivated brain areas and subcortical regions. We investigated neural activity during conscious object recognition using 7 Tesla fMRI while human participants viewed object images presented at liminal contrasts. Here, we show both recognized and unrecognized images recruit widely distributed cortical and subcortical regions; however, recognized images elicit enhanced activation of visual, frontoparietal, and subcortical networks and stronger deactivation of the default-mode network. For recognized images, object category information can be decoded from all of the involved cortical networks but not from subcortical regions. Phase-scrambled images trigger strong involvement of inferior frontal junction, anterior cingulate cortex and default-mode network, implicating these regions in inferential processing under increased uncertainty. Our results indicate that content-specific activity in both activated and deactivated cortical networks and non-content-specific subcortical activity support conscious recognition.

Cortical and subcortical neural activity supporting conscious object recognition has not yet been well defined. Here, the authors describe these networks and show recognition-related category information can be decoded from widespread cortical activity but not subcortical activity.

fMRI-based detection of alertness predicts behavioral response variability

Levels of alertness are closely linked with human behavior and cognition. However, while functional magnetic resonance imaging (fMRI) allows for investigating whole-brain dynamics during behavior and task engagement, concurrent measures of alertness (such as EEG or pupillometry) are often unavailable. Here, we extract a continuous, time-resolved marker of alertness from fMRI data alone. We demonstrate that this fMRI alertness marker, calculated in a short pre-stimulus interval, captures trial-to-trial behavioral responses to incoming sensory stimuli. In addition, we find that the prediction of both EEG and behavioral responses during the task may be accomplished using only a small fraction of fMRI voxels. Furthermore, we observe that accounting for alertness appears to increase the statistical detection of task-activated brain areas. These findings have broad implications for augmenting a large body of existing datasets with information about ongoing arousal states, enriching fMRI studies of neural variability in health and disease.

Anterior insula regulates brain network transitions that gate conscious access

Conscious access to sensory information is likely gated at an intermediate site between primary sensory and transmodal association cortices, but the structure responsible remains unknown. We perform functional neuroimaging to determine the neural correlates of conscious access using a volitional mental imagery task, a report paradigm not confounded by motor behavior. Titrating propofol to loss of behavioral responsiveness in healthy volunteers creates dysfunction of the anterior insular cortex (AIC) in association with an impairment of dynamic transitions of default-mode and dorsal attention networks. Candidate subcortical regions mediating sensory gating or arousal (thalamus, basal forebrain) fail to show this association. The gating role of the AIC is consistent with findings in awake participants, whose conscious access is predicted by pre-stimulus AIC activity near perceptual threshold. These data support the hypothesis that AIC, situated at an intermediate position of the cortical hierarchy, regulates brain network transitions that gate conscious access.

In a human neuroimaging study, Huang et al. manipulate the level and content of consciousness using independent experimental protocols to demonstrate that the anterior insula, situated between unimodal and transmodal cortical areas along the brain’s functional hierarchy, serves as a gate for conscious access of sensory information.

Overcoming Rest-Task Divide-Abnormal Temporospatial Dynamics and Its Cognition in Schizophrenia

Schizophrenia is a complex psychiatric disorder exhibiting alterations in spontaneous and task-related cerebral activity whose relation (termed "state dependence") remains unclear. For unraveling their relationship, we review recent electroencephalographic (and a few functional magnetic resonance imaging) studies in schizophrenia that assess and compare both rest/prestimulus and task states, ie, rest/prestimulus-task modulation. Results report reduced neural differentiation of task-related activity from rest/prestimulus activity across different regions, neural measures, cognitive domains, and imaging modalities. Together, the findings show reduced rest/prestimulus-task modulation, which is mediated by abnormal temporospatial dynamics of the spontaneous activity. Abnormal temporospatial dynamics, in turn, may lead to abnormal prediction, ie, predictive coding, which mediates cognitive changes and psychopathological symptoms, including confusion of internally and externally oriented cognition. In conclusion, reduced rest/prestimulus-task modulation in schizophrenia provides novel insight into the neuronal mechanisms that connect task-related changes to cognitive abnormalities and psychopathological symptoms.

From receptors to the brain: Psychophysical clues to taste physiology

To understand human taste requires not only physiological studies ranging from receptor mechanisms to brain circuitry, but also psychophysical studies that quantitatively describe the perceptual output of the system. As obvious as this requirement is, differences in research approaches, methodologies, and objectives complicate the ability to meet it. Discussed here is an example of how the discovery two decades ago of a perceptual taste illusion (thermal taste) has led to physiological and psychophysical research on both peripheral and central mechanisms of taste, including most recently a psychophysical study of the heat sensitivity of the human sweet taste receptor TAS1R2/T1R3, and an fMRI study of a possible central gain mechanism that may underlie, in part, differences in human taste sensitivity. In addition to the new data and hypotheses these studies have generated, they illustrate instances of research on taste motivated by evidence derived from different approaches and levels of analysis.

Behavior needs neural variability

Human and non-human animal behavior is highly malleable and adapts successfully to internal and external demands. Such behavioral success stands in striking contrast to the apparent instability in neural activity (i.e., variability) from which it arises. Here, we summon the considerable evidence across scales, species, and imaging modalities that neural variability represents a key, undervalued dimension for understanding brain-behavior relationships at inter- and intra-individual levels. We believe that only by incorporating a specific focus on variability will the neural foundation of behavior be comprehensively understood.

Task-related activity in human visual cortex

The brain exhibits widespread endogenous responses in the absence of visual stimuli, even at the earliest stages of visual cortical processing. Such responses have been studied in monkeys using optical imaging with a limited field of view over visual cortex. Here, we used functional MRI (fMRI) in human participants to study the link between arousal and endogenous responses in visual cortex. The response that we observed was tightly entrained to task timing, was spatially extensive, and was independent of visual stimulation. We found that this response follows dynamics similar to that of pupil size and heart rate, suggesting that task-related activity is related to arousal. Finally, we found that higher reward increased response amplitude while decreasing its trial-to-trial variability (i.e., the noise). Computational simulations suggest that increased temporal precision underlies both of these observations. Our findings are consistent with optical imaging studies in monkeys and support the notion that arousal increases precision of neural activity.

The brain exhibits widespread endogenous responses in the absence of visual stimuli, even at the earliest stages of visual cortical processing. This fMRI study characterizes a widespread hemodynamic response in early visual cortex that is not related to visual input but instead reflects a participant’s engagement in a task, is modulated by expected monetary reward, and may reflect neural quenching.

Bring the Noise: Reconceptualizing Spontaneous Neural Activity

Definitions of what constitutes the 'signal of interest' in neuroscience can be controversial, due in part to continuously evolving notions regarding the significance of spontaneous neural activity. This review highlights how the challenge of separating brain signal from noise has led to new conceptualizations of brain functional organization at both the micro- and macroscopic level. Recent debates in the functional neuroimaging community surrounding artifact removal processes have revived earlier discussions surrounding how to appropriately isolate and measure neuronal signals against a background of noise from various sources. Insights from electrophysiological studies and computational modeling can inform current theory and data analytic practices in human functional neuroimaging, given that signal and noise may be inextricably linked in the brain.

Endogenous activity modulates stimulus and circuit-specific neural tuning and predicts perceptual behavior

Perception reflects not only sensory inputs, but also the endogenous state when these inputs enter the brain. Prior studies show that endogenous neural states influence stimulus processing through non-specific, global mechanisms, such as spontaneous fluctuations of arousal. It is unclear if endogenous activity influences circuit and stimulus-specific processing and behavior as well. Here we use intracranial recordings from 30 pre-surgical epilepsy patients to show that patterns of endogenous activity are related to the strength of trial-by-trial neural tuning in different visual category-selective neural circuits. The same aspects of the endogenous activity that relate to tuning in a particular neural circuit also correlate to behavioral reaction times only for stimuli from the category that circuit is selective for. These results suggest that endogenous activity can modulate neural tuning and influence behavior in a circuit- and stimulus-specific manner, reflecting a potential mechanism by which endogenous neural states facilitate and bias perception.

Neural signs and mechanisms of consciousness: Is there a potential convergence of theories of consciousness in sight?

Various theories for the neural basis of consciousness have been proposed, suggesting a diversity of neural signs and mechanisms. We ask to what extent this diversity is real, or whether many theories share the same basic ideas with a potential for convergence towards a more unified theory of the neural basis of consciousness. For that purpose, we review and compare the various neural signs, measures, and mechanisms proposed in the different theories. We demonstrate that different theories focus on neural signs and measures of distinct aspects of neural activity including stimulus-related, prestimulus, and resting state activity as well as on distinct features of consciousness. Therefore, the various mechanisms proposed in the different theories may, in part, complement each other. Together, we provide insight into the shared basis and convergences (and, in part, discrepancies) of the different theories of consciousness. We conclude that the different theories concern distinct aspects of both neural activity and consciousness which, as we suppose, may be integrated and nested within the brain's overall temporo-spatial dynamics.

The temporal evolution of pre-stimulus slow cortical potentials is associated with an upcoming stimulus' access to visual consciousness

Slow cortical potentials (SCPs) have been proposed to be essential for the formation of conscious experience. To examine their temporal characteristics, we recorded electroencephalography during a visual backward-masking task, which required participants to localize the missing part of a target stimulus. A subsequent confidence rating was used as a proxy for the target's access to consciousness. Event-related potentials (ERPs) of all correct trials were determined relative to a brief period immediately before the target and then compared among consciousness levels. In an interval ranging from 2 s prior to target presentation up to this period, a negative relationship between slowly fluctuating ERP values and the level of consciousness became evident. After target presentation, high conscious awareness was characterized by an enhanced visual awareness negativity, an increased P3 component, and associated positive SCPs. Together, these findings add new evidence to the proposed role of SCPs in the emergence of visual consciousness.

Neuromodulation of Brain State and Behavior

Neural activity and behavior are both notoriously variable, with responses differing widely between repeated presentation of identical stimuli or trials. Recent results in humans and animals reveal that these variations are not random in their nature, but may in fact be due in large part to rapid shifts in neural, cognitive, and behavioral states. Here we review recent advances in the understanding of rapid variations in the waking state, how variations are generated, and how they modulate neural and behavioral responses in both mice and humans. We propose that the brain has an identifiable set of states through which it wanders continuously in a nonrandom fashion, owing to the activity of both ascending modulatory and fast-acting corticocortical and subcortical-cortical neural pathways. These state variations provide the backdrop upon which the brain operates, and understanding them is critical to making progress in revealing the neural mechanisms underlying cognition and behavior.