Discrete and continuous mechanisms of temporal selection in rapid visual streams

Anticipatory and evoked visual cortical dynamics of voluntary temporal attention

We can often anticipate the precise moment when a stimulus will be relevant for our behavioral goals. Voluntary temporal attention, the prioritization of sensory information at task-relevant time points, enhances visual perception. However, the neural mechanisms of voluntary temporal attention have not been isolated from those of temporal expectation, which reflects timing predictability rather than relevance. Here we use time-resolved steady-state visual evoked responses (SSVER) to investigate how temporal attention dynamically modulates visual activity when temporal expectation is controlled. We recorded magnetoencephalography while participants directed temporal attention to one of two sequential grating targets with predictable timing. Meanwhile, a co-localized SSVER probe continuously tracked visual cortical modulations both before and after the target stimuli. We find that in the pre-target period, the SSVER gradually ramps up as the targets approach, reflecting temporal expectation. Furthermore, we find a low-frequency modulation of the SSVER, which shifts approximately half a cycle in phase according to which target is attended. In the post-target period, temporal attention to the first target transiently modulates the SSVER shortly after target onset. Thus, temporal attention dynamically modulates visual cortical responses via both periodic pre-target and transient post-target mechanisms to prioritize sensory information at precise moments.

Decoding cognition in neurodevelopmental, psychiatric and neurological conditions with multivariate pattern analysis of EEG data

Multivariate pattern analysis (MVPA) of electroencephalographic (EEG) data represents a revolutionary approach to investigate how the brain encodes information. By considering complex interactions among spatio-temporal features at the individual level, MVPA overcomes the limitations of univariate techniques, which often fail to account for the significant inter- and intra-individual neural variability. This is particularly relevant when studying clinical populations, and therefore MVPA of EEG data has recently started to be employed as a tool to study cognition in brain disorders. Here, we review the insights offered by this methodology in the study of anomalous patterns of neural activity in conditions such as autism, ADHD, schizophrenia, dyslexia, neurological and neurodegenerative disorders, within different cognitive domains (perception, attention, memory, consciousness). Despite potential drawbacks that should be attentively addressed, these studies reveal a peculiar sensitivity of MVPA in unveiling dysfunctional and compensatory neurocognitive dynamics of information processing, which often remain blind to traditional univariate approaches. Such higher sensitivity in characterizing individual neurocognitive profiles can provide unique opportunities to optimise assessment and promote personalised interventions.

An early effect of the parafoveal preview on post-saccadic processing of English words

A key aspect of efficient visual processing is to use current and previous information to make predictions about what we will see next. In natural viewing, and when looking at words, there is typically an indication of forthcoming visual information from extrafoveal areas of the visual field before we make an eye movement to an object or word of interest. This "preview effect" has been studied for many years in the word reading literature and, more recently, in object perception. Here, we integrated methods from word recognition and object perception to investigate the timing of the preview on neural measures of word recognition. Through a combined use of EEG and eye-tracking, a group of multilingual participants took part in a gaze-contingent, single-shot saccade experiment in which words appeared in their parafoveal visual field. In valid preview trials, the same word was presented during the preview and after the saccade, while in the invalid condition, the saccade target was a number string that turned into a word during the saccade. As hypothesized, the valid preview greatly reduced the fixation-related evoked response. Interestingly, multivariate decoding analyses revealed much earlier preview effects than previously reported for words, and individual decoding performance correlated with participant reading scores. These results demonstrate that a parafoveal preview can influence relatively early aspects of post-saccadic word processing and help to resolve some discrepancies between the word and object literatures.

Multiband task related components enhance rapid cognition decoding for both small and similar objects

The cortically-coupled target recognition system based on rapid serial visual presentation (RSVP) has a wide range of applications in brain computer interface (BCI) fields such as medical and military. However, in the complex natural environment backgrounds, the identification of event-related potentials (ERP) of both small and similar objects that are quickly presented is a research challenge. Therefore, we designed corresponding experimental paradigms and proposed a multi-band task related components matching (MTRCM) method to improve the rapid cognitive decoding of both small and similar objects. We compared the areas under the receiver operating characteristic curve (AUC) between MTRCM and other 9 methods under different numbers of training sample using RSVP-ERP data from 50 subjects. The results showed that MTRCM maintained an overall superiority and achieved the highest average AUC (0.6562 ± 0.0091). We also optimized the frequency band and the time parameters of the method. The verification on public data sets further showed the necessity of designing MTRCM method. The MTRCM method provides a new approach for neural decoding of both small and similar RSVP objects, which is conducive to promote the further development of RSVP-BCI.

A measure centrality index for systematic empirical comparison of consciousness theories

Consciousness science is marred by disparate constructs and methodologies, making it challenging to systematically compare theories. This foundational crisis casts doubts on the scientific character of the field itself. Addressing it, we propose a framework for systematically comparing consciousness theories by introducing a novel inter-theory classification interface, the Measure Centrality Index (MCI). Recognizing its gradient distribution, the MCI assesses the degree of importance a specific empirical measure has for a given consciousness theory. We apply the MCI to probe how the empirical measures of the Global Neuronal Workspace Theory (GNW), Integrated Information Theory (IIT), and Temporospatial Theory of Consciousness (TTC) would fare within the context of the other two. We demonstrate that direct comparison of IIT, GNW, and TTC is meaningful and valid for some measures like Lempel-Ziv Complexity (LZC), Autocorrelation Window (ACW), and possibly Mutual Information (MI). In contrast, it is problematic for others like the anatomical and physiological neural correlates of consciousness (NCC) due to their MCI-based differential weightings within the structure of the theories. In sum, we introduce and provide proof-of-principle of a novel systematic method for direct inter-theory empirical comparisons, thereby addressing isolated evolution of theories and confirmatory bias issues in the state-of-the-art neuroscience of consciousness.

An integrative, multiscale view on neural theories of consciousness

How is conscious experience related to material brain processes? A variety of theories aiming to answer this age-old question have emerged from the recent surge in consciousness research, and some are now hotly debated. Although most researchers have so far focused on the development and validation of their preferred theory in relative isolation, this article, written by a group of scientists representing different theories, takes an alternative approach. Noting that various theories often try to explain different aspects or mechanistic levels of consciousness, we argue that the theories do not necessarily contradict each other. Instead, several of them may converge on fundamental neuronal mechanisms and be partly compatible and complementary, so that multiple theories can simultaneously contribute to our understanding. Here, we consider unifying, integration-oriented approaches that have so far been largely neglected, seeking to combine valuable elements from various theories.

A Multivariate analysis on evoked components of Chinese semantic congruity: an OP-MEG study with EEG

The application of wearable magnetoencephalography using optically-pumped magnetometers has drawn extensive attention in the field of neuroscience. Electroencephalogram system can cover the whole head and reflect the overall activity of a large number of neurons. The efficacy of optically-pumped magnetometer in detecting event-related components can be validated through electroencephalogram results. Multivariate pattern analysis is capable of tracking the evolution of neurocognitive processes over time. In this paper, we adopted a classical Chinese semantic congruity paradigm and separately collected electroencephalogram and optically-pumped magnetometer signals. Then, we verified the consistency of optically-pumped magnetometer and electroencephalogram in detecting N400 using mutual information index. Multivariate pattern analysis revealed the difference in decoding performance of these two modalities, which can be further validated by dynamic/stable coding analysis on the temporal generalization matrix. The results from searchlight analysis provided a neural basis for this dissimilarity at the magnetoencephalography source level and the electroencephalogram sensor level. This study opens a new avenue for investigating the brain's coding patterns using wearable magnetoencephalography and reveals the differences in sensitivity between the two modalities in reflecting neuron representation patterns.

Prolonged neural encoding of visual information in autism

Autism spectrum disorder (ASD) is associated with a hyper-focused visual attentional style, impacting higher-order social and affective domains. The understanding of such peculiarity can benefit from the use of multivariate pattern analysis (MVPA) of high-resolution electroencephalography (EEG) data, which has proved to be a powerful technique to investigate the hidden neural dynamics orchestrating sensory and cognitive processes. Here, we recorded EEG in typically developing (TD) children and in children with ASD during a visuo-spatial attentional task where attention was exogenously captured by a small (zoom-in) or large (zoom-out) cue in the visual field before the appearance of a target at different eccentricities. MVPA was performed both in the cue-locked period, to reveal potential differences in the modulation of the attentional focus, and in the target-locked period, to reveal potential cascade effects on stimulus processing. Cue-locked MVPA revealed that while in the TD group the pattern of neural activity contained information about the cue mainly before the target appearance, the ASD group showed a temporally sustained and topographically diffuse significant decoding of the cue neural response even after the target onset, suggesting a delayed extinction of cue-related neural activity. Crucially, this delayed extinction positively correlated with behavioral measures of attentional hyperfocusing. Results of target-locked MVPA were coherent with a hyper-focused attentional profile, highlighting an earlier and stronger decoding of target neural responses in small cue trials in the ASD group. The present findings document a spatially and temporally overrepresented encoding of visual information in ASD, which can constitute one of the main reasons behind their peculiar cognitive style.

The role of distractors in rapid serial visual presentation reveals the mechanism of attentional blink by EEG-based univariate and multivariate analyses

Attentional blink pertains to the performance of participants with a severe decline in identifying the second target presented after the first target reported correctly within 200-500 ms in a rapid serial visual presentation. The current study was conducted to investigate the neural mechanism of the effect of the distractor (D1) that immediately follows first target to attentional blink by altering whether D1 was substituted with a blank with electroencephalography recording. The results showed that D1 interfered with the attentional enhancement and working memory encoding in both single-target rapid serial visual presentation task and dual-target rapid serial visual presentation task, which were mainly manifested in delayed and attenuated P3a and diminished P3b of first target. Single-trial analysis indicated that first target and second target will compete with each other for working memory encoding resources in short lag, but not in the long lag. In addition, D1 interfered with the working memory encoding of second target under short lag rather than long lag in the dual-target rapid serial visual presentation task. These results suggested that attentional blink can be attributed to the limited working memory encoding resource, whereas the amount of available resources is subject to modulation by attention. The D1 hinders the attention enhancement of first target, thereby exacerbating attentional blink.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Analysis and Classification of Event-Related Potentials During Image Observation

In the field of cognitive neuroscience, researchers have conducted extensive studies on object categorization using Event-Related Potential (ERP) analysis, specifically by analyzing electroencephalographic (EEG) response signals triggered by visual stimuli. The most common approach for visual ERP analysis is to use a low presentation rate of images and an active task where participants actively discriminate between target and non-target images. However, researchers are also interested in understanding how the human brain processes visual information in real-world scenarios. To simulate real-life object recognition, this study proposes an analysis pipeline of visual ERPs evoked by images presented in a Rapid Serial Visual Presentation (RSVP) paradigm. Such an approach allows for the investigation of recurrent patterns of visual ERP signals across specific categories and subjects. The pipeline includes segmentation of the EEGs in epochs, and the use of the resulting features as inputs for Support Vector Machine (SVM) classification. Results demonstrate common ERP patterns across the selected categories and the ability to obtain discriminant information from single visual stimuli presented in the RSVP paradigm.

Mechanisms of human dynamic object recognition revealed by sequential deep neural networks

Humans can quickly recognize objects in a dynamically changing world. This ability is showcased by the fact that observers succeed at recognizing objects in rapidly changing image sequences, at up to 13 ms/image. To date, the mechanisms that govern dynamic object recognition remain poorly understood. Here, we developed deep learning models for dynamic recognition and compared different computational mechanisms, contrasting feedforward and recurrent, single-image and sequential processing as well as different forms of adaptation. We found that only models that integrate images sequentially via lateral recurrence mirrored human performance (N = 36) and were predictive of trial-by-trial responses across image durations (13-80 ms/image). Importantly, models with sequential lateral-recurrent integration also captured how human performance changes as a function of image presentation durations, with models processing images for a few time steps capturing human object recognition at shorter presentation durations and models processing images for more time steps capturing human object recognition at longer presentation durations. Furthermore, augmenting such a recurrent model with adaptation markedly improved dynamic recognition performance and accelerated its representational dynamics, thereby predicting human trial-by-trial responses using fewer processing resources. Together, these findings provide new insights into the mechanisms rendering object recognition so fast and effective in a dynamic visual world.

Decoding of the neural representation of the visual RGB color model

RGB color is a basic visual feature. Here we use machine learning and visual evoked potential (VEP) of electroencephalogram (EEG) data to investigate the decoding features of the time courses and space location that extract it, and whether they depend on a common brain cortex channel. We show that RGB color information can be decoded from EEG data and, with the task-irrelevant paradigm, features can be decoded across fast changes in VEP stimuli. These results are consistent with the theory of both event-related potential (ERP) and P300 mechanisms. The latency on time course is shorter and more temporally precise for RGB color stimuli than P300, a result that does not depend on a task-relevant paradigm, suggesting that RGB color is an updating signal that separates visual events. Meanwhile, distribution features are evident for the brain cortex of EEG signal, providing a space correlate of RGB color in classification accuracy and channel location. Finally, space decoding of RGB color depends on the channel classification accuracy and location obtained through training and testing EEG data. The result is consistent with channel power value distribution discharged by both VEP and electrophysiological stimuli mechanisms.

The temporal cost of deploying attention limits accurate target identification in rapid serial visual presentation

Lag-1 sparing is a common exception to the attentional blink, where a target presented directly after T1 can be identified and reported accurately. Prior work has proposed potential mechanisms for lag 1 sparing, including the boost and bounce model and the attentional gating model. Here, we apply a rapid serial visual presentation task to investigate the temporal limitations of lag 1 sparing by testing three distinct hypotheses. We found that endogenous engagement of attention to T2 requires between 50 and 100 ms. Critically, faster presentation rates yielded lower T2 performance, whereas decreased image duration did not impair T2 detection and report. These observations were reinforced by subsequent experiments controlling for short-term learning and capacity-dependent visual processing effects. Thus, lag-1 sparing was limited by the intrinsic dynamics of attentional boost engagement rather than by earlier perceptual bottlenecks such as insufficient exposure to images in the stimulus stream or visual processing capacity limitations. Taken together, these findings support the boost and bounce theory over earlier models that focus only on attentional gating or visual short-term memory storage, informing our understanding of how the human visual system deploys attention under challenging temporal constraints.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

The channel capacity of multilevel linguistic features constrains speech comprehension

Humans are expert at processing speech but how this feat is accomplished remains a major question in cognitive neuroscience. Capitalizing on the concept of channel capacity, we developed a unified measurement framework to investigate the respective influence of seven acoustic and linguistic features on speech comprehension, encompassing acoustic, sub-lexical, lexical and supra-lexical levels of description. We show that comprehension is independently impacted by all these features, but at varying degrees and with a clear dominance of the syllabic rate. Comparing comprehension of French words and sentences further reveals that when supra-lexical contextual information is present, the impact of all other features is dramatically reduced. Finally, we estimated the channel capacity associated with each linguistic feature and compared them with their generic distribution in natural speech. Our data reveal that while acoustic modulation, syllabic and phonemic rates unfold respectively at 5, 5, and 12 Hz in natural speech, they are associated with independent processing bottlenecks whose channel capacity are of 15, 15 and 35 Hz, respectively, as suggested by neurophysiological theories. They moreover point towards supra-lexical contextual information as the feature limiting the flow of natural speech. Overall, this study reveals how multilevel linguistic features constrain speech comprehension.

Decoding perceptual awareness across the brain with a no-report fMRI masking paradigm

Does perceptual awareness arise within the sensory regions of the brain or within higher-level regions (e.g., the frontal lobe)? To answer this question, researchers traditionally compare neural activity when observers report being aware versus being unaware of a stimulus. However, it is unclear whether the resulting activations are associated with the conscious perception of the stimulus or the post-perceptual processes associated with reporting that stimulus. To address this limitation, we used both report and no-report conditions in a visual masking paradigm while participants were scanned using functional MRI (fMRI). We found that the overall univariate response to visible stimuli in the frontal lobe was robust in the report condition but disappeared in the no-report condition. However, using multivariate patterns, we could still decode in both conditions whether a stimulus reached conscious awareness across the brain, including in the frontal lobe. These results help reconcile key discrepancies in the recent literature and provide a path forward for identifying the neural mechanisms associated with perceptual awareness.

Believing and social interactions: effects on bodily expressions and personal narratives

The processes of believing integrate external perceptual information from the environment with internal emotional states and prior experience to generate probabilistic neural representations of events, i.e., beliefs. As these neural representations manifest mostly below the level of a person's conscious awareness, they may inadvertently affect the spontaneous person's bodily expressions and prospective behavior. By yet to be understood mechanisms people can become aware of these representations and reflect upon them. Typically, people can communicate the content of their beliefs as personal statements and can summarize the narratives of others to themselves or to other people. Here, we describe that social interactions may benefit from the consistency between a person's bodily expressions and verbal statements because the person appears authentic and ultimately trustworthy. The transmission of narratives can thus lay the groundwork for social cooperation within and between groups and, ultimately, between communities and nations. Conversely, a discrepancy between bodily expressions and narratives may cause distrust in the addressee(s) and eventually may destroy social bonds.

Early and late electrophysiological correlates of gradual perceptual awareness in- and outside the Attentional Blink window

There is an ongoing debate on the neural correlates of consciousness (NCC) in the attentional blink (AB). Theoretical accounts propose that NCC during the attentional blink occur late in the processing hierarchy and that this quality is specific to the AB. We investigated this question by recording event-related potentials during an AB experiment with faces as T2. We analyzed ERPs to T2 stimuli inside (short lag) and outside (long lag) the AB window after carefully calibrating T2 stimuli to ensure equal visibility ratings across lags. We found that the N170, the visual awareness negativity (VAN), and the P3b showed an increased amplitude for seen compared to unseen face stimuli regardless of stimulus lag and that all these components scale linearly with subjective visibility. These findings suggest similar early and late mechanisms of graded perceptual awareness within and outside the AB across perceptual (N170, VAN) and post-perceptual (P3b) processing stages.

The nature of neural object representations during dynamic occlusion

Objects disappearing briefly from sight due to occlusion is an inevitable occurrence in everyday life. Yet we generally have a strong experience that occluded objects continue to exist, despite the fact that they objectively disappear. This indicates that neural object representations must be maintained during dynamic occlusion. However, it is unclear what the nature of such representation is and in particular whether it is perception-like or more abstract, for example, reflecting limited features such as position or movement direction only. In this study, we address this question by examining how different object features such as object shape, luminance, and position are represented in the brain when a moving object is dynamically occluded. We apply multivariate decoding methods to Magnetoencephalography (MEG) data to track how object representations unfold over time. Our methods allow us to contrast the representations of multiple object features during occlusion and enable us to compare the neural responses evoked by visible and occluded objects. The results show that object position information is represented during occlusion to a limited extent while object identity features are not maintained through the period of occlusion. Together, this suggests that the nature of object representations during dynamic occlusion is different from visual representations during perception.

What triggers explicit awareness in implicit sequence learning? Implications from theories of consciousness

This article aims to continue the debate on how explicit, conscious knowledge can arise in an implicit learning situation. We review hitherto existing theoretical views and evaluate their compatibility with two current, successful scientific concepts of consciousness: The Global Workspace Theory and Higher-Order Thought Theories. In this context, we introduce the Unexpected Event Hypothesis (Frensch et al., Attention and implicit learning, John Benjamins Publishing Company, 2003) in an elaborated form and discuss its advantage in explaining the emergence of conscious knowledge in an implicit learning situation.

The time-course of feature-based attention effects dissociated from temporal expectation and target-related processes

Selective attention prioritises relevant information amongst competing sensory input. Time-resolved electrophysiological studies have shown stronger representation of attended compared to unattended stimuli, which has been interpreted as an effect of attention on information coding. However, because attention is often manipulated by making only the attended stimulus a target to be remembered and/or responded to, many reported attention effects have been confounded with target-related processes such as visual short-term memory or decision-making. In addition, attention effects could be influenced by temporal expectation about when something is likely to happen. The aim of this study was to investigate the dynamic effect of attention on visual processing using multivariate pattern analysis of electroencephalography (EEG) data, while (1) controlling for target-related confounds, and (2) directly investigating the influence of temporal expectation. Participants viewed rapid sequences of overlaid oriented grating pairs while detecting a "target" grating of a particular orientation. We manipulated attention, one grating was attended and the other ignored (cued by colour), and temporal expectation, with stimulus onset timing either predictable or not. We controlled for target-related processing confounds by only analysing non-target trials. Both attended and ignored gratings were initially coded equally in the pattern of responses across EEG sensors. An effect of attention, with preferential coding of the attended stimulus, emerged approximately 230 ms after stimulus onset. This attention effect occurred even when controlling for target-related processing confounds, and regardless of stimulus onset expectation. These results provide insight into the effect of feature-based attention on the dynamic processing of competing visual information.

Decoding rapidly presented visual stimuli from prefrontal ensembles without report nor post-perceptual processing

The role of the primate prefrontal cortex (PFC) in conscious perception is debated. The global neuronal workspace theory of consciousness predicts that PFC neurons should contain a detailed code of the current conscious contents. Previous research showed that PFC is indeed activated in paradigms of conscious visual perception, including no-report paradigms where no voluntary behavioral report of the percept is given, thus avoiding a conflation of signals related to visual consciousness with signals related to the report. Still, it has been argued that prefrontal modulation could reflect post-perceptual processes that may be present even in the absence of report, such as thinking about the perceived stimulus, therefore reflecting a consequence rather than a direct correlate of conscious experience. Here, we investigate these issues by recording neuronal ensemble activity from the macaque ventrolateral PFC during briefly presented visual stimuli, either in isolated trials in which stimuli were clearly perceived or in sequences of rapid serial visual presentation (RSVP) in which perception and post-perceptual processing were challenged. We report that the identity of each stimulus could be decoded from PFC population activity even in the RSVP condition. The first visual signals could be detected at 60 ms after stimulus onset and information was maximal at 150 ms. However, in the RSVP condition, 200 ms after the onset of a stimulus, the decoding accuracy quickly dropped to chance level and the next stimulus started to be decodable. Interestingly, decoding in the ventrolateral PFC was stronger compared to posterior parietal cortex for both isolated and RSVP stimuli. These results indicate that neuronal populations in the macaque PFC reliably encode visual stimuli even under conditions that have been shown to challenge conscious perception and/or substantially reduce the probability of post-perceptual processing in humans. We discuss whether the observed activation reflects conscious access, phenomenal consciousness, or merely a preconscious bottom-up wave.

Temporal dynamics of the neural representation of hue and luminance polarity

Hue and luminance contrast are basic visual features. Here we use multivariate analyses of magnetoencephalography data to investigate the timing of the neural computations that extract them, and whether they depend on common neural circuits. We show that hue and luminance-contrast polarity can be decoded from MEG data and, with lower accuracy, both features can be decoded across changes in the other feature. These results are consistent with the existence of both common and separable neural mechanisms. The decoding time course is earlier and more temporally precise for luminance polarity than hue, a result that does not depend on task, suggesting that luminance contrast is an updating signal that separates visual events. Meanwhile, cross-temporal generalization is slightly greater for representations of hue compared to luminance polarity, providing a neural correlate of the preeminence of hue in perceptual grouping and memory. Finally, decoding of luminance polarity varies depending on the hues used to obtain training and testing data. The pattern of results is consistent with observations that luminance contrast is mediated by both L-M and S cone sub-cortical mechanisms.

The interplay between unexpected events and behavior in the development of explicit knowledge in implicit sequence learning

Some studies in implicit learning investigate the mechanisms by which implicitly acquired knowledge (e.g., learning a sequence of responses) becomes consciously aware. It has been suggested that unexpected changes in the own behavior can trigger search processes, of which the outcome then becomes aware. A consistent empirical finding is that participants who develop explicit knowledge show a sudden decrease in reaction times, when responding to sequential events. This so called RT-drop might indicate the point of time when explicit knowledge occurs. We investigated whether an RT-drop is a precursor for the development of explicit knowledge or the consequence of explicit knowledge. To answer this question, we manipulated in a serial reaction time task the timing of long and short stimulus-onset asynchronies (SOA). For some participants, the different SOAs were presented in blocks of either long or short SOAs, while for others, the SOAs changed randomly. We expected the participants who were given a blocked presentation to express an RT-drop because of the predictable timing. In contrast, randomly changing SOAs should hamper the expression of an RT-drop. We found that more participants in the blocked-SOA condition than in the random-SOA condition showed an RT-drop. Furthermore, the amount of explicit knowledge did not differ between the two conditions. The findings suggest that the RT-drop does not seem to be a presupposition to develop explicit knowledge. Rather, it seems that the RT-drop indicates a behavioral strategy shift as a consequence of explicit knowledge.

Mining Temporal Dynamics With Support Vector Machine for Predicting the Neural Fate of Target in Attentional Blink

Our brains do not mechanically process incoming stimuli; in contrast, the physiological state of the brain preceding stimuli has substantial consequences for subsequent behavior and neural processing. Although previous studies have acknowledged the importance of this top-down process, it was only recently that a growing interest was gained in exploring the underlying neural mechanism quantitatively. By utilizing the attentional blink (AB) effect, this study is aimed to identify the neural mechanism of brain states preceding T2 and predict its behavioral performance. Interarea phase synchronization and its role in prediction were explored using the phase-locking value and support vector machine classifiers. Our results showed that the phase coupling in alpha and beta frequency bands pre-T1 and during the T1-T2 interval could predict the detection of T2 in lag 3 with high accuracy. These findings indicated the important role of brain state before stimuli appear in predicting the behavioral performance in AB, thus, supporting the attention control theories.

Categorizing objects from MEG signals using EEGNet

Magnetoencephalography (MEG) signals have demonstrated their practical application to reading human minds. Current neural decoding studies have made great progress to build subject-wise decoding models to extract and discriminate the temporal/spatial features in neural signals. In this paper, we used a compact convolutional neural network-EEGNet-to build a common decoder across subjects, which deciphered the categories of objects (faces, tools, animals, and scenes) from MEG data. This study investigated the influence of the spatiotemporal structure of MEG on EEGNet's classification performance. Furthermore, the EEGNet replaced its convolution layers with two sets of parallel convolution structures to extract the spatial and temporal features simultaneously. Our results showed that the organization of MEG data fed into the EEGNet has an effect on EEGNet classification accuracy, and the parallel convolution structures in EEGNet are beneficial to extracting and fusing spatial and temporal MEG features. The classification accuracy demonstrated that the EEGNet succeeds in building the common decoder model across subjects, and outperforms several state-of-the-art feature fusing methods.

The Human Brain Encodes a Chronicle of Visual Events at Each Instant of Time Through the Multiplexing of Traveling Waves

The human brain continuously processes streams of visual input. Yet, a single image typically triggers neural responses that extend beyond 1s. To understand how the brain encodes and maintains successive images, we analyzed with electroencephalography the brain activity of human subjects while they watched ∼5000 visual stimuli presented in fast sequences. First, we confirm that each stimulus can be decoded from brain activity for ∼1s, and we demonstrate that the brain simultaneously represents multiple images at each time instant. Second, we source localize the corresponding brain responses in the expected visual hierarchy and show that distinct brain regions represent, at each time instant, different snapshots of past stimulations. Third, we propose a simple framework to further characterize the dynamical system of these traveling waves. Our results show that a chain of neural circuits, which each consist of (1) a hidden maintenance mechanism and (2) an observable update mechanism, accounts for the dynamics of macroscopic brain representations elicited by visual sequences. Together, these results detail a simple architecture explaining how successive visual events and their respective timings can be simultaneously represented in the brain.SIGNIFICANCE STATEMENT Our retinas are continuously bombarded with a rich flux of visual input. Yet, how our brain continuously processes such visual streams is a major challenge to neuroscience. Here, we developed techniques to decode and track, from human brain activity, multiple images flashed in rapid succession. Our results show that the brain simultaneously represents multiple successive images at each time instant by multiplexing them along a neural cascade. Dynamical modeling shows that these results can be explained by a hierarchy of neural assemblies that continuously propagate multiple visual contents. Overall, this study sheds new light on the biological basis of our visual experience.

Self-face and emotional faces-are they alike?

The image of one’s own face is a particularly distinctive feature of the self. The self-face differs from other faces not only in respect of its familiarity but also in respect of its subjective emotional significance and saliency. The current study aimed at elucidating similarities/dissimilarities between processing of one’s own face and emotional faces: happy faces (based on the self-positive bias) and fearful faces (because of their high perceptual saliency, a feature shared with self-face). Electroencephalogram data were collected in the group of 30 participants who performed a simple detection task. Event-related potential analyses indicated significantly increased P3 and late positive potential amplitudes to the self-face in comparison to all other faces: fearful, happy and neutral. Permutation tests confirmed the differences between the self-face and all three types of other faces for numerous electrode sites and in broad time windows. Representational similarity analysis, in turn, revealed distinct processing of the self-face and did not provide any evidence in favour of similarities between the self-face and emotional (either negative or positive) faces. These findings strongly suggest that the self-face processing do not resemble those of emotional faces, thus implying that prioritized self-referential processing is driven by the subjective relevance of one’s own face.

Special Issue: Consciousness science and its theories Time consciousness: the missing link in theories of consciousness

There are plenty of issues to be solved in order for researchers to agree on a neural model of consciousness. Here we emphasize an often under-represented aspect in the debate: time consciousness. Consciousness and the present moment both extend in time. Experience flows through a succession of moments and progresses from future predictions, to present experiences, to past memories. However, a brief review finds that many dominant theories of consciousness only refer to brief, static, and discrete "functional moments" of time. Very few refer to more extended, dynamic, and continuous time, which is associated with conscious experience (cf. the "experienced moment"). This confusion between short and discrete versus long and continuous is, we argue, one of the core issues in theories of consciousness. Given the lack of work dedicated to time consciousness, its study could test novel predictions of rival theories of consciousness. It may be that different theories of consciousness are compatible/complementary if the different aspects of time are taken into account. Or, if it turns out that no existing theory can fully accommodate time consciousness, then perhaps it has something new to add. Regardless of outcome, the crucial step is to make subjective time a central object of study.

Rapidly Decoding Image Categories From MEG Data Using a Multivariate Short-Time FC Pattern Analysis Approach

Recent advances in the development of multivariate analysis methods have led to the application of multivariate pattern analysis (MVPA) to investigate the interactions between brain regions using graph theory (functional connectivity, FC) and decode visual categories from functional magnetic resonance imaging (fMRI) data from a continuous multicategory paradigm. To estimate stable FC patterns from fMRI data, previous studies required long periods in the order of several minutes, in comparison to the human brain that categories visual stimuli within hundreds of milliseconds. Constructing short-time dynamic FC patterns in the order of milliseconds and decoding visual categories is a relatively novel concept. In this study, we developed a multivariate decoding algorithm based on FC patterns and applied it to magnetoencephalography (MEG) data. MEG data were recorded from participants presented with image stimuli in four categories (faces, scenes, animals and tools). MEG data from 17 participants demonstrate that short-time dynamic FC patterns yield brain activity patterns that can be used to decode visual categories with high accuracy. Our results show that FC patterns change over the time window, and FC patterns extracted in the time window of 0∼200 ms after the stimulus onset were most stable. Further, the categorizing accuracy peaked (the mean binary accuracy is above 78.6% at individual level) in the FC patterns estimated within the 0∼200 ms interval. These findings elucidate the underlying connectivity information during visual category processing on a relatively smaller time scale and demonstrate that the contribution of FC patterns to categorization fluctuates over time.

Multiple decisions about one object involve parallel sensory acquisition but time-multiplexed evidence incorporation

The brain is capable of processing several streams of information that bear on different aspects of the same problem. Here, we address the problem of making two decisions about one object, by studying difficult perceptual decisions about the color and motion of a dynamic random dot display. We find that the accuracy of one decision is unaffected by the difficulty of the other decision. However, the response times reveal that the two decisions do not form simultaneously. We show that both stimulus dimensions are acquired in parallel for the initial ∼0.1 s but are then incorporated serially in time-multiplexed bouts. Thus, there is a bottleneck that precludes updating more than one decision at a time, and a buffer that stores samples of evidence while access to the decision is blocked. We suggest that this bottleneck is responsible for the long timescales of many cognitive operations framed as decisions.

Bifurcation in brain dynamics reveals a signature of conscious processing independent of report

An outstanding challenge for consciousness research is to characterize the neural signature of conscious access independently of any decisional processes. Here we present a model-based approach that uses inter-trial variability to identify the brain dynamics associated with stimulus processing. We demonstrate that, even in the absence of any task or behavior, the electroencephalographic response to auditory stimuli shows bifurcation dynamics around 250–300 milliseconds post-stimulus. Namely, the same stimulus gives rise to late sustained activity on some trials, and not on others. This late neural activity is predictive of task-related reports, and also of reports of conscious contents that are randomly sampled during task-free listening. Source localization further suggests that task-free conscious access recruits the same neural networks as those associated with explicit report, except for frontal executive components. Studying brain dynamics through variability could thus play a key role for identifying the core signatures of conscious access, independent of report.

Current knowledge on the neural basis of consciousness mostly relies on situations where people report their perception. Here, the authors provide evidence for the idea that bifurcation in brain dynamics reflects conscious perception independent of report.

Representational dynamics preceding conscious access

Our senses are continuously bombarded with more information than our brain can process up to the level of awareness. The present study aimed to enhance understanding on how attentional selection shapes conscious access under conditions of rapidly changing input. Using an attention task, EEG, and multivariate decoding of individual target- and distractor-defining features, we specifically examined dynamic changes in the representation of targets and distractors as a function of conscious access and the task-relevance (target or distractor) of the preceding item in the RSVP stream. At the behavioral level, replicating previous work and suggestive of a flexible gating mechanism, we found a significant impairment in conscious access to targets (T2) that were preceded by a target (T1) followed by one or two distractors (i.e., the attentional blink), but striking facilitation of conscious access to targets shown directly after another target (i.e., lag-1 sparing and blink reversal). At the neural level, conscious access to T2 was associated with enhanced early- and late-stage T1 representations and enhanced late-stage D1 representations, and interestingly, could be predicted based on the pattern of EEG activation well before T1 was presented. Yet, across task conditions, we did not find convincing evidence for the notion that conscious access is affected by rapid top-down selection-related modulations of the strength of early sensory representations induced by the preceding visual event. These results cannot easily be explained by existing accounts of how attentional selection shapes conscious access under rapidly changing input conditions, and have important implications for theories of the attentional blink and consciousness more generally.

Native non-prototypicality in vowel perception induces prominent neuromagnetic mismatch intensities in non-native speakers: a pilot study

Neural mismatch response resulting from the difference between prediction and observation is related to change detection and discrimination. Robust neuromagnetic brain activity of auditory mismatch-related perception occurs in response to non-prototypical vowels in across-category contrasts for first-language speakers. However, whether this non-prototypicality effect applies to within-category vowel perception remains to be elucidated. Here, healthy Japanese adults (n = 7) were subjected to magnetoencephalography (MEG) while watching a silent movie, and passively listened to synthesized English vowels /i/. We observed the source-level mismatch effect to the mid-high near-front vowel deviant [ɪ] with the most non-prototypical, unspecified feature in the participants' native language system. The mismatch effect recruited the left posterior superior temporal sulcus with a peak latency of 225 ms post-stimulus onset. We further studied whether a longer F1 distance between vowel pairs would increase mismatch-activated intensities, however, we did not observe neuromagnetic changes when the prototypical anchor standard [i] was compared with three non-prototypical deviants differing in first resonance frequency (F1) values. Our results indicate that an F1 increase in within-category upper front vowel perception is a strong activator of mismatch responses measured by source-level activated intensities for non-native listeners.

The predictive global neuronal workspace: A formal active inference model of visual consciousness

The global neuronal workspace (GNW) model has inspired over two decades of hypothesis-driven research on the neural basis of consciousness. However, recent studies have reported findings that are at odds with empirical predictions of the model. Further, the macro-anatomical focus of current GNW research has limited the specificity of predictions afforded by the model. In this paper we present a neurocomputational model - based on Active Inference - that captures central architectural elements of the GNW and is able to address these limitations. The resulting 'predictive global workspace' casts neuronal dynamics as approximating Bayesian inference, allowing precise, testable predictions at both the behavioural and neural levels of description. We report simulations demonstrating the model's ability to reproduce: 1) the electrophysiological and behavioural results observed in previous studies of inattentional blindness; and 2) the previously introduced four-way taxonomy predicted by the GNW, which describes the relationship between consciousness, attention, and sensory signal strength. We then illustrate how our model can reconcile/explain (apparently) conflicting findings, extend the GNW taxonomy to include the influence of prior expectations, and inspire novel paradigms to test associated behavioural and neural predictions.

Head errors of syntactic dependency increase neuromagnetic mismatch intensities

Mismatch-related brain activation in healthy individuals is an important area of neural investigation. Previously, we evaluated sentence-level syntactic dependencies, composed of a head and a dependent between two syntactically related words in head-initial English structures. We demonstrated that prominent mismatch effects were induced by within-category dependent errors when semantic interpretation was preserved. However, the following issues were not addressed: (1) whether head errors of syntactic dependency in head-final structures would elicit large mismatch field (MMF) intensities, and (2) whether an MMF effect of syntactic errors would be seen in the left superior temporal cortex alone. In this study, auditory MMFs were obtained by magnetocephalography (MEG) from healthy Japanese adults (n = 8) who were subjected to a passive auditory oddball paradigm with syntactically legal or illegal utterances and single words in Japanese. The results demonstrate that the source waveforms had significantly higher MMF cortical activation in response to the head error, which involved altered polarity of the predicate. This resulted in a syntactically incorrect and semantically incomprehensible expression, when compared to the syntactically correct expression and the non-structural lexical item. This mismatch effect, with a peak latency of 164 ms, was confined to the anterior region of the left superior temporal cortex. The current results clearly indicate that the representation of syntactic dependency is stored in long-term memory and tends to be activated in automatic auditory processing.

Local syntactic violations evoke fast mismatch-related neural activity detected by optical neuroimaging

It remains to be investigated whether syntax-related mismatch activity would be evoked in event-related optical signals by syntactic violations that deviate from our language knowledge and expectations. In the current study, we have employed fast optical neuroimaging with a frequency-domain oximeter to examine whether syntactic violations of English bare infinitives in the non-finite complement clause would trigger syntax-related mismatch effects. Recorded sentences of bare or full infinitive structures (without or with the 'to' infinitival marker) with syntactically correct or incorrect versions and non-syntactic lexical items (verbs) were presented to native speakers of English (n = 8) during silent movie viewing as a passive oddball task. The analysis of source strength (i.e., minimum norm current amplitudes) revealed that the syntactic category violations of bare object infinitives led to significantly more robust optical mismatch effects than the other syntactic violation and non-structural, lexical elements. This mismatch response had a peak latency of 186 ms in the left anterior superior temporal gyrus. In combination with our prior MEG report (Kubota et al. in Neurosci Lett 662:195-204, 2018), the present optical neuroimaging findings show that syntactic marking (unmarked-to-marked) violations of the bare object infinitive against the rule of the mental grammar enhance the signal strength exactly in the same manner seen with MEG scanning, including the peak latency of mismatch activity and the activated area of the brain.

All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception

Is consciousness a continuous stream of percepts or is it discrete, occurring only at certain moments in time? This question has puzzled philosophers, psychologists, and neuroscientists for centuries. Both hypotheses have fallen repeatedly in and out of favor. Here, we review recent studies exploring long-lasting postdictive effects and show that the results favor a two-stage discrete model, in which substantial periods of continuous unconscious processing precede discrete conscious percepts. We propose that such a model marries the advantages of both continuous and discrete models and resolves centuries old debates about perception and consciousness.

All-or-none face categorization in the human brain

Visual categorization is integral for our interaction with the natural environment. In this process, similar selective responses are produced to a class of variable visual inputs. Whether categorization is supported by partial (graded) or absolute (all-or-none) neural responses in high-level human brain regions is largely unknown. We address this issue with a novel frequency-sweep paradigm probing the evolution of face categorization responses between the minimal and optimal stimulus presentation times. In a first experiment, natural images of variable non-face objects were progressively swept from 120 to 3 ​Hz (8.33-333 ​ms duration) in rapid serial visual presentation sequences. Widely variable face exemplars appeared every 1 ​s, enabling an implicit frequency-tagged face-categorization electroencephalographic (EEG) response at 1 ​Hz. Face-categorization activity emerged with stimulus durations as brief as 17 ​ms (17-83 ​ms across individual participants) but was significant with 33 ​ms durations at the group level. The face categorization response amplitude increased until 83 ​ms stimulus duration (12 ​Hz), implying graded categorization responses. In a second EEG experiment, faces appeared non-periodically throughout such sequences at fixed presentation rates, while participants explicitly categorized faces. A strong correlation between response amplitude and behavioral accuracy across frequency rates suggested that dilution from missed categorizations, rather than a decreased response to each face stimulus, accounted for the graded categorization responses as found in Experiment 1. This was supported by (1) the absence of neural responses to faces that participants failed to categorize explicitly in Experiment 2 and (2) equivalent amplitudes and spatio-temporal signatures of neural responses to behaviorally categorized faces across presentation rates. Overall, these observations provide original evidence that high-level visual categorization of faces, starting at about 100 ​ms following stimulus onset in the human brain, is variable across observers tested under tight temporal constraints, but occurs in an all-or-none fashion.

Auditory cortical alpha/beta desynchronization prioritizes the representation of memory items during a retention period

To-be-memorized information in working-memory could be protected against distracting influences by processes of functional inhibition or prioritization. Modulations of oscillations in the alpha to beta range in task-relevant sensory regions have been suggested to play an important role for both mechanisms. We adapted a Sternberg task variant to the auditory modality, with a strong or a weak distracting sound presented at a predictable time during the retention period. Using a time-generalized decoding approach, relatively decreased strength of memorized information was found prior to strong distractors, paralleled by decreased pre-distractor alpha/beta power in the left superior temporal gyrus (lSTG). Over the entire group, reduced beta power in lSTG was associated with relatively increased strength of memorized information. The extent of alpha power modulations within participants was negatively correlated with strength of memorized information. Overall, our results are compatible with a prioritization account, but point to nuanced differences between alpha and beta oscillations.

Conscious Processing and the Global Neuronal Workspace Hypothesis

We review the central tenets and neuroanatomical basis of the global neuronal workspace (GNW) hypothesis, which attempts to account for the main scientific observations regarding the elementary mechanisms of conscious processing in the human brain. The GNW hypothesis proposes that, in the conscious state, a non-linear network ignition associated with recurrent processing amplifies and sustains a neural representation, allowing the corresponding information to be globally accessed by local processors. We examine this hypothesis in light of recent data that contrast brain activity evoked by either conscious or non-conscious contents, as well as during conscious or non-conscious states, particularly general anesthesia. We also discuss the relationship between the intertwined concepts of conscious processing, attention, and working memory.

Under the Mind's Hood: What We Have Learned by Watching the Brain at Work

Imagine you were asked to investigate the workings of an engine, but to do so without ever opening the hood. Now imagine the engine fueled the human mind. This is the challenge faced by cognitive neuroscientists worldwide aiming to understand the neural bases of our psychological functions. Luckily, human ingenuity comes to the rescue. Around the same time as the Society for Neuroscience was being established in the 1960s, the first tools for measuring the human brain at work were becoming available. Noninvasive human brain imaging and neurophysiology have continued developing at a relentless pace ever since. In this 50 year anniversary, we reflect on how these methods have been changing our understanding of how brain supports mind.

Ensemble representations reveal distinct neural coding of visual working memory

We characterized the population-level neural coding of ensemble representations in visual working memory from human electroencephalography. Ensemble representations provide a unique opportunity to investigate structured representations of working memory because the visual system encodes high-order summary statistics as well as noisy sensory inputs in a hierarchical manner. Here, we consistently observe stable coding of simple features as well as the ensemble mean in frontocentral electrodes, which even correlated with behavioral indices of the ensemble across individuals. In occipitoparietal electrodes, however, we find that remembered features are dynamically coded over time, whereas neural coding of the ensemble mean is absent in the old/new judgment task. In contrast, both dynamic and stable coding are found in the continuous estimation task. Our findings suggest that the prefrontal cortex holds behaviorally relevant abstract representations while visual representations in posterior and visual areas are modulated by the task demands.

Decoding Images in the Mind's Eye: The Temporal Dynamics of Visual Imagery

Mental imagery is the ability to generate images in the mind in the absence of sensory input. Both perceptual visual processing and internally generated imagery engage large, overlapping networks of brain regions. However, it is unclear whether they are characterized by similar temporal dynamics. Recent magnetoencephalography work has shown that object category information was decodable from brain activity during mental imagery, but the timing was delayed relative to perception. The current study builds on these findings, using electroencephalography to investigate the dynamics of mental imagery. Sixteen participants viewed two images of the Sydney Harbour Bridge and two images of Santa Claus. On each trial, they viewed a sequence of the four images and were asked to imagine one of them, which was cued retroactively by its temporal location in the sequence. Time-resolved multivariate pattern analysis was used to decode the viewed and imagined stimuli. Although category and exemplar information was decodable for viewed stimuli, there were no informative patterns of activity during mental imagery. The current findings suggest stimulus complexity, task design and individual differences may influence the ability to successfully decode imagined images. We discuss the implications of these results in the context of prior findings of mental imagery.

Exploring the role of expectations and stimulus relevance on stimulus-specific neural representations and conscious report

Subjective experience can be influenced by top-down factors, such as expectations and stimulus relevance. Recently, it has been shown that expectations can enhance the likelihood that a stimulus is consciously reported, but the neural mechanisms supporting this enhancement are still unclear. We manipulated stimulus expectations within the attentional blink (AB) paradigm using letters and combined visual psychophysics with magnetoencephalographic (MEG) recordings to investigate whether prior expectations may enhance conscious access by sharpening stimulus-specific neural representations. We further explored how stimulus-specific neural activity patterns are affected by the factors expectation, stimulus relevance and conscious report. First, we show that valid expectations about the identity of an upcoming stimulus increase the likelihood that it is consciously reported. Second, using a series of multivariate decoding analyses, we show that the identity of letters presented in and out of the AB can be reliably decoded from MEG data. Third, we show that early sensory stimulus-specific neural representations are similar for reported and missed target letters in the AB task (active report required) and an oddball task in which the letter was clearly presented but its identity was task-irrelevant. However, later sustained and stable stimulus-specific representations were uniquely observed when target letters were consciously reported (decision-dependent signal). Fourth, we show that global pre-stimulus neural activity biased perceptual decisions for a ‘seen’ response. Fifth and last, no evidence was obtained for the sharpening of sensory representations by top-down expectations. We discuss these findings in light of emerging models of perception and conscious report highlighting the role of expectations and stimulus relevance.

Prestimulus feedback connectivity biases the content of visual experiences

Ongoing fluctuations in neural excitability and in networkwide activity patterns before stimulus onset have been proposed to underlie variability in near-threshold stimulus detection paradigms-that is, whether or not an object is perceived. Here, we investigated the impact of prestimulus neural fluctuations on the content of perception-that is, whether one or another object is perceived. We recorded neural activity with magnetoencephalography (MEG) before and while participants briefly viewed an ambiguous image, the Rubin face/vase illusion, and required them to report their perceived interpretation in each trial. Using multivariate pattern analysis, we showed robust decoding of the perceptual report during the poststimulus period. Applying source localization to the classifier weights suggested early recruitment of primary visual cortex (V1) and ∼160-ms recruitment of the category-sensitive fusiform face area (FFA). These poststimulus effects were accompanied by stronger oscillatory power in the gamma frequency band for face vs. vase reports. In prestimulus intervals, we found no differences in oscillatory power between face vs. vase reports in V1 or in FFA, indicating similar levels of neural excitability. Despite this, we found stronger connectivity between V1 and FFA before face reports for low-frequency oscillations. Specifically, the strength of prestimulus feedback connectivity (i.e., Granger causality) from FFA to V1 predicted not only the category of the upcoming percept but also the strength of poststimulus neural activity associated with the percept. Our work shows that prestimulus network states can help shape future processing in category-sensitive brain regions and in this way bias the content of visual experiences.

Automatic and feature-specific prediction-related neural activity in the human auditory system

Prior experience enables the formation of expectations of upcoming sensory events. However, in the auditory modality, it is not known whether prediction-related neural signals carry feature-specific information. Here, using magnetoencephalography (MEG), we examined whether predictions of future auditory stimuli carry tonotopic specific information. Participants passively listened to sound sequences of four carrier frequencies (tones) with a fixed presentation rate, ensuring strong temporal expectations of when the next stimulus would occur. Expectation of which frequency would occur was parametrically modulated across the sequences, and sounds were occasionally omitted. We show that increasing the regularity of the sequence boosts carrier-frequency-specific neural activity patterns during both the anticipatory and omission periods, indicating that prediction-related neural activity is indeed feature-specific. Our results illustrate that even without bottom-up input, auditory predictions can activate tonotopically specific templates.

After listening to a predictable sequence of sounds, we can anticipate and predict the next sound in the sequence. Here, the authors show that during expectation of a sound, the brain generates neural activity matching that which is produced by actually hearing the same sound.