Subjective visual perception: from local processing to emergent phenomena of brain activity

Perception-action Dissociations as a Window into Consciousness

Understanding the neural correlates of unconscious perception stands as a primary goal of experimental research in cognitive psychology and neuroscience. In this Perspectives paper, we explain why experimental protocols probing qualitative dissociations between perception and action provide valuable insights into conscious and unconscious processing, along with their corresponding neural correlates. We present research that utilizes human eye movements as a sensitive indicator of unconscious visual processing. Given the increasing reliance on oculomotor and pupillary responses in consciousness research, these dissociations also provide a cautionary tale about inferring conscious perception solely based on no-report protocols.

An integrative view of the role of prefrontal cortex in consciousness

The involvement of the prefrontal cortex (PFC) in consciousness is an ongoing focus of intense investigation. An important question is whether representations of conscious contents and experiences in the PFC are confounded by post-perceptual processes related to cognitive functions. Here, I review recent findings suggesting that neuronal representations of consciously perceived contents-in the absence of post-perceptual processes-can indeed be observed in the PFC. Slower ongoing fluctuations in the electrophysiological state of the PFC seem to control the stability and updates of these prefrontal representations of conscious awareness. In addition to conscious perception, the PFC has been shown to play a critical role in controlling the levels of consciousness as observed during anesthesia, while prefrontal lesions can result in severe loss of perceptual awareness. Together, the convergence of these processes in the PFC suggests its integrative role in consciousness and highlights the complex nature of consciousness itself.

Structural and Functional Neuroanatomy of Core Consciousness: A Primer for Disorders of Consciousness Clinicians

Understanding the structural and functional neuroanatomy of core consciousness (ie, wakefulness and awareness) is an asset to clinicians caring for persons with disorders of consciousness. This article provides a primer on the structural and functional neuroanatomy of wakefulness and awareness. The neuroanatomical structures supporting these elements of core consciousness functions are reviewed first, after which brief description of the clinically evaluable relationships between disruption of these structures and disorders of consciousness (ie, brain-behavior relationships) are outlined. Consideration of neuroanatomy at the mesoscale (ie, the mesocircuit hypothesis) as well as in relation to several large-scale neural networks is offered.

Bistability of prefrontal states gates access to consciousness

Access of sensory information to consciousness has been linked to the ignition of content-specific representations in association cortices. How does ignition interact with intrinsic cortical state fluctuations to give rise to conscious perception? We addressed this question in the prefrontal cortex (PFC) by combining multi-electrode recordings with a binocular rivalry (BR) paradigm inducing spontaneously driven changes in the content of consciousness, inferred from the reflexive optokinetic nystagmus (OKN) pattern. We find that fluctuations between low-frequency (LF, 1-9 Hz) and beta (∼20-40 Hz) local field potentials (LFPs) reflect competition between spontaneous updates and stability of conscious contents, respectively. Both LF and beta events were locally modulated. The phase of the former locked differentially to the competing populations just before a spontaneous transition while the latter synchronized the neuronal ensemble coding the consciously perceived content. These results suggest that prefrontal state fluctuations gate conscious perception by mediating internal states that facilitate perceptual update and stability.

Multistability, perceptual value, and internal foraging

Substantial experimental, theoretical, and computational insights into sensory processing have been derived from the phenomena of perceptual multistability-when two or more percepts alternate or switch in response to a single sensory input. Here, we review a range of findings suggesting that alternations can be seen as internal choices by the brain responding to values. We discuss how elements of external, experimenter-controlled values and internal, uncertainty- and aesthetics-dependent values influence multistability. We then consider the implications for the involvement in switching of regions, such as the anterior cingulate cortex, which are more conventionally tied to value-dependent operations such as cognitive control and foraging.

Decoding internally generated transitions of conscious contents in the prefrontal cortex without subjective reports

A major debate about the neural correlates of conscious perception concerns its cortical organization, namely, whether it includes the prefrontal cortex (PFC), which mediates executive functions, or it is constrained within posterior cortices. It has been suggested that PFC activity during paradigms investigating conscious perception is conflated with post-perceptual processes associated with reporting the contents of consciousness or feedforward signals originating from exogenous stimulus manipulations and relayed via posterior cortical areas. We addressed this debate by simultaneously probing neuronal populations in the rhesus macaque (Macaca mulatta) PFC during a no-report paradigm, capable of instigating internally generated transitions in conscious perception, without changes in visual stimulation. We find that feature-selective prefrontal neurons are modulated concomitantly with subjective perception and perceptual suppression of their preferred stimulus during both externally induced and internally generated changes in conscious perception. Importantly, this enables reliable single-trial, population decoding of conscious contents. Control experiments confirm significant decoding of stimulus contents, even when oculomotor responses, used for inferring perception, are suppressed. These findings suggest that internally generated changes in the contents of conscious visual perception are reliably reflected within the activity of prefrontal populations in the absence of volitional reports or changes in sensory input.

The role of the prefrontal cortex in conscious perception is debated because of its involvement in task relevant behaviour, such as subjective perceptual reports. Here, the authors show that prefrontal activity in rhesus macaques correlates with subjective perception and the contents of consciousness can be decoded from prefrontal population activity even without reports.

Pulvinar Modulates Contrast Responses in the Visual Cortex as a Function of Cortical Hierarchy

The pulvinar is the largest extrageniculate visual nucleus in mammals. Given its extensive reciprocal connectivity with the visual cortex, it allows the cortico-thalamocortical transfer of visual information. Nonetheless, knowledge of the nature of the pulvinar inputs to the cortex remains elusive. We investigated the impact of silencing the pulvinar on the contrast response function of neurons in 2 distinct hierarchical cortical areas in the cat (areas 17 and 21a). Pulvinar inactivation altered the response gain in both areas, but with larger changes observed in area 21a. A theoretical model was proposed, simulating the pulvinar contribution to cortical contrast responses by modifying the excitation-inhibition balanced state of neurons across the cortical hierarchy. Our experimental and theoretical data showed that the pulvinar exerts a greater modulatory influence on neuronal activity in area 21a than in the primary visual cortex, indicating that the pulvinar impact on cortical visual neurons varies along the cortical hierarchy.

Laminar distribution of cortical projection neurons to the pulvinar: A comparative study in cats and mice

The cortical processing of visual information is thought to follow a hierarchical framework. This framework of connections between visual areas is based on the laminar patterns of direct feedforward and feedback cortico-cortical projections. However, this view ignores the cortico-thalamo-cortical projections to the pulvinar nucleus in the thalamus, which provides an alternative transthalamic information transfer between cortical areas. It was proposed that corticothalamic (CT) pathways follow a similar hierarchical pattern as cortico-cortical connections. Two main types of CT projections have been recognized: drivers and modulators. Drivers originate mainly in Layer 5 whereas modulators are from Layer 6. Little is known about the laminar distribution of these projections to the pulvinar across visual cortical areas. Here, we analyzed the distribution of CT neurons projecting to the lateral posterior (LP) thalamus in two species: cats and mice. Injections of the retrograde tracer fragment B of cholera toxin (CTb) were performed in the LP. The morphology and cortical laminar distribution of CTb-labeled neurons was assessed. In cats, neurons were mostly found in Layer 6 except in Area 17, where they were mostly in Layer 5. In contrast, CT neurons in mice were mostly located in Layer 6 across all areas. Thus, our results demonstrate that CT projections in mice do not follow the same organization as cats suggesting that the transthalamic pathways play distinct roles in these species.

Intracranial Recordings Reveal Unique Shape and Timing of Responses in Human Visual Cortex during Illusory Visual Events

During binocular rivalry, perception spontaneously changes without any alteration to the visual stimulus. What neural events bring about this illusion that a constant stimulus is changing? We recorded from intracranial electrodes placed on the occipital and posterior temporal cortex of two patients with epilepsy while they experienced illusory changes of a face-house binocular-rivalry stimulus or observed a control stimulus that physically changed. We performed within-patient comparisons of broadband high-frequency responses, focusing on single epochs recorded along the ventral processing stream. We found transient face- and house-selective responses localized to the same electrodes for illusory and physical changes, but the temporal characteristics of these responses markedly differed. In comparison with physical changes, responses to illusory changes were longer lasting, in particular exhibiting a characteristic slow rise. Furthermore, the temporal order of responses across the visual hierarchy was reversed for illusory as compared to physical changes: for illusory changes, higher order fusiform and parahippocampal regions responded before lower order occipital regions. Our tentative interpretation of these findings is that two stages underlie the initiation of illusory changes: a destabilization stage in which activity associated with the impending change gradually accumulates across the visual hierarchy, ultimately graduating in a top-down cascade of activity that may stabilize the new perceptual interpretation of the stimulus.

Indicators and Criteria of Consciousness in Animals and Intelligent Machines: An Inside-Out Approach

In today's society, it becomes increasingly important to assess which non-human and non-verbal beings possess consciousness. This review article aims to delineate criteria for consciousness especially in animals, while also taking into account intelligent artifacts. First, we circumscribe what we mean with "consciousness" and describe key features of subjective experience: qualitative richness, situatedness, intentionality and interpretation, integration and the combination of dynamic and stabilizing properties. We argue that consciousness has a biological function, which is to present the subject with a multimodal, situational survey of the surrounding world and body, subserving complex decision-making and goal-directed behavior. This survey reflects the brain's capacity for internal modeling of external events underlying changes in sensory state. Next, we follow an inside-out approach: how can the features of conscious experience, correlating to mechanisms inside the brain, be logically coupled to externally observable ("outside") properties? Instead of proposing criteria that would each define a "hard" threshold for consciousness, we outline six indicators: (i) goal-directed behavior and model-based learning; (ii) anatomic and physiological substrates for generating integrative multimodal representations; (iii) psychometrics and meta-cognition; (iv) episodic memory; (v) susceptibility to illusions and multistable perception; and (vi) specific visuospatial behaviors. Rather than emphasizing a particular indicator as being decisive, we propose that the consistency amongst these indicators can serve to assess consciousness in particular species. The integration of scores on the various indicators yields an overall, graded criterion for consciousness, somewhat comparable to the Glasgow Coma Scale for unresponsive patients. When considering theoretically derived measures of consciousness, it is argued that their validity should not be assessed on the basis of a single quantifiable measure, but requires cross-examination across multiple pieces of evidence, including the indicators proposed here. Current intelligent machines, including deep learning neural networks (DLNNs) and agile robots, are not indicated to be conscious yet. Instead of assessing machine consciousness by a brief Turing-type of test, evidence for it may gradually accumulate when we study machines ethologically and across time, considering multiple behaviors that require flexibility, improvisation, spontaneous problem-solving and the situational conspectus typically associated with conscious experience.

Nondual awareness: Consciousness-as-such as non-representational reflexivity

I introduce arguments toward a non-representational reflexivity theory of consciousness-as-such to address one of the key issues in the science of consciousness today: lack of understanding of the nature of consciousness itself. An expanded map of consciousness is outlined, which includes, in addition to the well-known contents of awareness and levels of arousal, the indeterminate substrate and consciousness-as-such or nondual awareness. The central idea presented is that consciousness-as-such is a non-conceptual nondual awareness, whose essential property is non-representational reflexivity. This property makes consciousness-as-such phenomenologically, cognitively and neurobiologically a unique kind, different from and irreducible to any contents, functions and states, including the indeterminate substrate. Our previous hypothesis on the precuneus network for nondual awareness is further discussed in relation to non-representational reflexivity, and in the light of other hypotheses on the neural correlates of consciousness-as-such.

Parallel and functionally segregated processing of task phase and conscious content in the prefrontal cortex

The role of lateral prefrontal cortex (LPFC) in mediating conscious perception has been recently questioned due to potential confounds resulting from the parallel operation of task related processes. We have previously demonstrated encoding of contents of visual consciousness in LPFC neurons during a no-report task involving perceptual suppression. Here, we report a separate LPFC population that exhibits task-phase related activity during the same task. The activity profile of these neurons could be captured as canonical response patterns (CRPs), with their peak amplitudes sequentially distributed across different task phases. Perceptually suppressed visual input had a negligible impact on sequential firing and functional connectivity structure. Importantly, task-phase related neurons were functionally segregated from the neuronal population, which encoded conscious perception. These results suggest that neurons exhibiting task-phase related activity operate in the LPFC concurrently with, but segregated from neurons representing conscious content during a no-report task involving perceptual suppression.

Vishal Kapoor et al. identify a population of cells in the lateral prefrontal cortex that exhibits task phase-related activity during a no-report task. This cell population is functionally segregated from the population encoding conscious perception, although the two operate in parallel.

Neural dynamics of breaking continuous flash suppression

Sensory input to the human visual system often becomes accessible to cognition and overt report during processing. We investigated neural precursors of conscious vision using EEG recordings and the popular breaking continuous flash suppression (bCFS) paradigm. In this technique, a mask consisting of high-contrast dynamic patterns is presented to one eye, predominating over a target stimulus presented to the other eye. The time needed for the target stimulus to overcome the suppression is thought to reflect the transition from unconscious to conscious perception. In bCFS trials with slow responses, indicative of potent suppression, a time-frequency analysis showed reduced occipital gamma power (33-38 Hz) contralaterally to the visual hemifield where the target was presented 0.27 to 0.21 s prior to the behavioral response. This neural activity was concurrent with a local phase reset and enhanced long-range phase synchronization in the theta band (7 Hz). Such a pattern did not arise in a control condition in which suppression was not induced. Thus, the theta phase reset and synchronization in bCFS trials precede a break from suppression, likely initiating a re-routing of information such that the neural representation of the target is updated more efficiently than that of the competing mask. Overall, these findings mark the emergence of a binocularly integrated percept that can be consciously selected for a behavioral response.

Consciousness, Representation, Action: The Importance of Being Goal-Directed

Recent years have witnessed fierce debates on the dependence of consciousness on interactions between a subject and the environment. Reviewing neuroscientific, computational, and clinical evidence, I will address three questions. First, does conscious experience necessarily depend on acute interactions between a subject and the environment? Second, does it depend on specific perception-action loops in the longer run? Third, which types of action does consciousness cohere with, if not with all of them? I argue that conscious contents do not necessarily depend on acute or long-term brain-environment interactions. Instead, consciousness is proposed to be specifically associated with, and subserve, deliberate, goal-directed behavior (GDB). Brain systems implied in conscious representation are highly connected to, but distinct from, neural substrates mediating GDB and declarative memory.

Intracranial markers of conscious face perception in humans

Investigations of the neural basis of consciousness have greatly benefited from protocols that involve the presentation of stimuli at perceptual threshold, enabling the assessment of the patterns of brain activity that correlate with conscious perception, independently of any changes in sensory input. However, the comparison between perceived and unperceived trials would be expected to reveal not only the core neural substrate of a particular conscious perception, but also aspects of brain activity that facilitate, hinder or tend to follow conscious perception. We take a step towards the resolution of these confounds by combining an analysis of neural responses observed during the presentation of faces partially masked by Continuous Flash Suppression, and those responses observed during the unmasked presentation of faces and other images in the same subjects. We employed multidimensional classifiers to decode physical properties of stimuli or perceptual states from spectrotemporal representations of electrocorticographic signals (1071 channels in 5 subjects). Neural activity in certain face responsive areas located in both the fusiform gyrus and in the lateral-temporal/inferior-parietal cortex discriminated seen vs. unseen faces in the masked paradigm and upright faces vs. other categories in the unmasked paradigm. However, only the former discriminated upright vs. inverted faces in the unmasked paradigm. Our results suggest a prominent role for the fusiform gyrus in the configural perception of faces, and possibly other objects that are holistically processed. More generally, we advocate comparative analysis of neural recordings obtained during different, but related, experimental protocols as a promising direction towards elucidating the functional specificities of the patterns of neural activation that accompany our conscious experiences.

Bottom-Up and Top-Down Mechanisms of General Anesthetics Modulate Different Dimensions of Consciousness

There has been controversy regarding the precise mechanisms of anesthetic-induced unconsciousness, with two salient approaches that have emerged within systems neuroscience. One prominent approach is the “bottom up” paradigm, which argues that anesthetics suppress consciousness by modulating sleep-wake nuclei and neural circuits in the brainstem and diencephalon that have evolved to control arousal states. Another approach is the “top-down” paradigm, which argues that anesthetics suppress consciousness by modulating the cortical and thalamocortical circuits involved in the integration of neural information. In this article, we synthesize these approaches by mapping bottom-up and top-down mechanisms of general anesthetics to two distinct but inter-related dimensions of consciousness: level and content. We show how this explains certain empirical observations regarding the diversity of anesthetic drug effects. We conclude with a more nuanced discussion of how levels and contents of consciousness interact to generate subjective experience and what this implies for the mechanisms of anesthetic-induced unconsciousness.

Lights from the Dark: Neural Responses from a Blind Visual Hemifield

Here we present evidence that a hemianopic patient with a lesion of the left primary visual cortex (V1) showed an unconscious above-chance orientation discrimination with moving rather than static visual gratings presented to the blind hemifield. The patient did not report any perceptual experience of the stimulus features except for a feeling that something appeared in the blind hemifield. Interestingly, in the lesioned left hemisphere, following stimulus presentation to the blind hemifield, we found an event-related potential (ERP) N1 component at a post-stimulus onset latency of 180-260 ms and a source generator in the left BA 19. In contrast, we did not find evidence of the early visual components C1 and P1 and of the later component P300. A positive component (P2a) was recorded between 250 and 320 ms after stimulus onset frontally in both hemispheres. Finally, in the time range 320-440 ms there was a negative peak in right posterior electrodes that was present only for the moving condition. In sum, there were two noteworthy results: Behaviorally, we found evidence of above chance unconscious (blindsight) orientation discrimination with moving but not static stimuli. Physiologically, in contrast to previous studies, we found reliable ERP components elicited by stimuli presented to the blind hemifield at various electrode locations and latencies that are likely to index either the perceptual report of the patient (N1 and P2a) or, the above-chance unconscious performance with moving stimuli as is the case of the posterior ERP negative component. This late component can be considered as the neural correlate of a kind of blindsight enabling feature discrimination only when stimuli are moving and that is subserved by the intact right hemisphere through interhemispheric transfer.

Intracranial spectral amplitude dynamics of perceptual suppression in fronto-insular, occipito-temporal, and primary visual cortex

If conscious perception requires global information integration across active distant brain networks, how does the loss of conscious perception affect neural processing in these distant networks? Pioneering studies on perceptual suppression (PS) described specific local neural network responses in primary visual cortex, thalamus and lateral prefrontal cortex of the macaque brain. Yet the neural effects of PS have rarely been studied with intracerebral recordings outside these cortices and simultaneously across distant brain areas. Here, we combined (1) a novel experimental paradigm in which we produced a similar perceptual disappearance and also re-appearance by using visual adaptation with transient contrast changes, with (2) electrophysiological observations from human intracranial electrodes sampling wide brain areas. We focused on broadband high-frequency (50–150 Hz, i.e., gamma) and low-frequency (8–24 Hz) neural activity amplitude modulations related to target visibility and invisibility. We report that low-frequency amplitude modulations reflected stimulus visibility in a larger ensemble of recording sites as compared to broadband gamma responses, across distinct brain regions including occipital, temporal and frontal cortices. Moreover, the dynamics of the broadband gamma response distinguished stimulus visibility from stimulus invisibility earlier in anterior insula and inferior frontal gyrus than in temporal regions, suggesting a possible role of fronto-insular cortices in top–down processing for conscious perception. Finally, we report that in primary visual cortex only low-frequency amplitude modulations correlated directly with perceptual status. Interestingly, in this sensory area broadband gamma was not modulated during PS but became positively modulated after 300 ms when stimuli were rendered visible again, suggesting that local networks could be ignited by top–down influences during conscious perception.

Perceptual awareness and its neural basis: bridging experimental and theoretical paradigms

Understanding consciousness is a major scientific challenge of our times, and perceptual awareness is an integral part of that challenge. This Theme Issue aims to provide a timely focus on crucial insights from leading scientists on perceptual awareness and its neural basis. The issue refers to key research questions and findings in perceptual awareness research and aims to be a catalyst for further research, by bringing together the state-of-the-art. It shows how bridges are being built between empirical and theoretical research and proposes new directions for the study of multisensory awareness and the role of the states of the body therein. In this introduction, we highlight crucial problems that have characterized the development of the study of perceptual awareness. We then provide an overview of major experimental and theoretical paradigms related to perceptual awareness and its neural basis. Finally, we present an overview of the Theme Issue, with reference to the contributed articles and their relationships.