The autonomy of the visual systems and the modularity of conscious vision

Quality space computations for consciousness

The quality space hypothesis about conscious experience proposes that conscious sensory states are experienced in relation to other possible sensory states. For instance, the colour red is experienced as being more like orange, and less like green or blue. Recent empirical findings suggest that subjective similarity space can be explained in terms of similarities in neural activation patterns. Here, we consider how localist, workspace, and higher-order theories of consciousness can accommodate claims about the qualitative character of experience and functionally support a quality space. We review existing empirical evidence for each of these positions, and highlight novel experimental tools, such as altering local activation spaces via brain stimulation or behavioural training, that can distinguish these accounts.

Functional and structural brain network correlates of visual hallucinations in Lewy body dementia

Visual hallucinations are a common feature of Lewy body dementia. Previous studies have shown that visual hallucinations are highly specific in differentiating Lewy body dementia from Alzheimer’s disease dementia and Alzheimer–Lewy body mixed pathology cases. Computational models propose that impairment of visual and attentional networks is aetiologically key to the manifestation of visual hallucinations symptomatology. However, there is still a lack of experimental evidence on functional and structural brain network abnormalities associated with visual hallucinations in Lewy body dementia.

We used EEG source localization and network based statistics to assess differential topographical patterns in Lewy body dementia between 25 participants with visual hallucinations and 17 participants without hallucinations. Diffusion tensor imaging was used to assess structural connectivity between thalamus, basal forebrain and cortical regions belonging to the functionally affected network component in the hallucinating group, as assessed with network based statistics. The number of white matter streamlines within the cortex and between subcortical and cortical regions was compared between hallucinating and not hallucinating groups and correlated with average EEG source connectivity of the affected subnetwork. Moreover, modular organization of the EEG source network was obtained, compared between groups and tested for correlation with structural connectivity.

Network analysis showed that compared to non-hallucinating patients, those with hallucinations feature consistent weakened connectivity within the visual ventral network, and between this network and default mode and ventral attentional networks, but not between or within attentional networks. The occipital lobe was the most functionally disconnected region. Structural analysis yielded significantly affected white matter streamlines connecting the cortical regions to the nucleus basalis of Meynert and the thalamus in hallucinating compared to not hallucinating patients. The number of streamlines in the tract between the basal forebrain and the cortex correlated with cortical functional connectivity in non-hallucinating patients, while a correlation emerged for the white matter streamlines connecting the functionally affected cortical regions in the hallucinating group.

This study proposes, for the first time, differential functional networks between hallucinating and not hallucinating Lewy body dementia patients, and provides empirical evidence for existing models of visual hallucinations. Specifically, the outcome of the present study shows that the hallucinating condition is associated with functional network segregation in Lewy body dementia and supports the involvement of the cholinergic system as proposed in the current literature.

Refuting the unfolding-argument on the irrelevance of causal structure to consciousness

The unfolding argument (UA) was advanced as a refutation of prominent theories, which posit that phenomenal experience is determined by patterns of neural activation in a recurrent (neural) network (RN) structure. The argument is based on the statement that any input-output function of an RN can be approximated by an "equivalent" feedforward-network (FFN). According to UA, if consciousness depends on causal structure, its presence is unfalsifiable (thus non-scientific), as an equivalent FFN structure is behaviorally indistinguishable with regards to any behavioral test. Here I refute UA by appealing to computational theory and cognitive-neuroscience. I argue that a robust functional equivalence between FFN and RN is not supported by the mathematical work on the Universal Approximator theorem, and is also unlikely to hold, as a conjecture, given data in cognitive neuroscience; I argue that an equivalence of RN and FFN can only apply to static functions between input/output layers and not to the temporal patterns or to the network's reactions to structural perturbations. Finally, I review data indicating that consciousness has functional characteristics, such as a flexible control of behavior, and that cognitive/brain dynamics reveal interacting top-down and bottom-up processes, which are necessary for the mediation of such control processes.

The Strasbourg Visual Scale: A Novel Method to Assess Visual Hallucinations

The experience of oneself in the world is based on sensory afferences, enabling us to reach a first-perspective perception of our environment and to differentiate oneself from the world. Visual hallucinations may arise from a difficulty in differentiating one's own mental imagery from externally-induced perceptions. To specify the relationship between hallucinations and the disorders of the self, we need to understand the mechanisms of hallucinations. However, visual hallucinations are often under reported in individuals with psychosis, who sometimes appear to experience difficulties describing them. We developed the "Strasbourg Visual Scale (SVS)," a novel computerized tool that allows us to explore and capture the subjective experience of visual hallucinations by circumventing the difficulties associated with verbal descriptions. This scale reconstructs the hallucinated image of the participants by presenting distinct physical properties of visual information, step-by-step to help them communicate their internal experience. The strategy that underlies the SVS is to present a sequence of images to the participants whose choice at each step provides a feedback toward re-creating the internal image held by them. The SVS displays simple images on a computer screen that provide choices for the participants. Each step focuses on one physical property of an image, and the successive choices made by the participants help them to progressively build an image close to his/her hallucination, similar to the tools commonly used to generate facial composites. The SVS was constructed based on our knowledge of the visual pathways leading to an integrated perception of our environment. We discuss the rationale for the successive steps of the scale, and to which extent it could complement existing scales.

Demystifying Brain Tumor Segmentation Networks: Interpretability and Uncertainty Analysis

The accurate automatic segmentation of gliomas and its intra-tumoral structures is important not only for treatment planning but also for follow-up evaluations. Several methods based on 2D and 3D Deep Neural Networks (DNN) have been developed to segment brain tumors and to classify different categories of tumors from different MRI modalities. However, these networks are often black-box models and do not provide any evidence regarding the process they take to perform this task. Increasing transparency and interpretability of such deep learning techniques is necessary for the complete integration of such methods into medical practice. In this paper, we explore various techniques to explain the functional organization of brain tumor segmentation models and to extract visualizations of internal concepts to understand how these networks achieve highly accurate tumor segmentations. We use the BraTS 2018 dataset to train three different networks with standard architectures and outline similarities and differences in the process that these networks take to segment brain tumors. We show that brain tumor segmentation networks learn certain human-understandable disentangled concepts on a filter level. We also show that they take a top-down or hierarchical approach to localizing the different parts of the tumor. We then extract visualizations of some internal feature maps and also provide a measure of uncertainty with regards to the outputs of the models to give additional qualitative evidence about the predictions of these networks. We believe that the emergence of such human-understandable organization and concepts might aid in the acceptance and integration of such methods in medical diagnosis.

Transcranial Direct Current Stimulation Alters Functional Network Structure in Humans: A Graph Theoretical Analysis

Transcranial direct current stimulation (tDCS) is routinely used in basic and clinical research, but its efficacy has been challenged on a methodological, statistical and technical basis recently. The arguments against tDCS derive from an insufficient understanding of how this technique interacts with brain processes physiologically. Because of its potential as a central tool in neuroscience, it is important to clarify whether tDCS affects neuronal activity. Here, we investigate influences of offline tDCS on network architecture measured by functional magnetic resonance imaging. Applied to one network node only, offline tDCS affects the architecture of the entire functional network. Furthermore, offline tDCS exerts polarity-specific effects on the topology of the functional network attached. Our results confirm in a functioning brain and in a bias free and independent fashion that offline tDCS influences neuronal activity. Moreover, our results suggest that network-specific connectivity has an important role in improving our understanding of the effects of tDCS.

The contribution of single case studies to the neuroscience of vision

Visual neuroscience is concerned with the neurobiological foundations of visual perception, that is, the morphological, physiological, and functional organization of the visual brain and its co-operative partners. One important approach for understanding the functional organization of the visual brain is the study of visual perception from the pathological perspective. The study of patients with focal injury to the visual brain allows conclusions about the representation of visual perceptual functions in the framework of association and dissociation of functions. Selective disorders have been reported for more "elementary" visual capabilities, for example, color and movement vision, but also for visuo-cognitive capacities, such as visual agnosia or the visual field of attention. Because these visual disorders occur rather seldom as selective and specific dysfunctions, single cases have always played, and still play, a significant role in gaining insights into the functional organization of the visual brain.

Modular structure of brain functional networks: breaking the resolution limit by Surprise

The modular organization of brain networks has been widely investigated using graph theoretical approaches. Recently, it has been demonstrated that graph partitioning methods based on the maximization of global fitness functions, like Newman's Modularity, suffer from a resolution limit, as they fail to detect modules that are smaller than a scale determined by the size of the entire network. Here we explore the effects of this limitation on the study of brain connectivity networks. We demonstrate that the resolution limit prevents detection of important details of the brain modular structure, thus hampering the ability to appreciate differences between networks and to assess the topological roles of nodes. We show that Surprise, a recently proposed fitness function based on probability theory, does not suffer from these limitations. Surprise maximization in brain co-activation and functional connectivity resting state networks reveals the presence of a rich structure of heterogeneously distributed modules, and differences in networks' partitions that are undetectable by resolution-limited methods. Moreover, Surprise leads to a more accurate identification of the network's connector hubs, the elements that integrate the brain modules into a cohesive structure.

From sensation to perception: Using multivariate classification of visual illusions to identify neural correlates of conscious awareness in space and time

An important goal of cognitive neuroscience is understanding the neural underpinnings of conscious awareness. Although the low-level processing of sensory input is well understood in most modalities, it remains a challenge to understand how the brain translates such input into conscious awareness. Here, I argue that the application of multivariate pattern classification techniques to neuroimaging data acquired while observers experience perceptual illusions provides a unique way to dissociate sensory mechanisms from mechanisms underlying conscious awareness. Using this approach, it is possible to directly compare patterns of neural activity that correspond to the contents of awareness, independent from changes in sensory input, and to track these neural representations over time at high temporal resolution. I highlight five recent studies using this approach, and provide practical considerations and limitations for future implementations.

An invisible touch: Body-related multisensory conflicts modulate visual consciousness

The majority of scientific studies on consciousness have focused on vision, exploring the cognitive and neural mechanisms of conscious access to visual stimuli. In parallel, studies on bodily consciousness have revealed that bodily (i.e. tactile, proprioceptive, visceral, vestibular) signals are the basis for the sense of self. However, the role of bodily signals in the formation of visual consciousness is not well understood. Here we investigated how body-related visuo-tactile stimulation modulates conscious access to visual stimuli. We used a robotic platform to apply controlled tactile stimulation to the participants' back while they viewed a dot moving either in synchrony or asynchrony with the touch on their back. Critically, the dot was rendered invisible through continuous flash suppression. Manipulating the visual context by presenting the dot moving on either a body form, or a non-bodily object we show that: (i) conflict induced by synchronous visuo-tactile stimulation in a body context is associated with a delayed conscious access compared to asynchronous visuo-tactile stimulation, (ii) this effect occurs only in the context of a visual body form, and (iii) is not due to detection or response biases. The results indicate that body-related visuo-tactile conflicts impact visual consciousness by facilitating access of non-conflicting visual information to awareness, and that these are sensitive to the visual context in which they are presented, highlighting the interplay between bodily signals and visual experience.

A new look on S-R associations: How S and R link

Humans can learn associations between stimuli and responses which allow for faster, more efficient behavior when the same response is required to the same stimulus in the future. This is called stimulus-response (S-R) priming. Perceptual representations are known to be modular and hierarchical, i.e. different brain areas represent different perceptual features and higher brain areas represent increasingly abstract properties of the stimulus. In this study we investigated how perceptually specific the stimulus in S-R priming is. In particular we wanted to test whether basic visual features play a role in the S-R associations. We used a novel stimulus: images of objects built from basic visual features. Participants performed a classification task on the objects. We found no significant effect on reaction times of switching vs. repeating perceptual features between presentations of the same object. This suggests that S-R associations involve a perceptually non-specific stimulus representation.

Distributed processing of color and form in the visual cortex

To what extent does the visual system process color and form separately? Proponents of the segregation view claim that distinct regions of the cortex are dedicated to each of these two dimensions separately. However, evidence is accumulating that color and form processing may, at least to some extent, be intertwined in the brain. In this perspective, we review psychophysical and neurophysiological studies on color and form perception and evaluate their results in light of recent developments in population coding.

Timing of speech and display affects the linguistic mediation of visual search

Recent studies have shown that, instead, of a dichotomy between parallel and serial search strategies, in many instances we see a combination of both search strategies utilized. Consequently, computational models and theoretical accounts of visual search processing have evolved from traditional serial-parallel descriptions to a continuum from 'efficient' to 'inefficient' search. One of the findings, consistent with this blurring of the serial-parallel distinction, is that concurrent spoken linguistic input influences the efficiency of visual search. In our first experiment we replicate those findings using a between-subjects design. Next, we utilize a localist attractor network to simulate the results from the first experiment, and then employ the network to make quantitative predictions about the influence of subtle timing differences of real-time language processing on visual search. These model predictions are then tested and confirmed in our second experiment. The results provide further evidence toward understanding linguistically mediated influences on real-time visual search processing and support an interactive processing account of visual search and language comprehension.

Domain-specific impairment in metacognitive accuracy following anterior prefrontal lesions

Convergent evidence supports a role for anterior prefrontal cortex (PFC) in metacognition—the capacity to evaluate cognitive processes—but whether metacognition relies on global or domain-specific substrates is unknown. Fleming et al. report that patients with anterior PFC lesions show impaired perceptual metacognition despite intact memory metacognition, supporting a domain-specific account.

Humans have the capacity to evaluate the success of cognitive processes, known as metacognition. Convergent evidence supports a role for anterior prefrontal cortex in metacognitive judgements of perceptual processes. However, it is unknown whether metacognition is a global phenomenon, with anterior prefrontal cortex supporting metacognition across domains, or whether it relies on domain-specific neural substrates. To address this question, we measured metacognitive accuracy in patients with lesions to anterior prefrontal cortex (n = 7) in two distinct domains, perception and memory, by assessing the correspondence between objective performance and subjective ratings of performance. Despite performing equivalently to a comparison group with temporal lobe lesions (n = 11) and healthy controls (n = 19), patients with lesions to the anterior prefrontal cortex showed a selective deficit in perceptual metacognitive accuracy (meta-d’/d’, 95% confidence interval 0.28–0.64). Crucially, however, the anterior prefrontal cortex lesion group’s metacognitive accuracy on an equivalent memory task remained unimpaired (meta-d’/d’, 95% confidence interval 0.78–1.29). Metacognitive accuracy in the temporal lobe group was intact in both domains. Our results support a causal role for anterior prefrontal cortex in perceptual metacognition, and indicate that the neural architecture of metacognition, while often considered global and domain-general, comprises domain-specific components that may be differentially affected by neurological insult.

Effects of task and attentional selection on responses in human visual cortex

Multiple visual tasks can be performed on the same visual input, with different tasks presumably engaging different neuronal populations. The modular layout of the visual system implies that specific cortical regions carry more information about certain stimulus attributes than others. Thus it is reasonable to assume that decisions during a task will be optimal if they are based on the responses of the most informative neuronal signals, which presumably originate in regions with the sharpest tuning for the relevant stimulus feature. Previous studies have supported this position. Here we present the results of two fMRI experiments that confirm these findings and expand on earlier investigations by addressing the effects of the physical properties of an attended stimulus on task-related modulations in human visual cortex. Specifically, we ask whether performing two-alternative forced choice speed- and color-discrimination tasks (and other attentional processes) can modulate neural activity independent of visual stimulation and whether the effect of spatial attention depends on which task is being performed. The results indicate that 1) when stimulation and spatial attention are constant, responses in V4 and MT+ depend on the task being performed and are independent of the tested physical properties of the selected stimulus, 2) this task-dependent modulation might require a stimulus--task-specific preparatory mechanisms alone are not sufficient to drive responses, and 3) independent of which task is being performed, spatial attention adds a baseline shift to responses in MT+ and V4 when a stimulus is present.

Abnormal modular organization of functional networks in cognitively impaired children with frontal lobe epilepsy

Many children with frontal lobe epilepsy (FLE) have significant cognitive comorbidity, for which the underlying mechanism has not yet been unraveled, but is likely related to disturbed cerebral network integrity. Using resting-state fMRI, we investigated whether cerebral network characteristics are associated with epilepsy and cognitive comorbidity. We included 37 children with FLE and 41 healthy age-matched controls. Cognitive performance was determined by means of a computerized visual searching task. A connectivity matrix for 82 cortical and subcortical brain regions was generated for each subject by calculating the inter-regional correlation of the fMRI time signals. From the connectivity matrix, graph metrics were calculated and the anatomical configuration of aberrant connections and modular organization was investigated. Both patients and controls displayed efficiently organized networks. However, FLE patients displayed a higher modularity, implying that subnetworks are less interconnected. Impaired cognition was associated with higher modularity scores and abnormal modular organization of the brain, which was mainly expressed as a decrease in long-range and an increase in interhemispheric connectivity in patients. We showed that network modularity analysis provides a sensitive marker for cognitive impairment in FLE and suggest that abnormally interconnected functional subnetworks of the brain might underlie the cognitive problems in children with FLE.

Behavior in oblivion: the neurobiology of subliminal priming

Subliminal priming refers to behavioral modulation by an unconscious stimulus, and can thus be regarded as a form of unconscious visual processing. Theories on recurrent processing have suggested that the neural correlate of consciousness (NCC) comprises of the non-hierarchical transfer of stimulus-related information. According to these models, the neural correlate of subliminal priming (NCSP) corresponds to the visual processing within the feedforward sweep. Research from cognitive neuroscience on these two concepts and the relationship between them is discussed here. Evidence favoring the necessity of recurrent connectivity for visual awareness is accumulating, although some questions, such as the need for global versus local recurrent processing, are not clarified yet. However, this is not to say that recurrent processing is sufficient for consciousness, as a neural definition of consciousness in terms of recurrent connectivity would imply. We argue that the limited interest cognitive neuroscience currently has for the NCSP is undeserved, because the discovery of the NCSP can give insight into why people do (and do not) express certain behavior.

Enhancement of neural representation capacity by modular architecture in networks of cortical neurons

Biological networks are ubiquitously modular, a feature that is believed to be essential for the enhancement of their functional capacities. Here, we have used a simple modular in vitro design to examine the possibility that modularity enhances network functionality in the context of input representation. We cultured networks of cortical neurons obtained from newborn rats in vitro on substrate-integrated multi-electrode arrays, forcing the network to develop two well-defined modules of neural populations that are coupled by a narrow canal. We measured the neural activity, and examined the capacity of each module to individually classify (i.e. represent) spatially distinct electrical stimuli and propagate input-specific activity features to their downstream coupled counterpart. We show that, although each of the coupled modules maintains its autonomous functionality, a significant enhancement of representational capacity is achieved when the system is observed as a whole. We interpret our results in terms of a relative decorrelation effect imposed by weak coupling between modules.

Visual awareness suppression by pre-stimulus brain stimulation; a neural effect

Transcranial magnetic stimulation (TMS) has established the functional relevance of early visual cortex (EVC) for visual awareness with great temporal specificity non-invasively in conscious human volunteers. Many studies have found a suppressive effect when TMS was applied over EVC 80-100 ms after the onset of the visual stimulus (post-stimulus TMS time window). Yet, few studies found task performance to also suffer when TMS was applied even before visual stimulus presentation (pre-stimulus TMS time window). This pre-stimulus TMS effect, however, remains controversially debated and its origin had mainly been ascribed to TMS-induced eye-blinking artifacts. Here, we applied chronometric TMS over EVC during the execution of a visual discrimination task, covering an exhaustive range of visual stimulus-locked TMS time windows ranging from -80 pre-stimulus to 300 ms post-stimulus onset. Electrooculographical (EoG) recordings, sham TMS stimulation, and vertex TMS stimulation controlled for different types of non-neural TMS effects. Our findings clearly reveal TMS-induced masking effects for both pre- and post-stimulus time windows, and for both objective visual discrimination performance and subjective visibility. Importantly, all effects proved to be still present after post hoc removal of eye blink trials, suggesting a neural origin for the pre-stimulus TMS suppression effect on visual awareness. We speculate based on our data that TMS exerts its pre-stimulus effect via generation of a neural state which interacts with subsequent visual input.

Modular and hierarchically modular organization of brain networks

Brain networks are increasingly understood as one of a large class of information processing systems that share important organizational principles in common, including the property of a modular community structure. A module is topologically defined as a subset of highly inter-connected nodes which are relatively sparsely connected to nodes in other modules. In brain networks, topological modules are often made up of anatomically neighboring and/or functionally related cortical regions, and inter-modular connections tend to be relatively long distance. Moreover, brain networks and many other complex systems demonstrate the property of hierarchical modularity, or modularity on several topological scales: within each module there will be a set of sub-modules, and within each sub-module a set of sub-sub-modules, etc. There are several general advantages to modular and hierarchically modular network organization, including greater robustness, adaptivity, and evolvability of network function. In this context, we review some of the mathematical concepts available for quantitative analysis of (hierarchical) modularity in brain networks and we summarize some of the recent work investigating modularity of structural and functional brain networks derived from analysis of human neuroimaging data.

Generic aspects of complexity in brain imaging data and other biological systems

A key challenge for systems neuroscience is the question of how to understand the complex network organization of the brain on the basis of neuroimaging data. Similar challenges exist in other specialist areas of systems biology because complex networks emerging from the interactions between multiple non-trivially interacting agents are found quite ubiquitously in nature, from protein interactomes to ecosystems. We suggest that one way forward for analysis of brain networks will be to quantify aspects of their organization which are likely to be generic properties of a broader class of biological systems. In this introductory review article we will highlight four important aspects of complex systems in general: fractality or scale-invariance; criticality; small-world and related topological attributes; and modularity. For each concept we will provide an accessible introduction, an illustrative data-based example of how it can be used to investigate aspects of brain organization in neuroimaging experiments, and a brief review of how this concept has been applied and developed in other fields of biomedical and physical science. The aim is to provide a didactic, focussed and user-friendly introduction to the concepts of complexity science for neuroscientists and neuroimagers.

Age-related changes in modular organization of human brain functional networks

Graph theory allows us to quantify any complex system, e.g., in social sciences, biology or technology, that can be abstractly described as a set of nodes and links. Here we derived human brain functional networks from fMRI measurements of endogenous, low frequency, correlated oscillations in 90 cortical and subcortical regions for two groups of healthy (young and older) participants. We investigated the modular structure of these networks and tested the hypothesis that normal brain aging might be associated with changes in modularity of sparse networks. Newman's modularity metric was maximised and topological roles were assigned to brain regions depending on their specific contributions to intra- and inter-modular connectivity. Both young and older brain networks demonstrated significantly non-random modularity. The young brain network was decomposed into 3 major modules: central and posterior modules, which comprised mainly nodes with few inter-modular connections, and a dorsal fronto-cingulo-parietal module, which comprised mainly nodes with extensive inter-modular connections. The mean network in the older group also included posterior, superior central and dorsal fronto-striato-thalamic modules but the number of intermodular connections to frontal modular regions was significantly reduced, whereas the number of connector nodes in posterior and central modules was increased.

A theory of moving form perception: Synergy between masking, perceptual grouping, and motion computation in retinotopic and non-retinotopic representations

Because object and self-motion are ubiquitous in natural viewing conditions, understanding how the human visual system achieves a relatively clear perception for moving objects is a fundamental problem in visual perception. Several studies have shown that the visible persistence of a briefly presented stationary stimulus is approximately 120 ms under normal viewing conditions. Based on this duration of visible persistence, we would expect moving objects to appear highly blurred. However, in human vision, objects in motion typically appear relatively sharp and clear. We suggest that clarity of form in dynamic viewing is achieved by a synergy between masking, perceptual grouping, and motion computation across retinotopic and non-retinotopic representations. We also argue that dissociations observed in masking are essential to create and maintain this synergy.

Neuronal correlates of perceptual stability during eye movements

We are usually unaware of retinal image motion resulting from our own movement. For instance, during slow-tracking eye movements the world around us remains perceptually stable despite the retinal image slip induced by the eye movement. It is commonly held that this example of perceptual invariance is achieved by subtracting an internal reference signal, reflecting the eye movement, from the retinal motion signal. If the two cancel each other, visual objects, which do not move, will also be perceived as non-moving. If, however, the reference signal is too small or too large, a false eye movement-induced motion of the external world, the Filehne illusion, will be perceived. We have exploited our ability to manipulate the size of the reference signal in an attempt to identify neurons in the visual cortex of monkeys, influenced by the percept of self-induced visual motion or the reference signal rather than the retinal motion signal. We report here that such 'percept-related' neurons can already be found in the primary visual cortex area, although few in numbers. They become more frequent in areas middle temporal and medial superior temporal in the superior temporal sulcus, and comprise almost 50% of all neurons in area visual posterior sylvian (VPS) in the posterior part of the lateral sulcus. In summary, our findings suggest that our ability to perceive a visual world, which is stable despite self-motion, is based on a neuronal network, which culminates in the VPS located in the lateral sulcus below the classical dorsal stream of visual processing.

The cognitive and neural correlates of “tactile consciousness”: A multisensory perspective

People's awareness of tactile stimuli has been investigated in far less detail than their awareness of stimuli in other sensory modalities. In an attempt to fill this gap, we provide an overview of studies that are pertinent to the topic of tactile consciousness. We discuss the results of research that has investigated phenomena such as "change blindness", phantom limb sensations, and numerosity judgments in tactile perception, together with the results obtained from the study of patients affected by deficits that can adversely affect tactile perception such as neglect, extinction, and numbsense. The similarities as well as some of the important differences that have emerged when visual and tactile conscious information processing have been compared using similar experimental procedures are highlighted. We suggest that conscious information processing in the tactile modality cannot be separated completely from the more general processing of spatial information in the brain. Finally, the importance of considering tactile consciousness within the larger framework of multisensory information processing is also discussed.

The contribution of synaptic plasticity in the basal ganglia to the processing of visual information

A mechanism for the involvement of the basal ganglia in the processing of visual information, based on dopamine-dependent modulation of the efficiency of synaptic transmission in interconnected parallel associative and limbic cortex-basal ganglia-thalamus-cortex circuits, is proposed. Each circuit consists of a visual or prefrontal area of the cortex connected with the thalamic nucleus and the corresponding areas in different nuclei of the basal ganglia. The circulation of activity in these circuits is supported by the recurrent arrival of information in the thalamus and cortex. Dopamine released in response to a visual stimulus modulates the efficiencies of "strong" and "weak" corticostriatal inputs in different directions, and the subsequent reorganization of activity in the circuit leads to disinhibition (inhibition) of the activity of those cortical neurons which are "strongly" ("weakly") excited by the visual stimulus simultaneously with dopaminergic cells. The pattern in each cortical area is the neuronal reflection of the properties of the visual stimulus processed by this area. Excitation of dopaminergic cells by the visual stimulus via the superior colliculi requires parallel activation of the disinhibitory input to the superior colliculi via the thalamus and the "direct" pathway" in the basal ganglia. The prefrontal cortex, excited by the visual stimulus via the mediodorsal nucleus of the thalamus, mediates the descending influence on the activity of dopaminergic cells, simultaneously controlling dopamine release in different areas of the striatum and thus facilitating the mutual selection of neural reflections of the individual properties of the visual stimulus and their binding into an integral image.

Independent coding of object motion and position revealed by distinct contingent aftereffects

Despite several findings of perceptual asynchronies between object features, it remains unclear whether independent neuronal populations necessarily code these perceptually unbound properties. To examine this, we investigated the binding between an object's spatial frequency and its rotational motion using contingent motion aftereffects (MAE). Subjects adapted to an oscillating grating whose direction of rotation was paired with a high or low spatial frequency pattern. In separate adaptation conditions, we varied the moment when the spatial frequency change occurred relative to the direction reversal. After adapting to one stimulus, subjects made judgments of either the perceived MAE (rotational movement) or the position shift (instantaneous phase rotation) that accompanied the MAE. To null the spatial frequency-contingent MAE, motion reversals had to physically lag changes in spatial frequency during adaptation. To null the position shift that accompanied the MAE, however, no temporal lag between the attributes was required. This demonstrates that perceived motion and position can be perceptually misbound. Indeed, in certain conditions, subjects perceived the test pattern to drift in one direction while its position appeared shifted in the opposite direction. The dissociation between perceived motion and position of the same test pattern, following identical adaptation, demonstrates that distinguishable neural populations code for these object properties.

The chronoarchitecture of the cerebral cortex

We review here a new approach to mapping the human cerebral cortex into distinct subdivisions. Unlike cytoarchitecture or traditional functional imaging, it does not rely on specific anatomical markers or functional hypotheses. Instead, we propose that the unique activity time course (ATC) of each cortical subdivision, elicited during natural conditions, acts as a temporal fingerprint that can be used to segregate cortical subdivisions, map their spatial extent, and reveal their functional and potentially anatomical connectivity. We argue that since the modular organisation of the brain and its connectivity evolved and developed in natural conditions, these are optimal for revealing its organisation. We review the concepts, methodology and first results of this approach, relying on data obtained with functional magnetic resonance imaging (fMRI) when volunteers viewed traditional stimuli or a James Bond movie. Independent component analysis (ICA) was used to identify voxels belonging to distinct functional subdivisions, based on their differential spatio-temporal fingerprints. Many more regions could be segregated during natural viewing, demonstrating that the complexity of natural stimuli leads to more differential responses in more functional modules. We demonstrate that, in a single experiment, a multitude of distinct regions can be identified across the whole brain, even within the visual cortex, including areas V1, V4 and V5. This differentiation is based entirely on the differential ATCs of different areas during natural viewing. Distinct areas can therefore be identified without any a priori hypothesis about their function or spatial location. The areas we identified corresponded anatomically across subjects, and their ATCs showed highly area-specific inter-subject correlations. Furthermore, natural conditions led to a significant de-correlation of interregional ATCs compared to rest, indicating an increase in regional specificity during natural conditions. In contrast, the correlation between ATCs of distant regions of known substantial anatomical connections increased and reflected their known anatomical connectivity pattern. We demonstrate this using the example of the language network involving Broca's and Wernicke's area and homologous areas in the two hemispheres. In conclusion, this new approach to brain mapping may not only serve to identify novel functional subdivisions, but to reveal their connectivity as well.

Projections from orbitofrontal cortex to anterior piriform cortex in the rat suggest a role in olfactory information processing

The orbitofrontal cortex (OFC) has been characterized as a higher-order, multimodal sensory cortex. Evidence from electrophysiological and behavioral studies in the rat has suggested that OFC plays a role in modulating olfactory guided behavior, and a significant projection to OFC arises from piriform cortex, the traditional primary olfactory cortex. To discern how OFC interacts with primary olfactory structures, the anterograde tracer Phaseolus vulgaris leucoagglutinin was injected into orbitofrontal cortical areas in adult male rats. Labeled fibers were found in the piriform cortex and olfactory bulb on the side ipsilateral to the injection. Notably, the projection to piriform cortex was predominantly from ventrolateral orbital cortex, and was not uniform; rostrally, the projection to the ventral portion of the anterior piriform cortex (APC) was substantial, while the dorsal APC was virtually free of labeled fibers. Labeled fibers were found in both the dorsal and ventral portions in more caudal regions of APC. Most labeled fibers were found in layer III, although a substantial number of fibers were observed in layers Ib and II. Labeled fibers in posterior piriform cortex also were seen after injection into orbitofrontal areas. Taken together with previous reports, these findings suggest that piriform cortex includes multiple subdivisions, which may perform separate, parallel functions in olfactory information processing. Further, these results suggest that the OFC, in addition to its putative role in encoding information about the significance of olfactory stimuli, may play a role in modulating odor response properties of neurons in piriform cortex.

Dynamics of thalamo-cortical network oscillations and human perception

There is increasing evidence that human cognitive functions can be addressed from a robust neuroscience perspective. In particular, the distributed coherent electrical properties of central neuronal ensembles are considered to be a promising avenue of inquiry concerning global brain functions. The intrinsic oscillatory properties of neurons (Llinás, R. (1988) The intrinsic electrophysiological properties of mammalian neurons: Insights into central nervous system function. Science, 242: 1654-1664), supported by a large variety of voltage-gated ionic conductances are recognized to be the central elements in the generation of the temporal binding required for cognition. Research in neuroscience further indicates that oscillatory activity in the gamma band (25-50 Hz) can be correlated with both sensory acquisition and pre-motor planning, which are non-continuous functions in the time domain. From this perspective, gamma-band activity is viewed as serving a broad temporal binding function, where single-cell oscillators and the conduction time of the intervening pathways support large multicellular thalamo-cortical resonance that is closely linked with cognition and subjective experience. Our working hypothesis is that although dedicated units achieve sensory processing, the cognitive binding process is a common mechanism across modalities. Moreover, it is proposed that such time-dependent binding when altered, will result in modifications of the sensory motor integration that will affect and impair cognition and conscious perception.

Perceptual asynchronies for biological and non-biological visual events

Four experiments investigated the hypothesis that different attributes of a visual scene are processed by independent channels working asynchronously. Experiment 1 considered the attributes of colour, form, and movement of simple geometrical configurations. In each of three conditions, two of these attributes switched simultaneously between two fixed values (Green/Red, Circle/Square, Fixed/Moving). Participants indicated which of the two attributes changes was perceptually closer in time to a sound signal. Response probabilities varied as a function of the time of occurrence of the sound, showing that the processing of the movement channel is delayed with respect to the other two. A smaller but significant difference was also detected between the processing times for colour and form. Comparing Experiments 1 and 2 showed that movement velocity does not affect the delay with which movement onset is perceived with respect to colour. Experiment 3 contrasted colour and movement in the perception of a biological movement. The stimuli were video clips of a coloured ball being lifted by a hand. The colour of the ball changed a variable amount of time before or after the ball started moving. Participants indicated which of the two changes had occurred first. We found that, unlike in Experiments 1 and 2, movement perception no longer lagged colour perception. Experiment 4 tested the hypothesis that the disappearance of the asynchrony is due to perceptual anticipation. We discuss the implications of the results vis-à-vis current theories on perceptual binding and on the coding of dynamic events.

Of lasers, mutants, and see-through brains: functional neuroanatomy in zebrafish

Behavioral functions are carried out by localized circuits in the brain. Although this modular principle is clearly established, the boundaries of modules, and sometimes even their existence, are still debated. Zebrafish might offer distinct advantages in localizing behaviors to discrete brain regions because of the ability to visualize, record from, and lesion precisely identified populations of neurons in the brain. In addition, genetic screens in zebrafish enable the isolation of mutations that disrupt neural pathways and/or behaviors, as an alternative lesioning technique with complementary strengths to laser ablations. For example, the Mauthner cell, a large identified neuron in the hindbrain, has been postulated to be both necessary and sufficient for the execution of escapes. We discuss in this review how experiments, using laser ablations, calcium imaging, and mutants have eroded this notion. Even in a simple behavior, such as escape, many parallel pathways appear to be involved with no single one being absolutely necessary. Lesion studies and the analysis of behavioral mutants are now also beginning to elucidate the functional architecture of the zebrafish visual system. Although still in an embryonic stage, the neuroanatomy of behaviors in zebrafish has a bright future.

The effects of task and saliency on latencies for colour and motion processing

In human visual perception, there is evidence that different visual attributes, such as colour, form and motion, have different neural-processing latencies. Specifically, recent studies have suggested that colour changes are processed faster than motion changes. We propose that the processing latencies should not be considered as fixed quantities for different attributes, but instead depend upon attribute salience and the observer's task. We asked observers to respond to high- and low-salience colour and motion changes in three different tasks. The tasks varied from having a strong motor component to having a strong perceptual component. Increasing salience led to shorter processing times in all three tasks. We also found an interaction between task and attribute: motion was processed more quickly in reaction-time tasks, whereas colour was processed more quickly in more perceptual tasks. Our results caution against making direct comparisons between latencies for processing different visual attributes without equating salience or considering task effects. More-salient attributes are processed faster than less-salient ones, and attributes that are critical for the task are also processed more quickly.

Color vision

Color vision starts with the absorption of light in the retinal cone photoreceptors, which transduce electromagnetic energy into electrical voltages. These voltages are transformed into action potentials by a complicated network of cells in the retina. The information is sent to the visual cortex via the lateral geniculate nucleus (LGN) in three separate color-opponent channels that have been characterized psychophysically, physiologically, and computationally. The properties of cells in the retina and LGN account for a surprisingly large body of psychophysical literature. This suggests that several fundamental computations involved in color perception occur at early levels of processing. In the cortex, information from the three retino-geniculate channels is combined to enable perception of a large variety of different hues. Furthermore, recent evidence suggests that color analysis and coding cannot be separated from the analysis and coding of other visual attributes such as form and motion. Though there are some brain areas that are more sensitive to color than others, color vision emerges through the combined activity of neurons in many different areas.

Asynchronous perception of motion and luminance change

Observers were asked to indicate when a target moving on a circular trajectory changed its luminance. The judged position of the luminance change was displaced from the true position in the direction of motion, indicating differences between the times-to-consciousness of motion and luminance change. Motion was processed faster than luminance change. The latency difference was more pronounced for a small (116-134 ms) than for a large luminance decrement (37 ms). The results show that first-order motion is perceived before an accurate representation of luminance is available. These findings are consistent with current accounts of the flash-lag effect. Two control experiments ruled out that the results were due to a general forward tendency. Localization of the target when an auditory signal was presented did not produce forward displacement, and the judged onset of motion was not shifted in the direction of motion.

Differences in word associations to pictures and words

Normal subjects were asked to produce the "first word that comes to mind" in response to pictures or words that differed with respect to manipulability and animacy. In separate analyses across subjects and items, normal subjects produced a significantly higher proportion of action words (that is, verbs) to pictures as compared to words, to manipulable as compared to non-manipulable stimuli and to inanimate as compared to animate stimuli. The largest proportion of action words was elicited by pictures of non-living, manipulable objects. Furthermore, associates to words matched standard word associates significantly more often than those elicited by pictures. These data suggest that pictures and words initially contact different forms of conceptual information and are consistent with an account of semantic organization that assumes that information is distributed across different domains reflecting the mode of acquisition of that knowledge.

Physical, neural, and mental timing

The conclusions drawn by Benjamin Libet from his work with colleagues on the timing of somatosensorial conscious experiences has met with a lot of praise and criticism. In this issue we find three examples of the latter. Here I attempt to place the divide between the two opponent camps in a broader perspective by analyzing the question of the relation between physical timing, neural timing, and experiential (mental) timing. The nervous system does a sophisticated job of recombining and recoding messages from the sensorial surfaces and if these processes are slighted in a theory, it might become necessary to postulate weird operations, including subjective back-referral. Neuroscientifically inspired theories are of necessity still based on guesses, extrapolations, and philosophically dubious manners of speech. They often assume some neural correlate of consciousness (NCC) as a part of the nervous system that transforms neural activity in reportable experiences. The majority of neuroscientists appear to assume that the NCC can compare and bind activity patterns only if they arrive simultaneously at the NCC. This leads to a search for synchrony or to theories in terms of the compensation of differences in neural delays (latencies). This is the main dimension of the Libet discussion. Examples from vision research, such as "temporal-binding-by-synchrony" and the "flash-lag" effect, are then used to illustrate these reasoning patterns in more detail. Alternatively one could assume symbolic representations of time and space (symbolic "tags") that are not coded in their own dimension (not time in time and space in space). Unless such tags are multiplexed with the quality message (tickle, color, or motion), one gets a binding problem for tags. One of the hidden aspects of the discussion between Libet and opponents appears to be the following. Is the NCC smarter than the rest of the nervous system, so that it can solve the problems of local sign (e.g., "where is the event"?) and timing (e.g., "when did it occur?" and "how long did it last?") on its own, or are these pieces of information coded symbolically early on in the system? A supersmart NCC appears to be the assumption of Libet's camp (which includes Descartes, but also mystics). The wish to distribute the smartness evenly across all stages of processing in the nervous system (smart recodings) appears to motivate the opponents. I argue that there are reasons to side with the latter group.

What can functional imaging reveal about the role of attention in visual awareness?

This review focuses on neuroimaging studies that address the relationship between selective attention, neural activity and visual awareness. Withdrawing attention from particular visual stimuli reduces modality-specific processing in posterior visual cortex, and when attention is fully engaged elsewhere, even highly salient but task-irrelevant stimuli can fail to evoke activity and reach awareness. However, the link between visual attention and awareness extends beyond posterior visual cortex to also encompass regions of parietal and prefrontal cortex. Activity in the posterior visual cortex may be necessary but not sufficient for awareness, without a contribution from frontal and parietal cortex. Consistent with this, enhanced interactions between parietal, frontal and posterior visual cortex are observed as a function of both visual attention and visual awareness; and lesions of parietal cortex disrupt both visual attention and awareness. Taken together, these data suggest that distributed interactions between modality-specific posterior visual cortex and frontoparietal areas subserve both visual attention and visual awareness.

Colour, form, and movement are not perceived simultaneously

Behavioural, neuro-anatomical and clinical evidence suggests that different aspects of the visual scene are processed separately, but the extent to which the processing is carried out along segregated and independent parallel pathways is still debated. Moreover, it is also unclear whether these aspects are processed at the same rate, and their neural correlates reach consciousness at the same time. An experiment investigated this issue in the case of three attributes of 2D displays: colour, form, and movement. There were three conditions, one for each possible pairing of these attributes. Stimuli were combinations of two values for each attribute (red/green, circle/square, fixed/moving). In each condition the stimuli changed twice in close temporal succession, each attribute switching asynchronously between the two possible values. The observer's task was to report which change had occurred first. Response probabilities were computed for 13 values of the asynchrony, and transformed into estimates of perception time with the help of a psychophysical model. The results showed that colour and form are processed almost simultaneously. By contrast, movement perception is delayed by about 50 ms. The implications of these findings vis à vis the so-called perceptual binding problem are discussed.

Unique hues: an old problem for a new generation

The practical success of the classical theories of colour vision, such as that of Young-Helmholtz when applied to the measurement and reproduction of colour stimuli, and that of Hering's in art and architecture, has overshadowed the fact that neither theory achieved its main goal, namely to explain colour qualities. Neither the three types of cone, nor the first opponent stages of neural processing in the retina and the lateral geniculate nucleus can serve as direct correlates to the perception of elementary, or unique colours, such as red, green, yellow and blue. While our subjective experiences of these qualities do not submit to measurement, physiological conditions that are required to perceive colours of a constant hue can be identified. For instance, a constant ratio of responses of different types of opponent cells in the retina and the lateral geniculate nucleus of primates may serve as a neurophysiological correlate of a constant hue. This is, however, not the correlate for seeing a particular hue quality, say unique red. This latter correlate, if it exists as a separable entity, must be associated with yet unidentified, higher-level neural activities. The fundamental problems encountered in relating colour qualities to neural activities are discussed and references are made to the current debate about phenomenal consciousness.

Slow and fast visual motion channels have independent binocular-rivalry stages

We have previously reported a transparent motion after-effect indicating that the human visual system comprises separate slow and fast motion channels. Here, we report that the presentation of a fast motion in one eye and a slow motion in the other eye does not result in binocular rivalry but in a clear percept of transparent motion. We call this new visual phenomenon 'dichoptic motion transparency' (DMT). So far only the DMT phenomenon and the two motion after-effects (the 'classical' motion after-effect, seen after motion adaptation on a static test pattern, and the dynamic motion after-effect, seen on a dynamic-noise test pattern) appear to isolate the channels completely. The speed ranges of the slow and fast channels overlap strongly and are observer dependent. A model is presented that links after-effect durations of an observer to the probability of rivalry or DMT as a function of dichoptic velocity combinations. Model results support the assumption of two highly independent channels showing only within-channel rivalry, and no rivalry or after-effect interactions between the channels. The finding of two independent motion vision channels, each with a separate rivalry stage and a private line to conscious perception, might be helpful in visualizing or analysing pathways to consciousness.

Consciousness

Until recently, most neuroscientists did not regard consciousness as a suitable topic for scientific investigation. This reluctance was based on certain philosophical mistakes, primarily the mistake of supposing that the subjectivity of consciousness made it beyond the reach of an objective science. Once we see that consciousness is a biological phenomenon like any other, then it can be investigated neurobiologically. Consciousness is entirely caused by neurobiological processes and is realized in brain structures. The essential trait of consciousness that we need to explain is unified qualitative subjectivity. Consciousness thus differs from other biological phenomena in that it has a subjective or first-person ontology, but this subjective ontology does not prevent us from having an epistemically objective science of consciousness. We need to overcome the philosophical tradition that treats the mental and the physical as two distinct metaphysical realms. Two common approaches to consciousness are those that adopt the building block model, according to which any conscious field is made of its various parts, and the unified field model, according to which we should try to explain the unified character of subjective states of consciousness. These two approaches are discussed and reasons are given for preferring the unified field theory to the building block model. Some relevant research on consciousness involves the subjects of blindsight, the split-brain experiments, binocular rivalry, and gestalt switching.

The architecture of the colour centre in the human visual brain: new results and a review

We have used the technique of functional magnetic resonance imaging (fMRI) and a variety of colour paradigms to activate the human brain regions selective for colour. We show here that the region defined previously [Lueck et al. (1989) Nature, 340, 386-389; Zeki et al. (1991) J. Neurosci., 11, 641-649; McKeefry & Zeki (1997) Brain, 120, 2229-2242] as the human colour centre consists of two subdivisions, a posterior one, which we call V4 and an anterior one, which we refer to as V4alpha, the two together being part of the V4-complex. The posterior area is retinotopically organized while the anterior is not. We discuss our new findings in the context of previous studies of the cortical colour processing system in humans and monkeys. Our new insight into the organization of the colour centre in the human brain may also account for the variability in both severity and degree of recovery from lesions producing cerebral colour blindness (achromatopsia).

Binding visual features during high-rate serial presentation

The neural mechanism supporting performance during single and feature conjunction detection was investigated using event-related brain potentials. In different blocks of trials, participants responded to visual targets defined by one of two colors, one of two orientations, or both color and orientation. Participants were faster and more accurate in detecting targets defined by a single feature than for targets defined by a conjunction of features. Compared with the single feature conditions, conjunction targets were associated with enhanced negativity between 230 and 270 ms post-stimulus and showed a delayed P3 latency. The relative timing of feature specific attention effects isolated in difference potential shows that feature conjunction occurs concurrently with the analysis of single features.