Seeing Invisible Motion: A Human fMRI Study

Neural patterns of conscious visual awareness in the Riddoch syndrome

The Riddoch syndrome is one in which patients blinded by lesions to their primary visual cortex can consciously perceive visual motion in their blind field, an ability that correlates with activity in motion area V5. Our assessment of the characteristics of this syndrome in patient ST, using multimodal MRI, showed that: 1. ST's V5 is intact, receives direct subcortical input, and decodable neural patterns emerge in it only during the conscious perception of visual motion; 2. moving stimuli activate medial visual areas but, unless associated with decodable V5 activity, they remain unperceived; 3. ST's high confidence ratings when discriminating motion at chance levels, is associated with inferior frontal gyrus activity. Finally, we report that ST's Riddoch Syndrome results in hallucinatory motion with hippocampal activity as a correlate. Our results shed new light on perceptual experiences associated with this syndrome and on the neural determinants of conscious visual experience.

Neuroscientific Evidence for Processing Without Awareness

The extent to which we are affected by perceptual input of which we are unaware is widely debated. By measuring neural responses to sensory stimulation, neuroscientific data could complement behavioral results with valuable evidence. Here we review neuroscientific findings of processing of high-level information, as well as interactions with attention and memory. Although the results are mixed, we find initial support for processing object categories and words, possibly to the semantic level, as well as emotional expressions. Robust neural evidence for face individuation and integration of sentences or scenes is lacking. Attention affects the processing of stimuli that are not consciously perceived, and such stimuli may exogenously but not endogenously capture attention when relevant, and be maintained in memory over time. Sources of inconsistency in the literature include variability in control for awareness as well as individual differences, calling for future studies that adopt stricter measures of awareness and probe multiple processes within subjects.

Invisible Adaptation: The Effect of Awareness on the Strength of the Motion Aftereffect

The ability to process information despite the lack of perceptual awareness is one of the most fascinating aspects of the visual system. Such unconscious processing is often investigated using adaptation, where any presence of the former can be traced by its footprint on aftereffects following the latter. We have investigated the mechanisms of the motion aftereffect (MAE) using random dot displays of varying motion coherence as well as crowding to modulate both the physical as well as the perceptual strength of the adaptation stimulus. Perceptual strength was quantitatively measured as the performance in a forced-choice direction-discrimination task. A motion-nulling technique was used to quantitatively measure the strength of the MAE. We show that the strength of the dynamic MAE is independently influenced by both the physical stimulus strength as well as the subjective perceptual strength, with the effect of the former being more prominent than that of the latter. We further show that the MAE still persists under conditions of subthreshold perception. Our results suggest that perceptual awareness can influence the strength of visual processing, but the latter is not fully dependent on the former and can still take place at its partial or even total absence.

Stimulus dependencies of an illusory motion: Investigations of the Motion Bridging Effect

The Motion Bridging Effect (MBE) is an illusion in which a motion that is not consciously visible generates a visible motion aftereffect that is predominantly in the same direction as the adapter motion. In the initial study of the MBE (Mattler & Fendrich, 2010), a ring of 16 points was rotated at angular velocities as high as 2250°/s so that observers saw only an unbroken outline circle and performed at chance when asked to report the ring's rotation direction. However, when the rotating ring was replaced by a veridically stationary ring of 16 points, the stationary ring appeared to visibly spin to a halt, principally in the same direction as the initial ring's rotation. Here we continue to investigate the stimulus dependencies of the MBE. We find the MBE, measured by the correspondence between the direction of the invisible rotation of the spinning ring and perceived rotation of the stationary ring, increases as the number of points used to construct the rings decreases and grows stronger as the diameter of the rings get larger. We consider the potential contributions of temporal frequency, retinal eccentricity, luminance levels, and the separation between the points forming the rings as mediators of these effects. Data is discussed with regard to the detection of real movement and apparent motion. We conclude that the detection of the rapid rotation of the spinning ring is likely to be modulated by temporal frequency of luminance changes along the ring perimeter while the point-distance may modulate an apparent motion produced by the transition from the perceptually unbroken spinning ring to the point-defined stationary ring.

Crowding, visual awareness, and their respective neural loci

In peripheral vision, object identification can be impeded when a target object is flanked by other objects. This phenomenon of crowding has been attributed to basic processes associated with image encoding by the visual system, but the neural origin of crowding is not known. Determining whether crowding depends on subjective awareness of the flankers can provide information on the neural origin of crowding. However, recent studies that manipulated flanker awareness have yielded conflicting results. In the current study, we suppressed flanker awareness with two methods: interocular suppression (IOS) and adaptation-induced blindness (AIB). We tested two different types of stimuli: gratings and letters. With IOS, we found that the magnitude of crowding increased as the number of physical flankers increased, even when the observers did not report seeing any of the flankers. In contrast, when flanker awareness was manipulated with AIB, the magnitude of crowding increased with the number of perceived flankers. Our results show that whether crowding is contingent on awareness of the flankers depends on the method used to suppress awareness. In addition, our results imply that the locus of crowding is upstream from the neural locus of IOS and close to or downstream from that of AIB. Neurophysiology and neuroimaging studies jointly implicate mid-to-high level visual processing stages for IOS, while direct evidence regarding the neural locus of AIB is limited. The most consistent interpretation of our empirical findings is to place the neural locus of crowding at an early cortical site, such as V1 or V2.

Visual motion serves but is not under the purview of the dorsal pathway

Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual motion signals are swiftly transmitted via multiple non-hierarchical routes to primary motion-dedicated processing regions (MT/V5 and MST) that are not part of the dorsal pathway, and then propagated to a multiplicity of brain areas according to task demands, reaching these regions earlier than the dorsal/ventral hierarchical flow. This not only places MT/V5 at the same or even earlier visual processing stage as that of V1, but can also elucidate many findings with implications to visual awareness. While the integrity of the non-hierarchical motion pathway is necessary for all visual motion perception, it is insufficient on its own, and the transfer of visual motion signals to additional brain areas is crucial to allow the different motion perception tasks (e.g. optic flow, visuo-vestibular balance, movement observation, dynamic form detection and perception, and even reading). I argue that this lateral visual motion pathway can be distinguished from the dorsal pathway not only based on faster response latencies and distinct anatomical connections, but also based on its full field representation. I also distinguish between this primary lateral visual motion pathway sensitive to all motion in the visual field, and a much less investigated optic flow sensitive medial processing pathway (from V1 to V6 and V6A) that appears to be part of the dorsal pathway. Multiple additional predictions are provided that allow testing this proposal and distinguishing between the visual pathways.

A massively asynchronous, parallel brain

Whether the visual brain uses a parallel or a serial, hierarchical, strategy to process visual signals, the end result appears to be that different attributes of the visual scene are perceived asynchronously--with colour leading form (orientation) by 40 ms and direction of motion by about 80 ms. Whatever the neural root of this asynchrony, it creates a problem that has not been properly addressed, namely how visual attributes that are perceived asynchronously over brief time windows after stimulus onset are bound together in the longer term to give us a unified experience of the visual world, in which all attributes are apparently seen in perfect registration. In this review, I suggest that there is no central neural clock in the (visual) brain that synchronizes the activity of different processing systems. More likely, activity in each of the parallel processing-perceptual systems of the visual brain is reset independently, making of the brain a massively asynchronous organ, just like the new generation of more efficient computers promise to be. Given the asynchronous operations of the brain, it is likely that the results of activities in the different processing-perceptual systems are not bound by physiological interactions between cells in the specialized visual areas, but post-perceptually, outside the visual brain.

Area V5-a microcosm of the visual brain

Area V5 of the visual brain, first identified anatomically in 1969 as a separate visual area, is critical for the perception of visual motion. As one of the most intensively studied parts of the visual brain, it has yielded many insights into how the visual brain operates. Among these are: the diversity of signals that determine the functional capacities of a visual area; the relationship between single cell activity in a specialized visual area and perception of, and preference for, attributes of a visual stimulus; the multiple asynchronous inputs into, and outputs from, an area as well as the multiple operations that it undertakes asynchronously; the relationship between activity at given, specialized, areas of the visual brain and conscious awareness; and the mechanisms used to “bind” signals from one area with those from another, with a different specialization, to give us our unitary perception of the visual world. Hence V5 is, in a sense, a microcosm of the visual world and its study gives important insights into how the whole visual brain is organized—anatomically, functionally and perceptually.

A brain-derived metric for preferred kinetic stimuli

We here address the question of whether there is any correlation between subjective preference for simple configurations within a specific visual domain such as motion and strength of activity in visual areas in which that domain is emphasized. We prepared several distinctive patterns of dots in motion with various characteristics and asked humans to rate them according to their preference, before and while scanning the activity in their brains with functional magnetic resonance imaging. For simplicity, we restricted ourselves to motion in the fronto-parallel plane. Moving patterns produced activity in areas V1, V2, the V3 complex (V3, V3A, V3B) and V5, but only in areas V5, V3A/B and parietal cortex did the preferred kinetic patterns produce stronger activity when compared with the non-preferred ones. In addition, preferred patterns produced activity within field A1 of medial orbito-frontal cortex (mOFC), which is not otherwise activated by kinetic stimuli. Hence, for these areas, stronger neural activity correlated with subjective preference. We conclude that configurations of kinetic stimuli that are subjectively preferred correlate with stronger activity within early visual areas and within mOFC. This opens up the possibility of more detailed studies to relate subjective preferences to strength of activity in early visual areas and to relate activity in them to areas whose activity correlates with the subjective experience of beauty.

Brain activation and the locus of visual awareness

A major problem in visual neuroscience is to distinguish neuronal activity which is directly related to the conscious percept. The word "directly" is used here as opposed to an indirect relationship, as is for example the case with activity in the retina, which is produced by a stimulus in the outside world and will eventually lead to the perception of this stimulus. As for the word "related", it is used to mean activity which creates the perceptual experience or, even more extremely, activity that is the perceptual experience. The distinction between the two (is vs. creates) is not straightforward, although there might be some differences between them. Philosophers would argue that they have a different phenomenology, the percept existing only for the perceiving person, whereas the underlying neuronal activation exists for all to observe. One could go on to argue that it is actually not the neuronal activation that "everybody" observes, but each one observes his own percept of it, which is also unique and subjective. Still, the content of this percept and the one of the original stimulus are quite different. The purpose of the present review is not to dig deep into such philosophical issues, but rather to give an overview of neuroscientific approaches trying to locate the neural correlate of conscious perception.

Neural and BOLD responses across the brain

Functional Magnetic Resonance Imaging (fMRI) has quickly grown into one of the most important tools for studying brain function, especially in humans. Despite its prevalence, we still do not have a clear picture of what exactly the blood oxygenation level dependent (BOLD) signal represents or how it compares to the signals obtained with other methods (e.g., electrophysiology). We particularly refer to single neuron recordings and electroencephalography when we mention 'electrophysiological methods', given that these methods have been used for more than 50 years, and have formed the basis of much of our current understanding of brain function. Brain function involves the coordinated activity of many different areas and many different cell types that can participate in an enormous variety of processes (neural firing, inhibitory and excitatory synaptic activity, neuromodulation, oscillatory activity, etc.). Of these cells and processes, only a subset is sampled with electrophysiological techniques, and their contribution to the recorded signals is not exactly known. Functional imaging signals are driven by the metabolic needs of the active cells, and are most likely also biased toward certain cell types and certain neural processes, although we know even less about which processes actually drive the hemodynamic response. This article discusses the current status on the interpretation of the BOLD signal and how it relates to neural activity measured with electrophysiological techniques. WIREs Cogn Sci 2012, 3:75-86. doi: 10.1002/wcs.153 For further resources related to this article, please visit the WIREs website.

Neural bases of the non-conscious perception of emotional signals

Many emotional stimuli are processed without being consciously perceived. Recent evidence indicates that subcortical structures have a substantial role in this processing. These structures are part of a phylogenetically ancient pathway that has specific functional properties and that interacts with cortical processes. There is now increasing evidence that non-consciously perceived emotional stimuli induce distinct neurophysiological changes and influence behaviour towards the consciously perceived world. Understanding the neural bases of the non-conscious perception of emotional signals will clarify the phylogenetic continuity of emotion systems across species and the integration of cortical and subcortical activity in the human brain.

Enhancement of visual perception with use of dynamic cues

Institutional review board approval and signed informed consent were not needed, as medical images included in public databases were used in this study. The purpose of this study was to improve the detection of microcalcifications on mammograms and lung nodules on chest radiographs by using the dynamic cues algorithm and the motion and flickering sensitivity of the human visual system (HVS). Different sets of mammograms from the Mammographic Image Analysis Society database and chest radiographs from the Japanese Society of Radiological Technology database were presented statically, as is standard, and in a video sequence generated with the dynamic cues algorithm. Nine observers were asked to rate the presence of abnormalities with a five-point scale (1, definitely not present; 5, definitely present). The data were analyzed with receiver operating characteristic (ROC) techniques and the Dorfman-Berbaum-Metz method. The video sequence generated with the dynamic cues algorithm increased the rate of detection of microcalcifications by 10.2% (P = .002) compared with that obtained with the standard static method, as measured by the area under the ROC curve. Similar results were obtained for lung nodules, with an increase of 12.3% (P = .0054). The increase in the rate of correct detection did not come just from the image contrast change produced by the algorithm but also from the fact that image frames generated with the dynamic cues algorithm were put together in a video sequence so that the motion sensitivity of the HVS could be used to facilitate the detection of low-contrast objects.

SUPPLEMENTAL MATERIAL

http://radiology.rsnajnls.org/cgi/content/full/250/2/551/DC1.

Motion processing, directional selectivity, and conscious visual perception in the human brain

Several human and monkey studies have demonstrated a close relationship between motion perception and activation of area V5, leading to the general view that activity in this area correlates with the subjective experience of motion. In the present study, we investigate whether the responses of this area are still governed by the motion percept when the latter is in conflict with the reality of the physical visual stimulation. We simultaneously presented two different, specially designed random-dot kinematograms, one to each eye. These stimuli either both had a single direction of motion and worked in synergy, or had opposite motion directions and thus cancelled each other out perceptually. In this way, we were able to pit the visual stimulus (one vs. two stimulating directions) against the reported perception (directional motion vs. motion noise) of human volunteers during fMRI experiments. We found that a strong motion stimulus that is weakly perceived is more effective in activating V5 (as well as V3) than a weaker motion stimulus, which is nevertheless robustly perceived. Thus, contrary to the prevailing view of perception being the correlate of activity in higher visual areas, we show here that activity is instead dominated by the properties of the physical stimulus, raising the question of whether there is a subpopulation of cells in V5 whose activity is critical for generating the motion percept. In addition, our results provide the first robust evidence for the presence of directionally selective neuronal populations in human prestriate cortex.

fMRI and its interpretations: an illustration on directional selectivity in area V5/MT

fMRI is a tool to study brain function noninvasively that can reliably identify sites of neural involvement for a given task. However, to what extent can fMRI signals be related to measures obtained in electrophysiology? Can the blood-oxygen-level-dependent signal be interpreted as spatially pooled spiking activity? Here we combine knowledge from neurovascular coupling, functional imaging and neurophysiology to discuss whether fMRI has succeeded in demonstrating one of the most established functional properties in the visual brain, namely directional selectivity in the motion-processing region V5/MT+. We also discuss differences of fMRI and electrophysiology in their sensitivity to distinct physiological processes. We conclude that fMRI constitutes a complement, not a poor-resolution substitute, to invasive techniques, and that it deserves interpretations that acknowledge its stand as a separate signal.

Neural correlates of the contents of visual awareness in humans

The immediacy and directness of our subjective visual experience belies the complexity of the neural mechanisms involved, which remain incompletely understood. This review focuses on how the subjective contents of human visual awareness are encoded in neural activity. Empirical evidence to date suggests that no single brain area is both necessary and sufficient for consciousness. Instead, necessary and sufficient conditions appear to involve both activation of a distributed representation of the visual scene in primary visual cortex and ventral visual areas, plus parietal and frontal activity. The key empirical focus is now on characterizing qualitative differences in the type of neural activity in these areas underlying conscious and unconscious processing. To this end, recent progress in developing novel approaches to accurately decoding the contents of consciousness from brief samples of neural activity show great promise.

Synchronization of neural activity across cortical areas correlates with conscious perception

Subliminal stimuli can be deeply processed and activate similar brain areas as consciously perceived stimuli. This raises the question which signatures of neural activity critically differentiate conscious from unconscious processing. Transient synchronization of neural activity has been proposed as a neural correlate of conscious perception. Here we test this proposal by comparing the electrophysiological responses related to the processing of visible and invisible words in a delayed matching to sample task. Both perceived and nonperceived words caused a similar increase of local (gamma) oscillations in the EEG, but only perceived words induced a transient long-distance synchronization of gamma oscillations across widely separated regions of the brain. After this transient period of temporal coordination, the electrographic signatures of conscious and unconscious processes continue to diverge. Only words reported as perceived induced (1) enhanced theta oscillations over frontal regions during the maintenance interval, (2) an increase of the P300 component of the event-related potential, and (3) an increase in power and phase synchrony of gamma oscillations before the anticipated presentation of the test word. We propose that the critical process mediating the access to conscious perception is the early transient global increase of phase synchrony of oscillatory activity in the gamma frequency range.