How does binocular rivalry emerge from cortical mechanisms of 3-D vision?

Adaptation in the sensory cortex drives bistable switching during auditory stream segregation

Current theories of perception emphasize the role of neural adaptation, inhibitory competition, and noise as key components that lead to switches in perception. Supporting evidence comes from neurophysiological findings of specific neural signatures in modality-specific and supramodal brain areas that appear to be critical to switches in perception. We used functional magnetic resonance imaging to study brain activity around the time of switches in perception while participants listened to a bistable auditory stream segregation stimulus, which can be heard as one integrated stream of tones or two segregated streams of tones. The auditory thalamus showed more activity around the time of a switch from segregated to integrated compared to time periods of stable perception of integrated; in contrast, the rostral anterior cingulate cortex and the inferior parietal lobule showed more activity around the time of a switch from integrated to segregated compared to time periods of stable perception of segregated streams, consistent with prior findings of asymmetries in brain activity depending on the switch direction. In sound-responsive areas in the auditory cortex, neural activity increased in strength preceding switches in perception and declined in strength over time following switches in perception. Such dynamics in the auditory cortex are consistent with the role of adaptation proposed by computational models of visual and auditory bistable switching, whereby the strength of neural activity decreases following a switch in perception, which eventually destabilizes the current percept enough to lead to a switch to an alternative percept.

Perceptual Grouping During Binocular Rivalry in Mild Glaucoma

Purpose

This study tested perceptual grouping during binocular rivalry to probe the strength of neural connectivity of the visual cortex involved in early visual processing in patients with mild glaucoma.

Methods

Seventeen patients with mild glaucoma with no significant visual field defects and 14 healthy controls participated. Rivalry stimuli were 1.8°-diameter discs, containing horizontal or vertical sine-wave gratings, viewed dichoptically. To test the grouping, two spatially separated identical stimuli were presented eccentrically to the same or different eyes and to the same or different hemifields. The outcome measures were the time of exclusive dominance of the grouped percept (i.e., percept with synchronized orientations), the rivalry rate, and the epochs of exclusive dominance.

Results

For both groups, the grouping occurred primarily for the matching orientations in the same eye/same hemifield (MO SE/SH) and for the matching orientations in the same eye/different hemifield (MO SE/DH) conditions. Time dominance of the grouped percept of the glaucoma group was similar to that of the control group in all conditions. The rivalry rates in the MO SE/SH and MO SE/DH conditions were significantly larger in the control group than in the glaucoma group. The epochs of exclusive dominance of the grouped percept in the MO SE/SH condition were a median of 48-ms longer for the control group, but a median of 116-ms shorter for the glaucoma group when compared to those in the MO SE/DH condition.

Conclusion

Patients with mild glaucoma show clear impairments in binocular rivalry while evidence for deficits in perceptual grouping could be inferred only indirectly. If these deficits truly exist, they may have implications for higher levels of visual processing, such as object recognition and scene segmentation, but these predictions remain to be tested in future studies.

The Perceptual Magic of Binocular Rivalry

Binocular rivalry (BR) refers to the spontaneous, unpredictable fluctuations in visual awareness provoked by dissimilar stimulation of the two eyes. Reports of the phenomenon date back several centuries, but interest in BR has exploded in recent years as researchers in diverse disciplines—psychology, neuroscience, medicine, philosophy—have found reasons to study it. New ideas about BR have emerged, sparking controversies about its neural bases, which may be resolved thanks to new methodological developments. This essay provides a synopsis of some key empirically determined aspects of BR as well as an overview of theoretical developments in this field. Work published during the past decade or so is emphasized (and explicitly referenced); earlier key findings are mentioned and referenced in the annotated bibliography included in the Supplemental Material.

A Canonical Laminar Neocortical Circuit Whose Bottom-Up, Horizontal, and Top-Down Pathways Control Attention, Learning, and Prediction

All perceptual and cognitive circuits in the human cerebral cortex are organized into layers. Specializations of a canonical laminar network of bottom-up, horizontal, and top-down pathways carry out multiple kinds of biological intelligence across different neocortical areas. This article describes what this canonical network is and notes that it can support processes as different as 3D vision and figure-ground perception; attentive category learning and decision-making; speech perception; and cognitive working memory (WM), planning, and prediction. These processes take place within and between multiple parallel cortical streams that obey computationally complementary laws. The interstream interactions that are needed to overcome these complementary deficiencies mix cell properties so thoroughly that some authors have noted the difficulty of determining what exactly constitutes a cortical stream and the differences between streams. The models summarized herein explain how these complementary properties arise, and how their interstream interactions overcome their computational deficiencies to support effective goal-oriented behaviors.

A Unified Neural Theory of Conscious Seeing, Hearing, Feeling, and Knowing

Adaptive Resonance Theory does more than satisfy 'hard criteria' for ToCs.

Individual differences point to two separate processes involved in the resolution of binocular rivalry

Although binocular rivalry is different from other perceptually bistable phenomena in requiring interocular conflict, it also shares numerous features with those phenomena. This raises the question of whether, and to what extent, the neural bases of binocular rivalry and other bistable phenomena overlap. Here we examine this question using an individual-differences approach. In a first experiment, observers reported perception during four binocular rivalry tasks that differed in the features and retinal locations of the stimuli used. Perceptual dominance durations were highly correlated when compared between stimuli that differed in location only. Correlations were substantially weaker, however, when comparing stimuli comprised of different features. Thus, individual differences in binocular-rivalry perception partly reflect a feature-specific factor that is not shared among all variants of binocular rivalry. Our second experiment again included several binocular rivalry variants, but also a different form of bistability: moving plaid rivalry. Correlations in dominance durations between binocular rivalry variants that differed in feature content were again modest. Moreover, and surprisingly, correlations between binocular rivalry and moving plaid rivalry were of similar magnitude. This indicates a second, more general, factor underlying individual differences in binocular rivalry perception: one that is shared across binocular rivalry and moving plaid rivalry. We propose that the first, feature-specific factor corresponds to feature-tuned mechanisms involved in the treatment of interocular conflict, whereas the second, general factor corresponds to mechanisms involved in representing surfaces. These latter mechanisms would operate at a binocular level and be central to both binocular rivalry and other forms of bistability.

Developmental Designs and Adult Functions of Cortical Maps in Multiple Modalities: Perception, Attention, Navigation, Numbers, Streaming, Speech, and Cognition

This article unifies neural modeling results that illustrate several basic design principles and mechanisms that are used by advanced brains to develop cortical maps with multiple psychological functions. One principle concerns how brains use a strip map that simultaneously enables one feature to be represented throughout its extent, as well as an ordered array of another feature at different positions of the strip. Strip maps include circuits to represent ocular dominance and orientation columns, place-value numbers, auditory streams, speaker-normalized speech, and cognitive working memories that can code repeated items. A second principle concerns how feature detectors for multiple functions develop in topographic maps, including maps for optic flow navigation, reinforcement learning, motion perception, and category learning at multiple organizational levels. A third principle concerns how brains exploit a spatial gradient of cells that respond at an ordered sequence of different rates. Such a rate gradient is found along the dorsoventral axis of the entorhinal cortex, whose lateral branch controls the development of time cells, and whose medial branch controls the development of grid cells. Populations of time cells can be used to learn how to adaptively time behaviors for which a time interval of hundreds of milliseconds, or several seconds, must be bridged, as occurs during trace conditioning. Populations of grid cells can be used to learn hippocampal place cells that represent the large spaces in which animals navigate. A fourth principle concerns how and why all neocortical circuits are organized into layers, and how functionally distinct columns develop in these circuits to enable map development. A final principle concerns the role of Adaptive Resonance Theory top-down matching and attentional circuits in the dynamic stabilization of early development and adult learning. Cortical maps are modeled in visual, auditory, temporal, parietal, prefrontal, entorhinal, and hippocampal cortices.

Novel method to measure temporal windows based on eye movements during viewing of the Necker cube

Bistable stimuli can give rise to two different interpretations between which our perception will alternate. Recent results showed a strong coupling between eye movements and reports of perceptual alternations with motion stimuli, which provides useful tools to objectively assess perceptual alternations. However, motion might entrain eye movements, and here we check with a static picture, the Necker cube, whether eye movements and perceptual reports (manual responses) reveal similar or different alternation rates, and similar or different sensitivity to attention manipulations. Using a cluster analysis, ocular temporal windows were defined based on the dynamics of ocular fixations during viewing of the Necker cube and compared to temporal windows extracted from manual responses. Ocular temporal windows were measured also with a control condition, where the physical stimulus presented to viewers alternated between two non-ambiguous versions of the Necker cube. Attention was manipulated by asking subjects to either report spontaneous alternations, focus on one percept, or switch as fast as possible between percepts. The validity of the ocular temporal windows was confirmed by the correspondence between ocular fixations when the physical stimulus changed and when the bistable Necker cube was presented. Ocular movements defined smaller time windows than time windows extracted from manual responses. The number of manual and ocular windows both increased between the spontaneous condition and the switch condition. However, only manual, and not ocular windows, increased in duration in the focus condition. Manual responses involve decisional mechanisms, and they may be decoupled from automatic oscillations between the two percepts, as suggested by the fact that both the number and duration of ocular windows remained stable between the spontaneous and focus conditions. In all, the recording of eye movements provides an objective measure of time windows, and reveals faster perceptual alternations with the Necker cube and less sensitivity to attention manipulations than manual responses.

Temporal dynamics of binocular integration in primary visual cortex

Whenever we open our eyes, our brain quickly integrates the two eyes' perspectives into a combined view. This process of binocular integration happens so rapidly that even incompatible stimuli are briefly fused before one eye's view is suppressed in favor of the other (binocular rivalry). The neuronal basis for this brief period of fusion during incompatible binocular stimulation is unclear. Neuroanatomically, the eyes provide two largely separate streams of information that are integrated into a binocular response by the primary visual cortex (V1). However, the temporal dynamics underlying the formation of this binocular response are largely unknown. To address this question, we examined the temporal profile of binocular responses in V1 of fixating monkeys. We found that V1 processes binocular stimuli in a dynamic sequence that comprises at least two distinct temporal phases. An initial transient phase is characterized by enhanced spiking responses for both compatible and incompatible binocular stimuli compared to monocular stimulation. This transient is followed by a sustained response that differed markedly between congruent and incongruent binocular stimulation. Specifically, incompatible binocular stimulation resulted in overall response reduction relative to monocular stimulation (binocular suppression). In contrast, responses to compatible stimuli were either suppressed or enhanced (binocular facilitation) depending on the neurons' ocularity (selectivity for one eye over the other) and laminar location. These results suggest that binocular integration in V1 occurs in at least two sequential steps that comprise initial additive combination of the two eyes' signals followed by widespread differentiation between binocular concordance and discordance.

Do early neural correlates of visual consciousness show the oblique effect? A binocular rivalry and event-related potential study

When dissimilar images are presented one to each eye, we do not see both images; rather, we see one at a time, alternating unpredictably. This is called binocular rivalry, and it has recently been used to study brain processes that correlate with visual consciousness, because perception changes without any change in the sensory input. Such studies have used various types of images, but the most popular have been gratings: sets of bright and dark lines of orthogonal orientations presented one to each eye. We studied whether using cardinal rival gratings (vertical, 0°, and horizontal, 90°) versus oblique rival gratings (left-oblique, -45°, and right-oblique, 45°) influences early neural correlates of visual consciousness, because of the oblique effect: the tendency for visual performance to be greater for cardinal gratings than for oblique gratings. Participants viewed rival gratings and pressed keys indicating which of the two gratings they perceived, was dominant. Next, we changed one of the gratings to match the grating shown to the other eye, yielding binocular fusion. Participants perceived the rivalry-to-fusion change to the dominant grating and not to the other, suppressed grating. Using event-related potentials (ERPs), we found neural correlates of visual consciousness at the P1 for both sets of gratings, as well as at the P1-N1 for oblique gratings, and we found a neural correlate of the oblique effect at the N1, but only for perceived changes. These results show that the P1 is the earliest neural activity associated with visual consciousness and that visual consciousness might be necessary to elicit the oblique effect.

Acetylcholine Neuromodulation in Normal and Abnormal Learning and Memory: Vigilance Control in Waking, Sleep, Autism, Amnesia and Alzheimer's Disease

Adaptive Resonance Theory, or ART, is a neural model that explains how normal and abnormal brains may learn to categorize and recognize objects and events in a changing world, and how these learned categories may be remembered for a long time. This article uses ART to propose and unify the explanation of diverse data about normal and abnormal modulation of learning and memory by acetylcholine (ACh). In ART, vigilance control determines whether learned categories will be general and abstract, or specific and concrete. ART models how vigilance may be regulated by ACh release in layer 5 neocortical cells by influencing after-hyperpolarization (AHP) currents. This phasic ACh release is mediated by cells in the nucleus basalis (NB) of Meynert that are activated by unexpected events. The article additionally discusses data about ACh-mediated tonic control of vigilance. ART proposes that there are often dynamic breakdowns of tonic control in mental disorders such as autism, where vigilance remains high, and medial temporal amnesia, where vigilance remains low. Tonic control also occurs during sleep-wake cycles. Properties of Up and Down states during slow wave sleep arise in ACh-modulated laminar cortical ART circuits that carry out processes in awake individuals of contrast normalization, attentional modulation, decision-making, activity-dependent habituation, and mismatch-mediated reset. These slow wave sleep circuits interact with circuits that control circadian rhythms and memory consolidation. Tonic control properties also clarify how Alzheimer's disease symptoms follow from a massive structural degeneration that includes undermining vigilance control by ACh in cortical layers 3 and 5. Sleep disruptions before and during Alzheimer's disease, and how they contribute to a vicious cycle of plaque formation in layers 3 and 5, are also clarified from this perspective.

Stereopsis deficits in patients with schizophrenia in a Han Chinese population

Although cognitive and sensory deficits have been identified as a core feature of schizophrenia, only a small portion of visual sensorium has been explored. To date, studies on visual system of three-dimensional percepts based on two-dimensional information still are limited. This study is the first to examine the integrity of stereopsis of schizophrenia in a Han Chinese population, and to further investigate the correlation of stereopsis with clinical symptoms. 100 patients with schizophrenia and 80 healthy controls were recruited. We assessed stereoacuity using the Titmus Stereopsis Test and clinical symptoms using Chinese versions of the Scales for the Assessment of Positive and Negative Symptoms (SAPS and SANS). There was a significant difference in log seconds of arc between two groups (p < 0.0001). The percentage of patients with correct stereopsis detection was significantly reduced at 400, 200, 140, 100, 80, 60, 50, and 40 seconds of arc than healthy controls (all, p < 0.01). Log seconds of arc in patients was not correlated with total scores and subscores of SAPS and SANS (all, p > 0.05). Our findings support that patients with schizophrenia have a marked deficit of stereopsis in a Han Chinese population. However, clinical symptoms do not influence stereopsis of schizophrenia.

Towards solving the hard problem of consciousness: The varieties of brain resonances and the conscious experiences that they support

The hard problem of consciousness is the problem of explaining how we experience qualia or phenomenal experiences, such as seeing, hearing, and feeling, and knowing what they are. To solve this problem, a theory of consciousness needs to link brain to mind by modeling how emergent properties of several brain mechanisms interacting together embody detailed properties of individual conscious psychological experiences. This article summarizes evidence that Adaptive Resonance Theory, or ART, accomplishes this goal. ART is a cognitive and neural theory of how advanced brains autonomously learn to attend, recognize, and predict objects and events in a changing world. ART has predicted that "all conscious states are resonant states" as part of its specification of mechanistic links between processes of consciousness, learning, expectation, attention, resonance, and synchrony. It hereby provides functional and mechanistic explanations of data ranging from individual spikes and their synchronization to the dynamics of conscious perceptual, cognitive, and cognitive-emotional experiences. ART has reached sufficient maturity to begin classifying the brain resonances that support conscious experiences of seeing, hearing, feeling, and knowing. Psychological and neurobiological data in both normal individuals and clinical patients are clarified by this classification. This analysis also explains why not all resonances become conscious, and why not all brain dynamics are resonant. The global organization of the brain into computationally complementary cortical processing streams (complementary computing), and the organization of the cerebral cortex into characteristic layers of cells (laminar computing), figure prominently in these explanations of conscious and unconscious processes. Alternative models of consciousness are also discussed.

Phoneme restoration and empirical coverage of Interactive Activation and Adaptive Resonance models of human speech processing

Magnuson [J. Acoust. Soc. Am. 137, 1481-1492 (2015)] makes claims for Interactive Activation (IA) models and against Adaptive Resonance Theory (ART) models of speech perception. Magnuson also presents simulations that claim to show that the TRACE model can simulate phonemic restoration, which was an explanatory target of the cARTWORD ART model. The theoretical analysis and review herein show that these claims are incorrect. More generally, the TRACE and cARTWORD models illustrate two diametrically opposed types of neural models of speech and language. The TRACE model embodies core assumptions with no analog in known brain processes. The cARTWORD model defines a hierarchy of cortical processing regions whose networks embody cells in laminar cortical circuits as part of the paradigm of laminar computing. cARTWORD further develops ART speech and language models that were introduced in the 1970s. It builds upon Item-Order-Rank working memories, which activate learned list chunks that unitize sequences to represent phonemes, syllables, and words. Psychophysical and neurophysiological data support Item-Order-Rank mechanisms and contradict TRACE representations of time, temporal order, silence, and top-down processing that exhibit many anomalous properties, including hallucinations of non-occurring future phonemes. Computer simulations of the TRACE model are presented that demonstrate these failures.

Neural Computation of Surface Border Ownership and Relative Surface Depth from Ambiguous Contrast Inputs

The segregation of image parts into foreground and background is an important aspect of the neural computation of 3D scene perception. To achieve such segregation, the brain needs information about border ownership; that is, the belongingness of a contour to a specific surface represented in the image. This article presents psychophysical data derived from 3D percepts of figure and ground that were generated by presenting 2D images composed of spatially disjoint shapes that pointed inward or outward relative to the continuous boundaries that they induced along their collinear edges. The shapes in some images had the same contrast (black or white) with respect to the background gray. Other images included opposite contrasts along each induced continuous boundary. Psychophysical results demonstrate conditions under which figure-ground judgment probabilities in response to these ambiguous displays are determined by the orientation of contrasts only, not by their relative contrasts, despite the fact that many border ownership cells in cortical area V2 respond to a preferred relative contrast. Studies are also reviewed in which both polarity-specific and polarity-invariant properties obtain. The FACADE and 3D LAMINART models are used to explain these data.

Cortical Dynamics of Figure-Ground Separation in Response to 2D Pictures and 3D Scenes: How V2 Combines Border Ownership, Stereoscopic Cues, and Gestalt Grouping Rules

The FACADE model, and its laminar cortical realization and extension in the 3D LAMINART model, have explained, simulated, and predicted many perceptual and neurobiological data about how the visual cortex carries out 3D vision and figure-ground perception, and how these cortical mechanisms enable 2D pictures to generate 3D percepts of occluding and occluded objects. In particular, these models have proposed how border ownership occurs, but have not yet explicitly explained the correlation between multiple properties of border ownership neurons in cortical area V2 that were reported in a remarkable series of neurophysiological experiments by von der Heydt and his colleagues; namely, border ownership, contrast preference, binocular stereoscopic information, selectivity for side-of-figure, Gestalt rules, and strength of attentional modulation, as well as the time course during which such properties arise. This article shows how, by combining 3D LAMINART properties that were discovered in two parallel streams of research, a unified explanation of these properties emerges. This explanation proposes, moreover, how these properties contribute to the generation of consciously seen 3D surfaces. The first research stream models how processes like 3D boundary grouping and surface filling-in interact in multiple stages within and between the V1 interblob—V2 interstripe—V4 cortical stream and the V1 blob—V2 thin stripe—V4 cortical stream, respectively. Of particular importance for understanding figure-ground separation is how these cortical interactions convert computationally complementary boundary and surface mechanisms into a consistent conscious percept, including the critical use of surface contour feedback signals from surface representations in V2 thin stripes to boundary representations in V2 interstripes. Remarkably, key figure-ground properties emerge from these feedback interactions. The second research stream shows how cells that compute absolute disparity in cortical area V1 are transformed into cells that compute relative disparity in cortical area V2. Relative disparity is a more invariant measure of an object's depth and 3D shape, and is sensitive to figure-ground properties.

Resonant Cholinergic Dynamics in Cognitive and Motor Decision-Making: Attention, Category Learning, and Choice in Neocortex, Superior Colliculus, and Optic Tectum

Freely behaving organisms need to rapidly calibrate their perceptual, cognitive, and motor decisions based on continuously changing environmental conditions. These plastic changes include sharpening or broadening of cognitive and motor attention and learning to match the behavioral demands that are imposed by changing environmental statistics. This article proposes that a shared circuit design for such flexible decision-making is used in specific cognitive and motor circuits, and that both types of circuits use acetylcholine to modulate choice selectivity. Such task-sensitive control is proposed to control thalamocortical choice of the critical features that are cognitively attended and that are incorporated through learning into prototypes of visual recognition categories. A cholinergically-modulated process of vigilance control determines if a recognition category and its attended features are abstract (low vigilance) or concrete (high vigilance). Homologous neural mechanisms of cholinergic modulation are proposed to focus attention and learn a multimodal map within the deeper layers of superior colliculus. This map enables visual, auditory, and planned movement commands to compete for attention, leading to selection of a winning position that controls where the next saccadic eye movement will go. Such map learning may be viewed as a kind of attentive motor category learning. The article hereby explicates a link between attention, learning, and cholinergic modulation during decision making within both cognitive and motor systems. Homologs between the mammalian superior colliculus and the avian optic tectum lead to predictions about how multimodal map learning may occur in the mammalian and avian brain and how such learning may be modulated by acetycholine.

Cortical Correlates of Low-Level Perception: From Neural Circuits to Percepts

Low-level perception results from neural-based computations, which build a multimodal skeleton of unconscious or self-generated inferences on our environment. This review identifies bottleneck issues concerning the role of early primary sensory cortical areas, mostly in rodent and higher mammals (cats and non-human primates), where perception substrates can be searched at multiple scales of neural integration. We discuss the limitation of purely bottom-up approaches for providing realistic models of early sensory processing and the need for identification of fast adaptive processes, operating within the time of a percept. Future progresses will depend on the careful use of comparative neuroscience (guiding the choices of experimental models and species adapted to the questions under study), on the definition of agreed-upon benchmarks for sensory stimulation, on the simultaneous acquisition of neural data at multiple spatio-temporal scales, and on the in vivo identification of key generic integration and plasticity algorithms validated experimentally and in simulations.

The 'laws' of binocular rivalry: 50 years of Levelt's propositions

It has been fifty years since Levelt's monograph On Binocular Rivalry (1965) was published, but its four propositions that describe the relation between stimulus strength and the phenomenology of binocular rivalry remain a benchmark for theorists and experimentalists even today. In this review, we will revisit the original conception of the four propositions and the scientific landscape in which this happened. We will also provide a brief update concerning distributions of dominance durations, another aspect of Levelt's monograph that has maintained a prominent presence in the field. In a critical evaluation of Levelt's propositions against current knowledge of binocular rivalry we will then demonstrate that the original propositions are not completely compatible with what is known today, but that they can, in a straightforward way, be modified to encapsulate the progress that has been made over the past fifty years. The resulting modified, propositions are shown to apply to a broad range of bistable perceptual phenomena, not just binocular rivalry, and they allow important inferences about the underlying neural systems. We argue that these inferences reflect canonical neural properties that play a role in visual perception in general, and we discuss ways in which future research can build on the work reviewed here to attain a better understanding of these properties.

Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements

How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.

How the venetian blind percept emerges from the laminar cortical dynamics of 3D vision

The 3D LAMINART model of 3D vision and figure-ground perception is used to explain and simulate a key example of the Venetian blind effect and to show how it is related to other well-known perceptual phenomena such as Panum's limiting case. The model proposes how lateral geniculate nucleus (LGN) and hierarchically organized laminar circuits in cortical areas V1, V2, and V4 interact to control processes of 3D boundary formation and surface filling-in that simulate many properties of 3D vision percepts, notably consciously seen surface percepts, which are predicted to arise when filled-in surface representations are integrated into surface-shroud resonances between visual and parietal cortex. Interactions between layers 4, 3B, and 2/3 in V1 and V2 carry out stereopsis and 3D boundary formation. Both binocular and monocular information combine to form 3D boundary and surface representations. Surface contour surface-to-boundary feedback from V2 thin stripes to V2 pale stripes combines computationally complementary boundary and surface formation properties, leading to a single consistent percept, while also eliminating redundant 3D boundaries, and triggering figure-ground perception. False binocular boundary matches are eliminated by Gestalt grouping properties during boundary formation. In particular, a disparity filter, which helps to solve the Correspondence Problem by eliminating false matches, is predicted to be realized as part of the boundary grouping process in layer 2/3 of cortical area V2. The model has been used to simulate the consciously seen 3D surface percepts in 18 psychophysical experiments. These percepts include the Venetian blind effect, Panum's limiting case, contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and perceptual closure. These model mechanisms have also simulated properties of 3D neon color spreading, binocular rivalry, 3D Necker cube, and many examples of 3D figure-ground separation.

Real-time learning of predictive recognition categories that chunk sequences of items stored in working memory

How are sequences of events that are temporarily stored in a cognitive working memory unitized, or chunked, through learning? Such sequential learning is needed by the brain in order to enable language, spatial understanding, and motor skills to develop. In particular, how does the brain learn categories, or list chunks, that become selectively tuned to different temporal sequences of items in lists of variable length as they are stored in working memory, and how does this learning process occur in real time? The present article introduces a neural model that simulates learning of such list chunks. In this model, sequences of items are temporarily stored in an Item-and-Order, or competitive queuing, working memory before learning categorizes them using a categorization network, called a Masking Field, which is a self-similar, multiple-scale, recurrent on-center off-surround network that can weigh the evidence for variable-length sequences of items as they are stored in the working memory through time. A Masking Field hereby activates the learned list chunks that represent the most predictive item groupings at any time, while suppressing less predictive chunks. In a network with a given number of input items, all possible ordered sets of these item sequences, up to a fixed length, can be learned with unsupervised or supervised learning. The self-similar multiple-scale properties of Masking Fields interacting with an Item-and-Order working memory provide a natural explanation of George Miller's Magical Number Seven and Nelson Cowan's Magical Number Four. The article explains why linguistic, spatial, and action event sequences may all be stored by Item-and-Order working memories that obey similar design principles, and thus how the current results may apply across modalities. Item-and-Order properties may readily be extended to Item-Order-Rank working memories in which the same item can be stored in multiple list positions, or ranks, as in the list ABADBD. Comparisons with other models, including TRACE, MERGE, and TISK, are made.

Predicting visual consciousness electrophysiologically from intermittent binocular rivalry

PURPOSE

We sought brain activity that predicts visual consciousness.

METHODS

We used electroencephalography (EEG) to measure brain activity to a 1000-ms display of sine-wave gratings, oriented vertically in one eye and horizontally in the other. This display yields binocular rivalry: irregular alternations in visual consciousness between the images viewed by the eyes. We replaced both gratings with 200 ms of darkness, the gap, before showing a second display of the same rival gratings for another 1000 ms. We followed this by a 1000-ms mask then a 2000-ms inter-trial interval (ITI). Eleven participants pressed keys after the second display in numerous trials to say whether the orientation of the visible grating changed from before to after the gap or not. Each participant also responded to numerous non-rivalry trials in which the gratings had identical orientations for the two eyes and for which the orientation of both either changed physically after the gap or did not.

RESULTS

We found that greater activity from lateral occipital-parietal-temporal areas about 180 ms after initial onset of rival stimuli predicted a change in visual consciousness more than 1000 ms later, on re-presentation of the rival stimuli. We also found that less activity from parietal, central, and frontal electrodes about 400 ms after initial onset of rival stimuli predicted a change in visual consciousness about 800 ms later, on re-presentation of the rival stimuli. There was no such predictive activity when the change in visual consciousness occurred because the stimuli changed physically.

CONCLUSION

We found early EEG activity that predicted later visual consciousness. Predictive activity 180 ms after onset of the first display may reflect adaption of the neurons mediating visual consciousness in our displays. Predictive activity 400 ms after onset of the first display may reflect a less-reliable brain state mediating visual consciousness.

How do spatial learning and memory occur in the brain? Coordinated learning of entorhinal grid cells and hippocampal place cells

Spatial learning and memory are important for navigation and formation of episodic memories. The hippocampus and medial entorhinal cortex (MEC) are key brain areas for spatial learning and memory. Place cells in hippocampus fire whenever an animal is located in a specific region in the environment. Grid cells in the superficial layers of MEC provide inputs to place cells and exhibit remarkable regular hexagonal spatial firing patterns. They also exhibit a gradient of spatial scales along the dorsoventral axis of the MEC, with neighboring cells at a given dorsoventral location having different spatial phases. A neural model shows how a hierarchy of self-organizing maps, each obeying the same laws, responds to realistic rat trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with unimodal firing fields that fit neurophysiological data about their development in juvenile rats. The hippocampal place fields represent much larger spaces than the grid cells to support navigational behaviors. Both the entorhinal and hippocampal self-organizing maps amplify and learn to categorize the most energetic and frequent co-occurrences of their inputs. Top-down attentional mechanisms from hippocampus to MEC help to dynamically stabilize these spatial memories in both the model and neurophysiological data. Spatial learning through MEC to hippocampus occurs in parallel with temporal learning through lateral entorhinal cortex to hippocampus. These homologous spatial and temporal representations illustrate a kind of "neural relativity" that may provide a substrate for episodic learning and memory.

Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: a method for converting neural rate models into spiking models

A laminar cortical model of stereopsis and 3D surface perception is developed and simulated. The model shows how spiking neurons that interact in hierarchically organized laminar circuits of the visual cortex can generate analog properties of 3D visual percepts. The model describes how monocular and binocular oriented filtering interact with later stages of 3D boundary formation and surface filling-in in the LGN and cortical areas V1, V2, and V4. It proposes how interactions between layers 4, 3B, and 2/3 in V1 and V2 contribute to stereopsis, and how binocular and monocular information combine to form 3D boundary and surface representations. The model suggests how surface-to-boundary feedback from V2 thin stripes to pale stripes helps to explain how computationally complementary boundary and surface formation properties lead to a single consistent percept, eliminate redundant 3D boundaries, and trigger figure-ground perception. The model also shows how false binocular boundary matches may be eliminated by Gestalt grouping properties. In particular, the disparity filter, which helps to solve the correspondence problem by eliminating false matches, is realized using inhibitory interneurons as part of the perceptual grouping process by horizontal connections in layer 2/3 of cortical area V2. The 3D sLAMINART model simulates 3D surface percepts that are consciously seen in 18 psychophysical experiments. These percepts include contrast variations of dichoptic masking and the correspondence problem, the effect of interocular contrast differences on stereoacuity, Panum's limiting case, the Venetian blind illusion, stereopsis with polarity-reversed stereograms, da Vinci stereopsis, and perceptual closure. The model hereby illustrates a general method of unlumping rate-based models that use the membrane equations of neurophysiology into models that use spiking neurons, and which may be embodied in VLSI chips that use spiking neurons to minimize heat production.

Unmixing binocular signals

Incompatible images presented to the two eyes lead to perceptual oscillations in which one image at a time is visible. Early models portrayed this binocular rivalry as involving reciprocal inhibition between monocular representations of images, occurring at an early visual stage prior to binocular mixing. However, psychophysical experiments found conditions where rivalry could also occur at a higher, more abstract level of representation. In those cases, the rivalry was between image representations dissociated from eye-of-origin information, rather than between monocular representations from the two eyes. Moreover, neurophysiological recordings found the strongest rivalry correlate in inferotemporal cortex, a high-level, predominantly binocular visual area involved in object recognition, rather than early visual structures. An unresolved issue is how can the separate identities of the two images be maintained after binocular mixing in order for rivalry to be possible at higher levels? Here we demonstrate that after the two images are mixed, they can be unmixed at any subsequent stage using a physiologically plausible non-linear signal-processing algorithm, non-negative matrix factorization, previously proposed for parsing object parts during object recognition. The possibility that unmixed left and right images can be regenerated at late stages within the visual system provides a mechanism for creating various binocular representations and interactions de novo in different cortical areas for different purposes, rather than inheriting then from early areas. This is a clear example how non-linear algorithms can lead to highly non-intuitive behavior in neural information processing.

Stereopsis in drug naïve Parkinson's disease patients

BACKGROUND

Motor deficits associated with Parkinson's disease (PD) have been well described, yet little attention has been paid to non-motor symptoms, especially cortical visual dysfunction. We investigated stereopsis, as well as the relationship between stereopsis and other cognitive function, in a sample of PD patients.

METHODS

We used Titmus stereotest plates for assessing stereopsis. Fifty-nine subjects (29 PD patients and 30 normal controls) were included in this study. The included patients underwent a neurological examination, clinical rating scale and neuropsychological tests.

RESULTS

Drug naïve PD patients showed decreased stereopsis on the Titmus fly stereopsis test (Pearson χ2=23.80, p<0.001) compared to PD patients with normal stereopsis. The Hoehn-Yahr stages and Unified Parkinson's Disease Rating Scale motor scores were significantly higher in patients with PD with abnormal stereopsis than in patients with PD with normal stereopsis (p=0.026; p=0.046). The frequency of abnormal visual perception/constructive function was greater in patients with PD with abnormal stereopsis compared to patients with PD with normal stereopsis (Pearson χ2=5.11, p=0.024).

CONCLUSION

These findings suggest that stereopsis deficits and visual perception/constructive dysfunction are common in de novo PD patients.

Binocular vision

This essay reviews major developments - empirical and theoretical - in the field of binocular vision during the last 25years. We limit our survey primarily to work on human stereopsis, binocular rivalry and binocular contrast summation, with discussion where relevant of single-unit neurophysiology and human brain imaging. We identify several key controversies that have stimulated important work on these problems. In the case of stereopsis those controversies include position vs. phase encoding of disparity, dependence of disparity limits on spatial scale, role of occlusion in binocular depth and surface perception, and motion in 3D. In the case of binocular rivalry, controversies include eye vs. stimulus rivalry, role of "top-down" influences on rivalry dynamics, and the interaction of binocular rivalry and stereopsis. Concerning binocular contrast summation, the essay focuses on two representative models that highlight the evolving complexity in this field of study.

Experience-driven plasticity in binocular vision

Experience-driven neuronal plasticity allows the brain to adapt its functional connectivity to recent sensory input. Here we use binocular rivalry, an experimental paradigm in which conflicting images are presented to the individual eyes, to demonstrate plasticity in the neuronal mechanisms that convert visual information from two separated retinas into single perceptual experiences. Perception during binocular rivalry tended to initially consist of alternations between exclusive representations of monocularly defined images, but upon prolonged exposure, mixture percepts became more prevalent. The completeness of suppression, reflected in the incidence of mixture percepts, plausibly reflects the strength of inhibition that likely plays a role in binocular rivalry. Recovery of exclusivity was possible but required highly specific binocular stimulation. Documenting the prerequisites for these observed changes in perceptual exclusivity, our experiments suggest experience-driven plasticity at interocular inhibitory synapses, driven by the correlated activity (and also the lack thereof) of neurons representing the conflicting stimuli. This form of plasticity is consistent with a previously proposed but largely untested anti-Hebbian learning mechanism for inhibitory synapses in vision. Our results implicate experience-driven plasticity as one governing principle in the neuronal organization of binocular vision.

Modeling filling-in of afterimages

Van Lier, Vergeer, and Anstis (2009) reported that color information in a visual afterimage could spread across regions that were not colored in the inducing stimulus. The perceived color and shape of the afterimage could be manipulated by drawn contours that apparently trap the spread of afterimage color signals. They further hypothesized that the observed effects indicated a common mechanism for afterimage color filling-in and real-color filling-in phenomena. New simulations of the existing boundary contour system/feature contour system model of visual perception (Grossberg & Mingolla, 1985a, 1985b) demonstrate the connection between these phenomena.

Reduced models for binocular rivalry

Binocular rivalry occurs when two very different images are presented to the two eyes, but a subject perceives only one image at a given time. A number of computational models for binocular rivalry have been proposed; most can be categorised as either "rate" models, containing a small number of variables, or as more biophysically-realistic "spiking neuron" models. However, a principled derivation of a reduced model from a spiking model is lacking. We present two such derivations, one heuristic and a second using recently-developed data-mining techniques to extract a small number of "macroscopic" variables from the results of a spiking neuron model simulation. We also consider bifurcations that can occur as parameters are varied, and the role of noise in such systems. Our methods are applicable to a number of other models of interest.