Neural dynamics of motion integration and segmentation within and across apertures

Understanding visual processing of motion: completing the picture using experimentally driven computational models of MT

Computational modeling helps neuroscientists to integrate and explain experimental data obtained through neurophysiological and anatomical studies, thus providing a mechanism by which we can better understand and predict the principles of neural computation. Computational modeling of the neuronal pathways of the visual cortex has been successful in developing theories of biological motion processing. This review describes a range of computational models that have been inspired by neurophysiological experiments. Theories of local motion integration and pattern motion processing are presented, together with suggested neurophysiological experiments designed to test those hypotheses.

A Canonical Laminar Neocortical Circuit Whose Bottom-Up, Horizontal, and Top-Down Pathways Control Attention, Learning, and Prediction

All perceptual and cognitive circuits in the human cerebral cortex are organized into layers. Specializations of a canonical laminar network of bottom-up, horizontal, and top-down pathways carry out multiple kinds of biological intelligence across different neocortical areas. This article describes what this canonical network is and notes that it can support processes as different as 3D vision and figure-ground perception; attentive category learning and decision-making; speech perception; and cognitive working memory (WM), planning, and prediction. These processes take place within and between multiple parallel cortical streams that obey computationally complementary laws. The interstream interactions that are needed to overcome these complementary deficiencies mix cell properties so thoroughly that some authors have noted the difficulty of determining what exactly constitutes a cortical stream and the differences between streams. The models summarized herein explain how these complementary properties arise, and how their interstream interactions overcome their computational deficiencies to support effective goal-oriented behaviors.

Individual differences point to two separate processes involved in the resolution of binocular rivalry

Although binocular rivalry is different from other perceptually bistable phenomena in requiring interocular conflict, it also shares numerous features with those phenomena. This raises the question of whether, and to what extent, the neural bases of binocular rivalry and other bistable phenomena overlap. Here we examine this question using an individual-differences approach. In a first experiment, observers reported perception during four binocular rivalry tasks that differed in the features and retinal locations of the stimuli used. Perceptual dominance durations were highly correlated when compared between stimuli that differed in location only. Correlations were substantially weaker, however, when comparing stimuli comprised of different features. Thus, individual differences in binocular-rivalry perception partly reflect a feature-specific factor that is not shared among all variants of binocular rivalry. Our second experiment again included several binocular rivalry variants, but also a different form of bistability: moving plaid rivalry. Correlations in dominance durations between binocular rivalry variants that differed in feature content were again modest. Moreover, and surprisingly, correlations between binocular rivalry and moving plaid rivalry were of similar magnitude. This indicates a second, more general, factor underlying individual differences in binocular rivalry perception: one that is shared across binocular rivalry and moving plaid rivalry. We propose that the first, feature-specific factor corresponds to feature-tuned mechanisms involved in the treatment of interocular conflict, whereas the second, general factor corresponds to mechanisms involved in representing surfaces. These latter mechanisms would operate at a binocular level and be central to both binocular rivalry and other forms of bistability.

STDP Plasticity in TRN Within Hierarchical Spike Timing Model of Visual Information Processing

We investigated age related synaptic plasticity in thalamic reticular nucleus (TRN) as a part of visual information processing system in the brain. Simulation experiments were performed using a hierarchical spike timing neural network model in NEST simulator. The model consists of multiple layers starting with retinal photoreceptors through thalamic relay, primary visual cortex layers up to the lateral intraparietal cortex (LIP) responsible for decision making and preparation of motor response. All synaptic inter- and intra-layer connections of our model are structured according to the literature information. The present work extends the model with spike timing dependent plastic (STDP) synapses within TRN as well as from visual cortex to LIP area. Synaptic strength changes were forced by teaching signal typical for three different age groups (young, middle and elderly) determined experimentally from eye movement data collected by eye tracking device from human subjects preforming a simplified simulated visual navigation task.

Spike Timing Neural Model of Motion Perception and Decision Making

The paper presents a hierarchical spike timing neural network model developed in NEST simulator aimed to reproduce human decision making in simplified simulated visual navigation tasks. It includes multiple layers starting from retina photoreceptors and retinal ganglion cells (RGC) via thalamic relay including lateral geniculate nucleus (LGN), thalamic reticular nucleus (TRN), and interneurons (IN) mediating connections to the higher brain areas-visual cortex (V1), middle temporal (MT), and medial superior temporal (MTS) areas, involved in dorsal pathway processing of spatial and dynamic visual information. The last layer-lateral intraparietal cortex (LIP)-is responsible for decision making and organization of the subsequent motor response (saccade generation). We simulated two possible decision options having LIP layer with two sub-regions with mutual inhibitory connections whose increased firing rate corresponds to the perceptual decision about motor response-left or right saccade. Each stage of the model was tested by appropriately chosen stimuli corresponding to its selectivity to specific stimulus characteristics (orientation for V1, direction for MT, and expansion/contraction movement templates for MST, respectively). The overall model performance was tested with stimuli simulating optic flow patterns of forward self-motion on a linear trajectory to the left or to the right from straight ahead with a gaze in the direction of heading.

A biologically-based computational model of visual cortex that overcomes the X-junction illusion

The end-points of a moving bar (intrinsic terminators) contain unambiguous information that can be used to extract the bar's correct direction of motion, regardless of the orientation of the bar. However, extrinsic terminators, formed at the intersection of two overlapping bars, can result in motion signals with conflicting directions compared to those of the intrinsic terminators. Using a computational model, we propose that interactions between form and motion information may assist neurons in the motion-specific regions of primate cortex to differentiate intrinsic from extrinsic terminators. The motion processing model has two stages. The first stage is a model of V1 complex neurons, including end-stopped neurons. The resulting first stage motion signals are transmitted to the second stage, which is a model of MT neurons. In the proposed model, MT neurons additionally receive form information from neurons in V1 that are not orientation or direction selective but respond strongly to the contrast of the stimulus. These neurons have polarity-dependent center-surround receptive fields, as found in layer 4 of V1 in primates. As the inhibitory surrounds of these neurons are less activated at the intrinsic terminators, the signals generated by the end-points of the objects are stronger than the signals from the extrinsic terminators, which are inhibited by strong suppression from the surround. Therefore, the excitatory inputs received by integration MT neurons from center-surround V1 neurons enhance the unambiguous motion signals at the intrinsic terminators, which therefore dominate over the local motion signals generated at X-junctions. The results show that, despite the inability of V1 end-stopped neurons to distinguish between the two different types of terminators, center-surround V1 neurons provide the capacity for the second stage of the model to preferentially respond to the intrinsic terminators and, therefore, predict the true directions of the crossing bars.

Desirability, availability, credit assignment, category learning, and attention: Cognitive-emotional and working memory dynamics of orbitofrontal, ventrolateral, and dorsolateral prefrontal cortices

BACKGROUND

The prefrontal cortices play an essential role in cognitive-emotional and working memory processes through interactions with multiple brain regions.

METHODS

This article further develops a unified neural architecture that explains many recent and classical data about prefrontal function and makes testable predictions.

RESULTS

Prefrontal properties of desirability, availability, credit assignment, category learning, and feature-based attention are explained. These properties arise through interactions of orbitofrontal, ventrolateral prefrontal, and dorsolateral prefrontal cortices with the inferotemporal cortex, perirhinal cortex, parahippocampal cortices; ventral bank of the principal sulcus, ventral prearcuate gyrus, frontal eye fields, hippocampus, amygdala, basal ganglia, hypothalamus, and visual cortical areas V1, V2, V3A, V4, middle temporal cortex, medial superior temporal area, lateral intraparietal cortex, and posterior parietal cortex. Model explanations also include how the value of visual objects and events is computed, which objects and events cause desired consequences and which may be ignored as predictively irrelevant, and how to plan and act to realise these consequences, including how to selectively filter expected versus unexpected events, leading to movements towards, and conscious perception of, expected events. Modelled processes include reinforcement learning and incentive motivational learning; object and spatial working memory dynamics; and category learning, including the learning of object categories, value categories, object-value categories, and sequence categories, or list chunks.

CONCLUSION

This article hereby proposes a unified neural theory of prefrontal cortex and its functions.

A new principle of figure-ground segregation: The accentuation

The problem of perceptual organization was studied by Gestalt psychologists in terms of figure-ground segregation. In this paper we explore a new principle of figure-ground segregation: accentuation. We demonstrate the effectiveness of accentuation relative to other Gestalt principles, and also consider it autonomous as it can agree with or oppose them. We consider three dynamic aspects of the principle, namely: attraction, accentuation and assignment. Each creature needs to attract, fascinate, seduce, draw attention (e.g., a mate or a prey animal) or distract, refuse, dissuade, discourage, repulse (e.g., a predator). Similarly, each organism needs to accentuate, highlight, stress, underline, emphasize or distract from another. Thus, accentuation assigns meaning to a visual pattern such as a coat, a plumage or a flower. False eyes (ocelli) and dots (diematic patterns) demonstrate "deceiving camouflage by accentuation" that confuses predators/preys and hides or highlights vital body parts (butterflies/flowers). They also display the deceiving appearance and exhibition of biological fitness. The same accents may serve different or even opposite goals. We conclude that accentuation improves the adaptive fitness of organisms in multifarious ways.

Recurrent network dynamics reconciles visual motion segmentation and integration

In sensory systems, a range of computational rules are presumed to be implemented by neuronal subpopulations with different tuning functions. For instance, in primate cortical area MT, different classes of direction-selective cells have been identified and related either to motion integration, segmentation or transparency. Still, how such different tuning properties are constructed is unclear. The dominant theoretical viewpoint based on a linear-nonlinear feed-forward cascade does not account for their complex temporal dynamics and their versatility when facing different input statistics. Here, we demonstrate that a recurrent network model of visual motion processing can reconcile these different properties. Using a ring network, we show how excitatory and inhibitory interactions can implement different computational rules such as vector averaging, winner-take-all or superposition. The model also captures ordered temporal transitions between these behaviors. In particular, depending on the inhibition regime the network can switch from motion integration to segmentation, thus being able to compute either a single pattern motion or to superpose multiple inputs as in motion transparency. We thus demonstrate that recurrent architectures can adaptively give rise to different cortical computational regimes depending upon the input statistics, from sensory flow integration to segmentation.

Towards solving the hard problem of consciousness: The varieties of brain resonances and the conscious experiences that they support

The hard problem of consciousness is the problem of explaining how we experience qualia or phenomenal experiences, such as seeing, hearing, and feeling, and knowing what they are. To solve this problem, a theory of consciousness needs to link brain to mind by modeling how emergent properties of several brain mechanisms interacting together embody detailed properties of individual conscious psychological experiences. This article summarizes evidence that Adaptive Resonance Theory, or ART, accomplishes this goal. ART is a cognitive and neural theory of how advanced brains autonomously learn to attend, recognize, and predict objects and events in a changing world. ART has predicted that "all conscious states are resonant states" as part of its specification of mechanistic links between processes of consciousness, learning, expectation, attention, resonance, and synchrony. It hereby provides functional and mechanistic explanations of data ranging from individual spikes and their synchronization to the dynamics of conscious perceptual, cognitive, and cognitive-emotional experiences. ART has reached sufficient maturity to begin classifying the brain resonances that support conscious experiences of seeing, hearing, feeling, and knowing. Psychological and neurobiological data in both normal individuals and clinical patients are clarified by this classification. This analysis also explains why not all resonances become conscious, and why not all brain dynamics are resonant. The global organization of the brain into computationally complementary cortical processing streams (complementary computing), and the organization of the cerebral cortex into characteristic layers of cells (laminar computing), figure prominently in these explanations of conscious and unconscious processes. Alternative models of consciousness are also discussed.

A Possible Role for End-Stopped V1 Neurons in the Perception of Motion: A Computational Model

We present a model of the early stages of processing in the visual cortex, in particular V1 and MT, to investigate the potential role of end-stopped V1 neurons in solving the aperture problem. A hierarchical network is used in which the incoming motion signals provided by complex V1 neurons and end-stopped V1 neurons proceed to MT neurons at the next stage. MT neurons are categorized into two types based on their function: integration and segmentation. The role of integration neurons is to propagate unambiguous motion signals arriving from those V1 neurons that emphasize object terminators (e.g. corners). Segmentation neurons detect the discontinuities in the input stimulus to control the activity of integration neurons. Although the activity of the complex V1 neurons at the terminators of the object accurately represents the direction of the motion, their level of activity is less than the activity of the neurons along the edges. Therefore, a model incorporating end-stopped neurons is essential to suppress ambiguous motion signals along the edges of the stimulus. It is shown that the unambiguous motion signals at terminators propagate over the rest of the object to achieve an accurate representation of motion.

From Grouping to Coupling: A New Perceptual Organization in Vision, Psychology, and Biology

In this work, perceptual organization has been studied with the same spirit and phenomenological methods used by Gestalt psychologists. This was accomplished through new conditions that cannot be explained in terms of the classical principles of grouping. Perceptual grouping represents the way through which our visual system builds integrated elements on the basis of the maximal homogeneity among the components of the stimulus pattern. Our results demonstrated the inconsistency of organization by grouping, and more importantly, the inconsistency of the principle of similarity. On the contrary, they suggested the unique role played by the principle of dissimilarity among elements that behaves like an accent or a visual emphasis within a whole. The principle of accentuation was here considered as imparting a directional structure to the elements and to the whole object thus creating new phenomena. The salience of the resulting phenomena reveals the supremacy of dissimilarity in relation to similarity and the fact that it belongs to a further organization dynamics that we called “coupling.” In biology, coupling and its principle of accentuation are very strongly related to disruptive camouflage. Moreover, they are source of sexual attraction. They advertise the presence and elicit species identification/communication. In human beings accentuation is needed to show ourselves to others, to understand the way we dress, choose, and create clothes or invent fashion, the way we change our body accentuating several parts and hiding some others, the way we use maquillage. The existence of maquillage itself is derived from the need to accentuate something with the purpose to increase sexual attraction, to exhibit physical strength and beauty, to show or hide social status (e.g., being the king, a warrior, a priest, etc.). Last but not least, accentuation plays a basic role also in making it easier or difficult to read and understand written words.

Neural Computation of Surface Border Ownership and Relative Surface Depth from Ambiguous Contrast Inputs

The segregation of image parts into foreground and background is an important aspect of the neural computation of 3D scene perception. To achieve such segregation, the brain needs information about border ownership; that is, the belongingness of a contour to a specific surface represented in the image. This article presents psychophysical data derived from 3D percepts of figure and ground that were generated by presenting 2D images composed of spatially disjoint shapes that pointed inward or outward relative to the continuous boundaries that they induced along their collinear edges. The shapes in some images had the same contrast (black or white) with respect to the background gray. Other images included opposite contrasts along each induced continuous boundary. Psychophysical results demonstrate conditions under which figure-ground judgment probabilities in response to these ambiguous displays are determined by the orientation of contrasts only, not by their relative contrasts, despite the fact that many border ownership cells in cortical area V2 respond to a preferred relative contrast. Studies are also reviewed in which both polarity-specific and polarity-invariant properties obtain. The FACADE and 3D LAMINART models are used to explain these data.

Plaid Motion Rivalry: Correlates with Binocular Rivalry and Positive Mood State

Recently Hupé and Rubin (2003, Vision Research43 531–548) re-introduced the plaid as a form of perceptual rivalry by using two sets of drifting gratings behind a circular aperture to produce quasi-regular perceptual alternations between a coherent moving plaid of diamond-shaped intersections and the two sets of component ‘sliding’ gratings. We call this phenomenon plaid motion rivalry (PMR), and have compared its temporal dynamics with those of binocular rivalry in a sample of subjects covering a wide range of perceptual alternation rates. In support of the proposal that all rivalries may be mediated by a common switching mechanism, we found a high correlation between alternation rates induced by PMR and binocular rivalry. In keeping with a link discovered between the phase of rivalry and mood, we also found a link between PMR and an individual's mood state that is consistent with suggestions that each opposing phase of rivalry is associated with one or the other hemisphere, with the ‘diamonds’ phase of PMR linked with the ‘positive’ left hemisphere.

The Evolution of Explanations of a Perceptual Phenomenon: A Case History Using the Ternus Effect

The Ternus effect involves a multi-element stimulus that can lead to either of two different percepts of apparent movement depending upon a variety of stimulus conditions. Since Ternus's 1926 discussion of this phenomenon, many researchers have attempted to explain it. We examine the history of explanations of the Ternus effect and show that they have evolved to contemporary theoretical positions that are very similar to Ternus's own ideas. Additionally, we describe a new experiment showing that theoretical positions that emphasize element grouping and element identity within groups can predict the effects of certain stimulus manipulations on the Ternus effect.

A review of human sensory dynamics for application to models of driver steering and speed control

In comparison with the high level of knowledge about vehicle dynamics which exists nowadays, the role of the driver in the driver-vehicle system is still relatively poorly understood. A large variety of driver models exist for various applications; however, few of them take account of the driver's sensory dynamics, and those that do are limited in their scope and accuracy. A review of the literature has been carried out to consolidate information from previous studies which may be useful when incorporating human sensory systems into the design of a driver model. This includes information on sensory dynamics, delays, thresholds and integration of multiple sensory stimuli. This review should provide a basis for further study into sensory perception during driving.

How Does the Cerebral Cortex Work? Development, Learning, Attention, and 3-D Vision by Laminar Circuits of Visual Cortex

A key goal of behavioral and cognitive neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress toward explaining how the visual cortex sees. Visual cortex, like many parts of perceptual and cognitive neocortex, is organized into six main layers of cells, as well as characteristic sublamina. Here it is proposed how these layered circuits help to realize processes of development, learning, perceptual grouping, attention, and 3-D vision through a combination of bottom-up, horizontal, and top-down interactions. A main theme is that the mechanisms which enable development and learning to occur in a stable way imply properties of adult behavior. These results thus begin to unify three fields: infant cortical development, adult cortical neurophysiology and anatomy, and adult visual perception. The identified cortical mechanisms promise to generalize to explain how other perceptual and cognitive processes work.

Neural Mechanisms of Cortical Motion Computation Based on a Neuromorphic Sensory System

The visual cortex analyzes motion information along hierarchically arranged visual areas that interact through bidirectional interconnections. This work suggests a bio-inspired visual model focusing on the interactions of the cortical areas in which a new mechanism of feedforward and feedback processing are introduced. The model uses a neuromorphic vision sensor (silicon retina) that simulates the spike-generation functionality of the biological retina. Our model takes into account two main model visual areas, namely V1 and MT, with different feature selectivities. The initial motion is estimated in model area V1 using spatiotemporal filters to locally detect the direction of motion. Here, we adapt the filtering scheme originally suggested by Adelson and Bergen to make it consistent with the spike representation of the DVS. The responses of area V1 are weighted and pooled by area MT cells which are selective to different velocities, i.e. direction and speed. Such feature selectivity is here derived from compositions of activities in the spatio-temporal domain and integrating over larger space-time regions (receptive fields). In order to account for the bidirectional coupling of cortical areas we match properties of the feature selectivity in both areas for feedback processing. For such linkage we integrate the responses over different speeds along a particular preferred direction. Normalization of activities is carried out over the spatial as well as the feature domains to balance the activities of individual neurons in model areas V1 and MT. Our model was tested using different stimuli that moved in different directions. The results reveal that the error margin between the estimated motion and synthetic ground truth is decreased in area MT comparing with the initial estimation of area V1. In addition, the modulated V1 cell activations shows an enhancement of the initial motion estimation that is steered by feedback signals from MT cells.

Binocular fusion and invariant category learning due to predictive remapping during scanning of a depthful scene with eye movements

How does the brain maintain stable fusion of 3D scenes when the eyes move? Every eye movement causes each retinal position to process a different set of scenic features, and thus the brain needs to binocularly fuse new combinations of features at each position after an eye movement. Despite these breaks in retinotopic fusion due to each movement, previously fused representations of a scene in depth often appear stable. The 3D ARTSCAN neural model proposes how the brain does this by unifying concepts about how multiple cortical areas in the What and Where cortical streams interact to coordinate processes of 3D boundary and surface perception, spatial attention, invariant object category learning, predictive remapping, eye movement control, and learned coordinate transformations. The model explains data from single neuron and psychophysical studies of covert visual attention shifts prior to eye movements. The model further clarifies how perceptual, attentional, and cognitive interactions among multiple brain regions (LGN, V1, V2, V3A, V4, MT, MST, PPC, LIP, ITp, ITa, SC) may accomplish predictive remapping as part of the process whereby view-invariant object categories are learned. These results build upon earlier neural models of 3D vision and figure-ground separation and the learning of invariant object categories as the eyes freely scan a scene. A key process concerns how an object's surface representation generates a form-fitting distribution of spatial attention, or attentional shroud, in parietal cortex that helps maintain the stability of multiple perceptual and cognitive processes. Predictive eye movement signals maintain the stability of the shroud, as well as of binocularly fused perceptual boundaries and surface representations.

How visual illusions illuminate complementary brain processes: illusory depth from brightness and apparent motion of illusory contours

Neural models of perception clarify how visual illusions arise from adaptive neural processes. Illusions also provide important insights into how adaptive neural processes work. This article focuses on two illusions that illustrate a fundamental property of global brain organization; namely, that advanced brains are organized into parallel cortical processing streams with computationally complementary properties. That is, in order to process certain combinations of properties, each cortical stream cannot process complementary properties. Interactions between these streams, across multiple processing stages, overcome their complementary deficiencies to compute effective representations of the world, and to thereby achieve the property of complementary consistency. The two illusions concern how illusory depth can vary with brightness, and how apparent motion of illusory contours can occur. Illusory depth from brightness arises from the complementary properties of boundary and surface processes, notably boundary completion and surface-filling in, within the parvocellular form processing cortical stream. This illusion depends upon how surface contour signals from the V2 thin stripes to the V2 interstripes ensure complementary consistency of a unified boundary/surface percept. Apparent motion of illusory contours arises from the complementary properties of form and motion processes across the parvocellular and magnocellular cortical processing streams. This illusion depends upon how illusory contours help to complete boundary representations for object recognition, how apparent motion signals can help to form continuous trajectories for target tracking and prediction, and how formotion interactions from V2-to-MT enable completed object representations to be continuously tracked even when they move behind intermittently occluding objects through time.

Integration of motion responses underlying directional motion anisotropy in human early visual cortical areas

Recent imaging studies have reported directional motion biases in human visual cortex when perceiving moving random dot patterns. It has been hypothesized that these biases occur as a result of the integration of motion detector activation along the path of motion in visual cortex. In this study we investigate the nature of such motion integration with functional MRI (fMRI) using different motion stimuli. Three types of moving random dot stimuli were presented, showing either coherent motion, motion with spatial decorrelations or motion with temporal decorrelations. The results from the coherent motion stimulus reproduced the centripetal and centrifugal directional motion biases in V1, V2 and V3 as previously reported. The temporally decorrelated motion stimulus resulted in both centripetal and centrifugal biases similar to coherent motion. In contrast, the spatially decorrelated motion stimulus resulted in small directional motion biases that were only present in parts of visual cortex coding for higher eccentricities of the visual field. In combination with previous results, these findings indicate that biased motion responses in early visual cortical areas most likely depend on the spatial integration of a simultaneously activated motion detector chain.

Bifurcation analysis applied to a model of motion integration with a multistable stimulus

A computational study into the motion perception dynamics of a multistable psychophysics stimulus is presented. A diagonally drifting grating viewed through a square aperture is perceived as moving in the actual grating direction or in line with the aperture edges (horizontally or vertically). The different percepts are the product of interplay between ambiguous contour cues and specific terminator cues. We present a dynamical model of motion integration that performs direction selection for such a stimulus and link the different percepts to coexisting steady states of the underlying equations. We apply the powerful tools of bifurcation analysis and numerical continuation to study changes to the model’s solution structure under the variation of parameters. Indeed, we apply these tools in a systematic way, taking into account biological and mathematical constraints, in order to fix model parameters. A region of parameter space is identified for which the model reproduces the qualitative behaviour observed in experiments. The temporal dynamics of motion integration are studied within this region; specifically, the effect of varying the stimulus gain is studied, which allows for qualitative predictions to be made.

Motion-based prediction is sufficient to solve the aperture problem

In low-level sensory systems, it is still unclear how the noisy information collected locally by neurons may give rise to a coherent global percept. This is well demonstrated for the detection of motion in the aperture problem: as luminance of an elongated line is symmetrical along its axis, tangential velocity is ambiguous when measured locally. Here, we develop the hypothesis that motion-based predictive coding is sufficient to infer global motion. Our implementation is based on a context-dependent diffusion of a probabilistic representation of motion. We observe in simulations a progressive solution to the aperture problem similar to physiology and behavior. We demonstrate that this solution is the result of two underlying mechanisms. First, we demonstrate the formation of a tracking behavior favoring temporally coherent features independent of their texture. Second, we observe that incoherent features are explained away, while coherent information diffuses progressively to the global scale. Most previous models included ad hoc mechanisms such as end-stopped cells or a selection layer to track specific luminance-based features as necessary conditions to solve the aperture problem. Here, we have proved that motion-based predictive coding, as it is implemented in this functional model, is sufficient to solve the aperture problem. This solution may give insights into the role of prediction underlying a large class of sensory computations.

A neuromorphic architecture for object recognition and motion anticipation using burst-STDP

In this work we investigate the possibilities offered by a minimal framework of artificial spiking neurons to be deployed in silico. Here we introduce a hierarchical network architecture of spiking neurons which learns to recognize moving objects in a visual environment and determine the correct motor output for each object. These tasks are learned through both supervised and unsupervised spike timing dependent plasticity (STDP). STDP is responsible for the strengthening (or weakening) of synapses in relation to pre- and post-synaptic spike times and has been described as a Hebbian paradigm taking place both in vitro and in vivo. We utilize a variation of STDP learning, called burst-STDP, which is based on the notion that, since spikes are expensive in terms of energy consumption, then strong bursting activity carries more information than single (sparse) spikes. Furthermore, this learning algorithm takes advantage of homeostatic renormalization, which has been hypothesized to promote memory consolidation during NREM sleep. Using this learning rule, we design a spiking neural network architecture capable of object recognition, motion detection, attention towards important objects, and motor control outputs. We demonstrate the abilities of our design in a simple environment with distractor objects, multiple objects moving concurrently, and in the presence of noise. Most importantly, we show how this neural network is capable of performing these tasks using a simple leaky-integrate-and-fire (LIF) neuron model with binary synapses, making it fully compatible with state-of-the-art digital neuromorphic hardware designs. As such, the building blocks and learning rules presented in this paper appear promising for scalable fully neuromorphic systems to be implemented in hardware chips.

Representation of motion onset and offset in an augmented Barlow-Levick model of motion detection

Kinetic occlusion produces discontinuities in the optic flow field, whose perception requires the detection of an unexpected onset or offset of otherwise predictably moving or stationary contrast patches. Many cells in primate visual cortex are directionally selective for moving contrasts, and recent reports suggest that this selectivity arises through the inhibition of contrast signals moving in the cells' null direction, as in the rabbit retina. This nulling inhibition circuit (Barlow-Levick) is here extended to also detect motion onsets and offsets. The selectivity of extended circuit units, measured as a peak evidence accumulation response to motion onset/offset compared to the peak response to constant motion, is analyzed as a function of stimulus speed. Model onset cells are quiet during constant motion, but model offset cells activate during constant motion at slow speeds. Consequently, model offset cell speed tuning is biased towards higher speeds than onset cell tuning, similarly to the speed tuning of cells in the middle temporal area when exposed to speed ramps. Given a population of neurons with different preferred speeds, this asymmetry addresses a behavioral paradox-why human subjects in a simple reaction time task respond more slowly to motion offsets than onsets for low speeds, even though monkey neuron firing rates react more quickly to the offset of a preferred stimulus than to its onset.

Neural dynamics of object-based multifocal visual spatial attention and priming: object cueing, useful-field-of-view, and crowding

How are spatial and object attention coordinated to achieve rapid object learning and recognition during eye movement search? How do prefrontal priming and parietal spatial mechanisms interact to determine the reaction time costs of intra-object attention shifts, inter-object attention shifts, and shifts between visible objects and covertly cued locations? What factors underlie individual differences in the timing and frequency of such attentional shifts? How do transient and sustained spatial attentional mechanisms work and interact? How can volition, mediated via the basal ganglia, influence the span of spatial attention? A neural model is developed of how spatial attention in the where cortical stream coordinates view-invariant object category learning in the what cortical stream under free viewing conditions. The model simulates psychological data about the dynamics of covert attention priming and switching requiring multifocal attention without eye movements. The model predicts how "attentional shrouds" are formed when surface representations in cortical area V4 resonate with spatial attention in posterior parietal cortex (PPC) and prefrontal cortex (PFC), while shrouds compete among themselves for dominance. Winning shrouds support invariant object category learning, and active surface-shroud resonances support conscious surface perception and recognition. Attentive competition between multiple objects and cues simulates reaction-time data from the two-object cueing paradigm. The relative strength of sustained surface-driven and fast-transient motion-driven spatial attention controls individual differences in reaction time for invalid cues. Competition between surface-driven attentional shrouds controls individual differences in detection rate of peripheral targets in useful-field-of-view tasks. The model proposes how the strength of competition can be mediated, though learning or momentary changes in volition, by the basal ganglia. A new explanation of crowding shows how the cortical magnification factor, among other variables, can cause multiple object surfaces to share a single surface-shroud resonance, thereby preventing recognition of the individual objects.

How do spatial learning and memory occur in the brain? Coordinated learning of entorhinal grid cells and hippocampal place cells

Spatial learning and memory are important for navigation and formation of episodic memories. The hippocampus and medial entorhinal cortex (MEC) are key brain areas for spatial learning and memory. Place cells in hippocampus fire whenever an animal is located in a specific region in the environment. Grid cells in the superficial layers of MEC provide inputs to place cells and exhibit remarkable regular hexagonal spatial firing patterns. They also exhibit a gradient of spatial scales along the dorsoventral axis of the MEC, with neighboring cells at a given dorsoventral location having different spatial phases. A neural model shows how a hierarchy of self-organizing maps, each obeying the same laws, responds to realistic rat trajectories by learning grid cells with hexagonal grid firing fields of multiple spatial scales and place cells with unimodal firing fields that fit neurophysiological data about their development in juvenile rats. The hippocampal place fields represent much larger spaces than the grid cells to support navigational behaviors. Both the entorhinal and hippocampal self-organizing maps amplify and learn to categorize the most energetic and frequent co-occurrences of their inputs. Top-down attentional mechanisms from hippocampus to MEC help to dynamically stabilize these spatial memories in both the model and neurophysiological data. Spatial learning through MEC to hippocampus occurs in parallel with temporal learning through lateral entorhinal cortex to hippocampus. These homologous spatial and temporal representations illustrate a kind of "neural relativity" that may provide a substrate for episodic learning and memory.

A new motion illusion based on competition between two kinds of motion processing units: the accordion grating

Parametric psychophysical investigations are reported for two related illusory effects that occur when viewing an elementary square-wave grating while making "back and forth" head movements along the projection line. Observers report a non-rigid distortion of the pattern, including: (i) an expansion in a direction perpendicular to the stripes, and (ii) a perceived curvature of the stripes. We investigated these two phenomena independently. The first depends on the classical physiological aperture problem that confronts early cells in the vision system. Interactions between ambiguous and unambiguous motion signals, generated at line interiors and line ends, respectively, can explain why the perceived expansion occurs only in directions perpendicular to the stripes. A simple model is presented and successfully tested by a nulling psychophysical experiment with four subjects. The experiment varies key stimulus attributes that generate ambiguous and unambiguous motion signals. Regarding the illusory curvature, a differential geometry model of the optics of our display, which identifies a non-classical three-dimensional (3D) aperture problem, is proposed (Yazdanbakhsh & Gori, 2011). We tested that model by implementing its closed form prediction of distortion to design displays for a second psychophysical experiment that also uses a nulling technique. Results from four subjects allow the quantification of the degree of perceived curvature as a function of speed, distance and stimulus type (blurred vs. unblurred grating) and are compatible with the predictions of the model.

Modelling the dynamics of motion integration with a new luminance-gated diffusion mechanism

The dynamics of motion integration show striking similarities when observed at neuronal, psychophysical, and oculomotor levels. Based on the inter-relation and complementary insights given by those dynamics, our goal was to test how basic mechanisms of dynamical cortical processing can be incorporated in a dynamical model to solve several aspects of 2D motion integration and segmentation. Our model is inspired by the hierarchical processing stages of the primate visual cortex: we describe the interactions between several layers processing local motion and form information through feedforward, feedback, and inhibitive lateral connections. Also, following perceptual studies concerning contour integration and physiological studies of receptive fields, we postulate that motion estimation takes advantage of another low-level cue, which is luminance smoothness along edges or surfaces, in order to gate recurrent motion diffusion. With such a model, we successfully reproduced the temporal dynamics of motion integration on a wide range of simple motion stimuli: line segments, rotating ellipses, plaids, and barber poles. Furthermore, we showed that the proposed computational rule of luminance-gated diffusion of motion information is sufficient to explain a large set of contextual modulations of motion integration and segmentation in more elaborated stimuli such as chopstick illusions, simulated aperture problems, or rotating diamonds. As a whole, in this paper we proposed a new basal luminance-driven motion integration mechanism as an alternative to less parsimonious models, we carefully investigated the dynamics of motion integration, and we established a distinction between simple and complex stimuli according to the kind of information required to solve their ambiguities.

Conditional spatial-frequency selective pooling of one-dimensional motion signals into global two-dimensional motion

This study examined spatial-frequency effects on a motion-pooling process in which spatially distributed local one-dimensional motion signals are integrated into the perception of global two-dimensional motion. Motion pooling over two- to three-octave frequency differences was found to be nearly impossible when all Gabor elements had circular envelopes, but possible when the width of high-frequency elements was reduced, and the stimulus as a whole formed a closed contour configuration. These results are consistent with a view that motion pooling is controlled by form information, and that spatial-frequency difference is one, but not an absolute, form cue of segmentation.

Low-level sensory plasticity during task-irrelevant perceptual learning: evidence from conventional and double training procedures

Studies of perceptual learning have focused on aspects of learning that are related to early stages of sensory processing. However, conclusions that perceptual learning results in low-level sensory plasticity are controversial, since such learning may also be attributed to plasticity in later stages of sensory processing or in readout from sensory to decision stages, or to changes in high-level central processing. To address this controversy, we developed a novel random dot motion (RDM) stimulus to target motion cells selective to contrast polarity by ensuring the motion direction information arises only from signal dot onsets and not their offsets, and used these stimuli in the paradigm of task-irrelevant perceptual learning (TIPL). In TIPL, learning is achieved in response to a stimulus by subliminally pairing that stimulus with the targets of an unrelated training task. In this manner, we are able to probe learning for an aspect of motion processing thought to be a function of directional V1 simple cells with a learning procedure that dissociates the learned stimulus from the decision processes relevant to the training task. Our results show direction-selective learning for the designated contrast polarity that does not transfer to the opposite contrast polarity. This polarity specificity was replicated in a double training procedure in which subjects were additionally exposed to the opposite polarity. Taken together, these results suggest that TIPL for motion stimuli may occur at the stage of directional V1 simple cells. Finally, a theoretical explanation is provided to understand the data.

Peeling plaids apart: context counteracts cross-orientation contrast masking

Background

Contrast discrimination for an image is usually harder if another image is superimposed on top. We asked whether such contrast masking may be enhanced or relieved depending on cues promoting integration of both images as a single pattern, versus segmentation into two independent components.

Methodology & Principal Findings

Contrast discrimination thresholds for a foveal test grating were sharply elevated in the presence of a perfectly overlapping orthogonally-oriented mask grating. However thresholds returned to the unmasked baseline when a surround grating was added, having the same orientation and phase of either the test or mask grating. Both such masking and ‘unmasking’ effects were much stronger for moving than static stimuli.

Conclusions & Significance

Our results suggest that common-fate motion reinforces the perception of a single coherent plaid pattern, while the surround helps to identify each component independently, thus peeling the plaid apart again. These results challenge current models of early vision, suggesting that higher-level surface organization influences contrast encoding, determining whether the contrast of a grating may be recovered independently from that of its mask.

A neural model of the temporal dynamics of figure-ground segregation in motion perception

How does the visual system manage to segment a visual scene into surfaces and objects and manage to attend to a target object? Based on psychological and physiological investigations, it has been proposed that the perceptual organization and segmentation of a scene is achieved by the processing at different levels of the visual cortical hierarchy. According to this, motion onset detection, motion-defined shape segregation, and target selection are accomplished by processes which bind together simple features into fragments of increasingly complex configurations at different levels in the processing hierarchy. As an alternative to this hierarchical processing hypothesis, it has been proposed that the processing stages for feature detection and segregation are reflected in different temporal episodes in the response patterns of individual neurons. Such temporal epochs have been observed in the activation pattern of neurons as low as in area V1. Here, we present a neural network model of motion detection, figure-ground segregation and attentive selection which explains these response patterns in an unifying framework. Based on known principles of functional architecture of the visual cortex, we propose that initial motion and motion boundaries are detected at different and hierarchically organized stages in the dorsal pathway. Visual shapes that are defined by boundaries, which were generated from juxtaposed opponent motions, are represented at different stages in the ventral pathway. Model areas in the different pathways interact through feedforward and modulating feedback, while mutual interactions enable the communication between motion and form representations. Selective attention is devoted to shape representations by sending modulating feedback signals from higher levels (working memory) to intermediate levels to enhance their responses. Areas in the motion and form pathway are coupled through top-down feedback with V1 cells at the bottom end of the hierarchy. We propose that the different temporal episodes in the response pattern of V1 cells, as recorded in recent experiments, reflect the strength of modulating feedback signals. This feedback results from the consolidated shape representations from coherent motion patterns and the attentive modulation of responses along the cortical hierarchy. The model makes testable predictions concerning the duration and delay of the temporal episodes of V1 cell responses as well as their response variations that were caused by modulating feedback signals.

A neural model of how the brain computes heading from optic flow in realistic scenes

Visually-based navigation is a key competence during spatial cognition. Animals avoid obstacles and approach goals in novel cluttered environments using optic flow to compute heading with respect to the environment. Most navigation models try either explain data, or to demonstrate navigational competence in real-world environments without regard to behavioral and neural substrates. The current article develops a model that does both. The ViSTARS neural model describes interactions among neurons in the primate magnocellular pathway, including V1, MT(+), and MSTd. Model outputs are quantitatively similar to human heading data in response to complex natural scenes. The model estimates heading to within 1.5 degrees in random dot or photo-realistically rendered scenes, and within 3 degrees in video streams from driving in real-world environments. Simulated rotations of less than 1 degrees /s do not affect heading estimates, but faster simulated rotation rates do, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.

Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance

Visually guided navigation through a cluttered natural scene is a challenging problem that animals and humans accomplish with ease. The ViSTARS neural model proposes how primates use motion information to segment objects and determine heading for purposes of goal approach and obstacle avoidance in response to video inputs from real and virtual environments. The model produces trajectories similar to those of human navigators. It does so by predicting how computationally complementary processes in cortical areas MT(-)/MSTv and MT(+)/MSTd compute object motion for tracking and self-motion for navigation, respectively. The model's retina responds to transients in the input stream. Model V1 generates a local speed and direction estimate. This local motion estimate is ambiguous due to the neural aperture problem. Model MT(+) interacts with MSTd via an attentive feedback loop to compute accurate heading estimates in MSTd that quantitatively simulate properties of human heading estimation data. Model MT(-) interacts with MSTv via an attentive feedback loop to compute accurate estimates of speed, direction and position of moving objects. This object information is combined with heading information to produce steering decisions wherein goals behave like attractors and obstacles behave like repellers. These steering decisions lead to navigational trajectories that closely match human performance.

What a difference a parameter makes: a psychophysical comparison of random dot motion algorithms

Random dot motion (RDM) displays have emerged as one of the standard stimulus types employed in psychophysical and physiological studies of motion processing. RDMs are convenient because it is straightforward to manipulate the relative motion energy for a given motion direction in addition to stimulus parameters such as the speed, contrast, duration, density, aperture, etc. However, as widely as RDMs are employed so do they vary in their details of implementation. As a result, it is often difficult to make direct comparisons across studies employing different RDM algorithms and parameters. Here, we systematically measure the ability of human subjects to estimate motion direction for four commonly used RDM algorithms under a range of parameters in order to understand how these different algorithms compare in their perceptibility. We find that parametric and algorithmic differences can produce dramatically different performances. These effects, while surprising, can be understood in relationship to pertinent neurophysiological data regarding spatiotemporal displacement tuning properties of cells in area MT and how the tuning function changes with stimulus contrast and retinal eccentricity. These data help give a baseline by which different RDM algorithms can be compared, demonstrate a need for clearly reporting RDM details in the methods of papers, and also pose new constraints and challenges to models of motion direction processing.

A neurally plausible model of the dynamics of motion integration in smooth eye pursuit based on recursive Bayesian estimation

In this study, we describe a model of motion integration in smooth eye pursuit based on a recursive Bayesian estimation process, which displays a dynamic behaviour qualitatively similar to the dynamics of the motion integration process observed experimentally, both psychophysically in humans and monkeys, and physiologically in monkeys. By formulating the model as an approximate version of a Kalman filter algorithm, we have been able to show that it can be put into a neurally plausible, distributed recurrent form which coarsely corresponds to the recurrent circuitry of visual cortical areas V1 and MT. The model thus provides further support for the notion that the motion integration process is based on a form of Bayesian estimation, as has been suggested by many psychophysical studies, and moreover suggests that the observed dynamic properties of this process are the result of the recursive nature of the motion estimation.

Temporal dynamics of decision-making during motion perception in the visual cortex

How does the brain make decisions? Speed and accuracy of perceptual decisions covary with certainty in the input, and correlate with the rate of evidence accumulation in parietal and frontal cortical "decision neurons". A biophysically realistic model of interactions within and between Retina/LGN and cortical areas V1, MT, MST, and LIP, gated by basal ganglia, simulates dynamic properties of decision-making in response to ambiguous visual motion stimuli used by Newsome, Shadlen, and colleagues in their neurophysiological experiments. The model clarifies how brain circuits that solve the aperture problem interact with a recurrent competitive network with self-normalizing choice properties to carry out probabilistic decisions in real time. Some scientists claim that perception and decision-making can be described using Bayesian inference or related general statistical ideas, that estimate the optimal interpretation of the stimulus given priors and likelihoods. However, such concepts do not propose the neocortical mechanisms that enable perception, and make decisions. The present model explains behavioral and neurophysiological decision-making data without an appeal to Bayesian concepts and, unlike other existing models of these data, generates perceptual representations and choice dynamics in response to the experimental visual stimuli. Quantitative model simulations include the time course of LIP neuronal dynamics, as well as behavioral accuracy and reaction time properties, during both correct and error trials at different levels of input ambiguity in both fixed duration and reaction time tasks. Model MT/MST interactions compute the global direction of random dot motion stimuli, while model LIP computes the stochastic perceptual decision that leads to a saccadic eye movement.

Motion-induced illusory contours: priority of global aspects on motion perception

The present work deals with the perception of illusory surfaces assessed by the apparent motion of equal, parallel, and equidistant vertical stripes. Particularly, the influence of a reduction in length of one single stripe was studied. Two alternative perceptions were reported by participants: (a) the stripes seemed to move in a plastic (nonrigid) fashion, or (b) an apparent illusory rectangle was perceived in motion at the top of a complete row of stripes, which appeared to move in a rigid fashion in the opposite direction. In both perceptions, the single shorter stripe lost its identity and was absorbed by the global pattern of stripes in motion. The tendency to maintain identity seems to regard first of all the whole structure. It was found that these perceptions depended on the velocity of presentation of stripes and on the size of the shorter stripe.

SOVEREIGN: An autonomous neural system for incrementally learning planned action sequences to navigate towards a rewarded goal

How do reactive and planned behaviors interact in real time? How are sequences of such behaviors released at appropriate times during autonomous navigation to realize valued goals? Controllers for both animals and mobile robots, or animats, need reactive mechanisms for exploration, and learned plans to reach goal objects once an environment becomes familiar. The SOVEREIGN (Self-Organizing, Vision, Expectation, Recognition, Emotion, Intelligent, Goal-oriented Navigation) animat model embodies these capabilities, and is tested in a 3D virtual reality environment. SOVEREIGN includes several interacting subsystems which model complementary properties of cortical What and Where processing streams and which clarify similarities between mechanisms for navigation and arm movement control. As the animat explores an environment, visual inputs are processed by networks that are sensitive to visual form and motion in the What and Where streams, respectively. Position-invariant and size-invariant recognition categories are learned by real-time incremental learning in the What stream. Estimates of target position relative to the animat are computed in the Where stream, and can activate approach movements toward the target. Motion cues from animat locomotion can elicit head-orienting movements to bring a new target into view. Approach and orienting movements are alternately performed during animat navigation. Cumulative estimates of each movement are derived from interacting proprioceptive and visual cues. Movement sequences are stored within a motor working memory. Sequences of visual categories are stored in a sensory working memory. These working memories trigger learning of sensory and motor sequence categories, or plans, which together control planned movements. Predictively effective chunk combinations are selectively enhanced via reinforcement learning when the animat is rewarded. Selected planning chunks effect a gradual transition from variable reactive exploratory movements to efficient goal-oriented planned movement sequences. Volitional signals gate interactions between model subsystems and the release of overt behaviors. The model can control different motor sequences under different motivational states and learns more efficient sequences to rewarded goals as exploration proceeds.