Neural dynamics of feedforward and feedback processing in figure-ground segregation

Psychophysics of neon color spreading: Chromatic and temporal factors are not limiting

Neon color spreading (NCS) is an illusory color phenomenon that provides a dramatic example of surface completion and filling-in. Numerous studies have varied both spatial and temporal aspects of the neon-generating stimulus to explore variations in the strength of the effect. Here, we take a novel, parametric, low-level psychophysical approach to studying NCS in two experiments. In Experiment 1, we test the ability of both cone-isolating and equiluminant stimuli to generate neon color spreading for both increments and decrements in cone modulations. As expected, sensitivity was low to S(hort-wavelength) cone stimuli due to their poor spatial resolution, but sensitivity was similar for the other color directions. We show that when these differences in detection sensitivity are accounted for, the particular cone type, and the polarity (increment or decrement), make little difference in generating neon color spreading, with NCS visible at about twice detection threshold level in all cases. In Experiment 2, we use L-cone flicker modulations (reddish and greenish excursions around grey) to study sensitivity to NCS as a function of temporal frequency from 0.5 to 8 Hz. After accounting for detectability, the temporal contrast sensitivity functions for NCS are approximately constant or even increase over the studied frequency range. Therefore there is no evidence in this study that the processes underlying NCS are slower than the low-level processes of simple flicker detection. These results point to relatively fast mechanisms, not slow diffusion processes, as the substrate for NCS.

A neural modeling approach to study mechanisms underlying the heterogeneity of visual spatial frequency sensitivity in schizophrenia

Patients with schizophrenia exhibit abnormalities in spatial frequency sensitivity, and it is believed that these abnormalities indicate more widespread dysfunction and dysregulation of bottom-up processing. The early visual system, including the first-order Lateral Geniculate Nucleus of the thalamus (LGN) and the primary visual cortex (V1), are key contributors to spatial frequency sensitivity. Medicated and unmedicated patients with schizophrenia exhibit contrasting changes in spatial frequency sensitivity, thus making it a useful probe for examining potential effects of the disorder and antipsychotic medications in neural processing. We constructed a parameterized, rate-based neural model of on-center/off-surround neurons in the early visual system to investigate the impacts of changes to the excitatory and inhibitory receptive field subfields. By incorporating changes in both the excitatory and inhibitory subfields that are associated with pathophysiological findings in schizophrenia, the model successfully replicated perceptual data from behavioral/functional studies involving medicated and unmedicated patients. Among several plausible mechanisms, our results highlight the dampening of excitation and/or increase in the spread and strength of the inhibitory subfield in medicated patients and the contrasting decreased spread and strength of inhibition in unmedicated patients. Given that the model was successful at replicating results from perceptual data under a variety of conditions, these elements of the receptive field may be useful markers for the imbalances seen in patients with schizophrenia.

Visual mental imagery abilities in autism

In autistic individuals, the role, performance, and autonomy of perceptual functioning are atypical. Overlapping underlying mechanisms of perception and mental imagery predict that the mental imagery abilities of autistic individuals should differ from those of non‐autistic individuals. While enhanced abilities to manipulate mental images have been demonstrated in autism, the other stages of mental imagery (generation, maintenance, inspection) remain to be explored. Forty‐four autistic adults and 42 typical participants performed four tasks to assess different stages of mental imagery: the Image generation task (mentally generating a letter on a grid and indicating whether it passes over a probe located in the grid), the Visual pattern test (maintaining visual patterns in memory), the Image scanning test (inspecting mental images) and the Mental rotation test (mentally manipulating representations of geometric figures). In the image generation task and the mental rotation test, autistic and typical individuals performed equivalently, both in accuracy and response time. The span observed in the visual pattern test was significantly higher in the autistic group, indicating better maintenance of mental images. In the image scanning test, response times were influenced by the distance to mentally inspect in the typical group but not in the autistic group. Autistic participants were equally fast regardless of distance to inspect. Preserved, greater or differently influenced visual mental imagery abilities are in line with an atypical perceptual functioning in autism, possibly reflecting an increased weight of perception‐based information relatively to the top‐down effect of knowledge and language‐based influence.

A spatial map: a propitious choice for constraining the binding problem

Many studies have shown that the human visual system has two major functionally distinct cortical visual pathways: a ventral pathway, thought to be important for object recognition, and a dorsal pathway, thought to be important for spatial cognition. According to our and others previous studies, artificial neural networks with two segregated pathways can determine objects' identities and locations more accurately and efficiently than one-pathway artificial neural networks. In addition, we showed that these two segregated artificial cortical visual pathways can each process identity and spatial information of visual objects independently and differently. However, when using such networks to process multiple objects' identities and locations, a binding problem arises because the networks may not associate each object's identity with its location correctly. In a previous study, we constrained the binding problem by training the artificial identity pathway to retain relative location information of objects. This design uses a location map to constrain the binding problem. One limitation of that study was that we only considered two attributes of our objects (identity and location) and only one possible map (location) for binding. However, typically the brain needs to process and bind many attributes of an object, and any of these attributes could be used to constrain the binding problem. In our current study, using visual objects with multiple attributes (identity, luminance, orientation, and location) that need to be recognized, we tried to find the best map (among an identity map, a luminance map, an orientation map, or a location map) to constrain the binding problem. We found that in our experimental simulations, when visual attributes are independent of each other, a location map is always a better choice than the other kinds of maps examined for constraining the binding problem. Our findings agree with previous neurophysiological findings that show that the organization or map in many visual cortical areas is primarily retinotopic or spatial.

Interaction of contour geometry and optic flow in determining relative depth of surfaces

Dynamic occlusion, such as the accretion and deletion of texture near a boundary, is a major factor in determining relative depth of surfaces. However, the shape of the contour bounding the dynamic texture can significantly influence what kind of 3D shape, and what relative depth, are conveyed by the optic flow. This can lead to percepts that are inconsistent with traditional accounts of shape and depth from motion, where accreting/deleting texture can indicate the figural region, and/or 3D rotation can be perceived despite the constant speed of the optic flow. This suggests that the speed profile of the dynamic texture and the shape of its bounding contours combine to determine relative depth in a way that is not explained by existing models. Here, we investigated how traditional structure-from-motion principles and contour geometry interact to determine the relative-depth interpretation of dynamic textures. We manipulated the consistency of the dynamic texture with rotational or translational motion by varying the speed profile of the texture. In Experiment 1, we used a multi-region figure-ground display consisting of regions with dots moving horizontally in opposite directions in adjacent regions. In Experiment 2, we used stimuli including two regions separated by a common border, with dot textures moving horizontally in opposite directions. Both contour geometry (convexity) and the speed profile of the dynamic dot texture influenced relative-depth judgments, but contour geometry was the stronger factor. The results underscore the importance of contour geometry, which most current models disregard, in determining depth from motion.

A look into feedback neural computation upon collision selectivity

Physiological studies have shown that a group of locust's lobula giant movement detectors (LGMDs) has a diversity of collision selectivity to approaching objects, relatively darker or brighter than their backgrounds in cluttered environments. Such diversity of collision selectivity can serve locusts to escape from attack by natural enemies, and migrate in swarm free of collision. For computational studies, endeavours have been made to realize the diverse selectivity which, however, is still one of the most challenging tasks especially in complex and dynamic real world scenarios. The existing models are mainly formulated as multi-layered neural networks with merely feed-forward information processing, and do not take into account the effect of re-entrant signals in feedback loop, which is an essential regulatory loop for motion perception, yet never been explored in looming perception. In this paper, we inaugurate feedback neural computation for constructing a new LGMD-based model, named F-LGMD to look into the efficacy upon implementing different collision selectivity. Accordingly, the proposed neural network model features both feed-forward processing and feedback loop. The feedback control propagates output signals of parallel ON/OFF channels back into their starting neurons, thus makes part of the feed-forward neural network, i.e. the ON/OFF channels and the feedback loop form an iterative cycle system. Moreover, the feedback control is instantaneous, which leads to the existence of a fixed point whereby the fixed point theorem is applied to rigorously derive valid range of feedback coefficients. To verify the effectiveness of the proposed method, we conduct systematic experiments covering synthetic and natural collision datasets, and also online robotic tests. The experimental results show that the F-LGMD, with a unified network, can fulfil the diverse collision selectivity revealed in physiology, which not only reduces considerably the handcrafted parameters compared to previous studies, but also offers a both efficient and robust scheme for collision perception through feedback neural computation.

Sleep spindles in primates: Modeling the effects of distinct laminar thalamocortical connectivity in core, matrix, and reticular thalamic circuits

Sleep spindles are associated with the beginning of deep sleep and memory consolidation and are disrupted in schizophrenia and autism. In primates, distinct core and matrix thalamocortical (TC) circuits regulate sleep spindle activity through communications that are filtered by the inhibitory thalamic reticular nucleus (TRN); however, little is known about typical TC network interactions and the mechanisms that are disrupted in brain disorders. We developed a primate-specific, circuit-based TC computational model with distinct core and matrix loops that can simulate sleep spindles. We implemented novel multilevel cortical and thalamic mixing, and included local thalamic inhibitory interneurons, and direct layer 5 projections of variable density to TRN and thalamus to investigate the functional consequences of different ratios of core and matrix node connectivity contribution to spindle dynamics. Our simulations showed that spindle power in primates can be modulated based on the level of cortical feedback, thalamic inhibition, and engagement of model core versus matrix, with the latter having a greater role in spindle dynamics. The study of the distinct spatial and temporal dynamics of core-, matrix-, and mix-generated sleep spindles establishes a framework to study disruption of TC circuit balance underlying deficits in sleep and attentional gating seen in autism and schizophrenia.

Severely Attenuated Visual Feedback Processing in Children on the Autism Spectrum

Individuals on the autism spectrum often exhibit atypicality in their sensory perception, but the neural underpinnings of these perceptual differences remain incompletely understood. One proposed mechanism is an imbalance in higher-order feedback re-entrant inputs to early sensory cortices during sensory perception, leading to increased propensity to focus on local object features over global context. We explored this theory by measuring visual evoked potentials during contour integration as considerable work has revealed that these processes are largely driven by feedback inputs from higher-order ventral visual stream regions. We tested the hypothesis that autistic individuals would have attenuated evoked responses to illusory contours compared with neurotypical controls. Electrophysiology was acquired while 29 autistic and 31 neurotypical children (7-17 years old, inclusive of both males and females) passively viewed a random series of Kanizsa figure stimuli, each consisting of four inducers that were aligned either at random rotational angles or such that contour integration would form an illusory square. Autistic children demonstrated attenuated automatic contour integration over lateral occipital regions relative to neurotypical controls. The data are discussed in terms of the role of predictive feedback processes on perception of global stimulus features and the notion that weakened "priors" may play a role in the visual processing anomalies seen in autism.SIGNIFICANCE STATEMENT Children on the autism spectrum differ from typically developing children in many aspects of their processing of sensory stimuli. One proposed mechanism for these differences is an imbalance in higher-order feedback to primary sensory regions, leading to an increased focus on local object features rather than global context. However, systematic investigation of these feedback mechanisms remains limited. Using EEG and a visual illusion paradigm that is highly dependent on intact feedback processing, we demonstrated significant disruptions to visual feedback processing in children with autism. This provides much needed experimental evidence that advances our understanding of the contribution of feedback processing to visual perception in autism spectrum disorder.

Visual illusion susceptibility in autism: A neural model

While atypical sensory perception is reported among individuals with autism spectrum disorder (ASD), the underlying neural mechanisms of autism that give rise to disruptions in sensory perception remain unclear. We developed a neural model with key physiological, functional and neuroanatomical parameters to investigate mechanisms underlying the range of representations of visual illusions related to orientation perception in typically developed subjects compared to individuals with ASD. Our results showed that two theorized autistic traits, excitation/inhibition imbalance and weakening of top‐down modulation, could be potential candidates for reduced susceptibility to some visual illusions. Parametric correlation between cortical suppression, balance of excitation/inhibition, feedback from higher visual areas on one hand and susceptibility to a class of visual illusions related to orientation perception on the other hand provide the opportunity to investigate the contribution and complex interactions of distinct sensory processing mechanisms in ASD. The novel approach used in this study can be used to link behavioural, functional and neuropathological studies; estimate and predict perceptual and cognitive heterogeneity in ASD; and form a basis for the development of novel diagnostics and therapeutics.

Where there is no object formation, there is no perceptual organization: Evidence from the configural superiority effect

Object formation is considered the aim of perceptual organization, but such a proposition has been neglected in empirical studies. In the current study, we investigated the role of object formation in configural superiority. Essentially, discrimination on bar orientations was enhanced by adding a right angle to each of the bars. Such facilitation is due to the emergent feature (EF) of closure formed by combining the bars with right angles. To study object formation, visual stimuli were generated by random dot stereograms to form objects or holes in 3D. Behaviorally, we found that the EF of closure facilitated oddball discrimination on objects, as demonstrated by previous studies, but did not facilitate oddball discrimination on holes with the same shape as objects. Multivariate pattern analysis of functional magnetic resonance imaging (fMRI) data showed that the EF of closure increased the object classification accuracy compared to the holes in the lateral occipital cortex (LOC), where object information is encoded, but not in the early visual cortex (EVC). The neural representations of objects and holes with and without EFs were further investigated using representational similarity analysis. The results demonstrate that in the LOC, the neural representations of objects with EFs showed a greater difference than those of the other three, that is, objects without EFs and holes with or without EFs. However, the uniqueness of objects with EFs was not observed in the EVC. Thus, our results suggest that the EF of closure, which leads to the configural superiority effect, only emerges for objects but not for holes, and only in the LOC but not the EVC. Our study provides the first empirical evidence suggesting that object formation plays an indispensable role in perceptual organization.

Learning Invariant Features in Modulatory Networks through Conflict and Ambiguity

This work lays the foundation for a framework of cortical learning based on the idea of a competitive column, which is inspired by the functional organization of neurons in the cortex. A column describes a prototypical organization for neurons that gives rise to an ability to learn scale, rotation, and translation-invariant features. This is empowered by a recently developed learning rule, conflict learning, which enables the network to learn over both driving and modulatory feedforward, feedback, and lateral inputs. The framework is further supported by introducing both a notion of neural ambiguity and an adaptive threshold scheme. Ambiguity, which captures the idea that too many decisions lead to indecision, gives the network a dynamic way to resolve locally ambiguous decisions. The adaptive threshold operates over multiple timescales to regulate neural activity under the varied arrival timings of input in a highly interconnected multilayer network with feedforward and feedback. The competitive column architecture is demonstrated on a large-scale (54,000 neurons and 18 million synapses), invariant model of border ownership. The model is trained on four simple, fixed-scale shapes: two squares, one rectangle, and one symmetric L-shape. Tested on 1899 synthetic shapes of varying scale and complexity, the model correctly assigned border ownership with 74% accuracy. The model's abilities were also illustrated on contours of objects taken from natural images. Combined with conflict learning, the competitive column and ambiguity give a better intuitive understanding of how feedback, modulation, and inhibition may interact in the brain to influence activation and learning.

Modeling the Time-Course of Responses for the Border Ownership Selectivity Based on the Integration of Feedforward Signals and Visual Cortical Interactions

Border ownership (BO) indicates which side of a contour owns a border, and it plays a fundamental role in figure-ground segregation. The majority of neurons in V2 and V4 areas of monkeys exhibit BO selectivity. A physiological work reported that the responses of BO-selective cells show a rapid transition when a presented square is flipped along its classical receptive field (CRF) so that the opposite BO is presented, whereas the transition is significantly slower when a square with a clear BO is replaced by an ambiguous edge, e.g., when the square is enlarged greatly. The rapid transition seemed to reflect the influence of feedforward processing on BO selectivity. Herein, we investigated the role of feedforward signals and cortical interactions for time-courses in BO-selective cells by modeling a visual cortical network comprising V1, V2, and posterior parietal (PP) modules. In our computational model, the recurrent pathways among these modules gradually established the visual progress and the BO assignments. Feedforward inputs mainly determined the activities of these modules. Surrounding suppression/facilitation of early-level areas modulates the activities of V2 cells to provide BO signals. Weak feedback signals from the PP module enhanced the contrast gain extracted in V1, which underlies the attentional modulation of BO signals. Model simulations exhibited time-courses depending on the BO ambiguity, which were caused by the integration delay of V1 and V2 cells and the local inhibition therein given the difference in input stimulus. However, our model did not fully explain the characteristics of crucially slow transition: the responses of BO-selective physiological cells indicated the persistent activation several times longer than that of our model after the replacement with the ambiguous edge. Furthermore, the time-course of BO-selective model cells replicated the attentional modulation of response time in human psychophysical experiments. These attentional modulations for time-courses were induced by selective enhancement of early-level features due to interactions between V1 and PP. Our proposed model suggests fundamental roles of surrounding suppression/facilitation based on feedforward inputs as well as the interactions between early and parietal visual areas with respect to the ambiguity dependence of the neural dynamics in intermediate-level vision.