Recurrent network with large representational capacity

A multi-scale neurodynamic implementation of incremental grouping

Incremental grouping is a process entailing serial binding of distal image elements into a unified object representation. At the neural level, incremental grouping involves propagation of the enhanced firing rate among feature-tuned neurons in the early visual cortex. Here, we developed a multi-resolution neural model of incremental grouping. In the model, propagation of the enhanced firing rate is achieved by computing the activity difference between two sets of units: attentional or A-units, whose firing rate is modulated by their horizontal collaterals, and non-attentional or N-units that receive only feedforward input. The activity difference is computed on dendrites that act as independent computational subunits. The proposed model employs multiple spatial scales to account for a variable speed of incremental grouping. In addition, the model incorporates the L-junction detection network that enables incremental grouping over L-junctions. Computer simulations show that the timing of attentional modulations in the model is comparable with neurophysiological measurements in monkey primary visual cortex.

A Neurodynamic Model of Feature-Based Spatial Selection

Huang and Pashler (2007) suggested that feature-based attention creates a special form of spatial representation, which is termed a Boolean map. It partitions the visual scene into two distinct and complementary regions: selected and not selected. Here, we developed a model of a recurrent competitive network that is capable of state-dependent computation. It selects multiple winning locations based on a joint top-down cue. We augmented a model of the WTA circuit that is based on linear-threshold units with two computational elements: dendritic non-linearity that acts on the excitatory units and activity-dependent modulation of synaptic transmission between excitatory and inhibitory units. Computer simulations showed that the proposed model could create a Boolean map in response to a featured cue and elaborate it using the logical operations of intersection and union. In addition, it was shown that in the absence of top-down guidance, the model is sensitive to bottom-up cues such as saliency and abrupt visual onset.

Selectivity and stability via dendritic nonlinearity

Inspired by recent studies regarding dendritic computation, we constructed a recurrent neural network model incorporating dendritic lateral inhibition. Our model consists of an input layer and a neuron layer that includes excitatory cells and an inhibitory cell; this inhibitory cell is activated by the pooled activities of all the excitatory cells, and it in turn inhibits each dendritic branch of the excitatory cells that receive excitations from the input layer. Dendritic nonlinear operation consisting of branch-specifically rectified inhibition and saturation is described by imposing nonlinear transfer functions before summation over the branches. In this model with sufficiently strong recurrent excitation, on transiently presenting a stimulus that has a high correlation with feed- forward connections of one of the excitatory cells, the corresponding cell becomes highly active, and the activity is sustained after the stimulus is turned off, whereas all the other excitatory cells continue to have low activities. But on transiently presenting a stimulus that does not have high correlations with feedforward connections of any of the excitatory cells, all the excitatory cells continue to have low activities. Interestingly, such stimulus-selective sustained response is preserved for a wide range of stimulus intensity. We derive an analytical formulation of the model in the limit where individual excitatory cells have an infinite number of dendritic branches and prove the existence of an equilibrium point corresponding to such a balanced low-level activity state as observed in the simulations, whose stability depends solely on the signal-to-noise ratio of the stimulus. We propose this model as a model of stimulus selectivity equipped with self-sustainability and intensity-invariance simultaneously, which was difficult in the conventional competitive neural networks with a similar degree of complexity in their network architecture. We discuss the biological relevance of the model in a general framework of computational neuroscience.