Recurrent inhibition and clustered connectivity as a basis for Gabor-like receptive fields in the visual cortex

Recurrent models of orientation selectivity enable robust early-vision processing in mixed-signal neuromorphic hardware

Mixed signal analog/digital neuromorphic circuits represent an ideal medium for reproducing bio-physically realistic dynamics of biological neural systems in real-time. However, similar to their biological counterparts, these circuits have limited resolution and are affected by a high degree of variability. By developing a recurrent spiking neural network model of the retinocortical visual pathway, we show how such noisy and heterogeneous computing substrate can produce linear receptive fields tuned to visual stimuli with specific orientations and spatial frequencies. Compared to strictly feed-forward schemes, the model generates highly structured Gabor-like receptive fields of any phase symmetry, making optimal use of the hardware resources available in terms of synaptic connections and neuron numbers. Experimental results validate the approach, demonstrating how principles of neural computation can lead to robust sensory processing electronic systems, even when they are affected by high degree of heterogeneity, e.g., due to the use of analog circuits or memristive devices.

Diversity priors for learning early visual features

This paper investigates how utilizing diversity priors can discover early visual features that resemble their biological counterparts. The study is mainly motivated by the sparsity and selectivity of activations of visual neurons in area V1. Most previous work on computational modeling emphasizes selectivity or sparsity independently. However, we argue that selectivity and sparsity are just two epiphenomena of the diversity of receptive fields, which has been rarely exploited in learning. In this paper, to verify our hypothesis, restricted Boltzmann machines (RBMs) are employed to learn early visual features by modeling the statistics of natural images. Considering RBMs as neural networks, the receptive fields of neurons are formed by the inter-weights between hidden and visible nodes. Due to the conditional independence in RBMs, there is no mechanism to coordinate the activations of individual neurons or the whole population. A diversity prior is introduced in this paper for training RBMs. We find that the diversity prior indeed can assure simultaneously sparsity and selectivity of neuron activations. The learned receptive fields yield a high degree of biological similarity in comparison to physiological data. Also, corresponding visual features display a good generative capability in image reconstruction.

Dynamics of Receptive Field Size in Primary Visual Cortex

Recent studies have shown that the initial responses evoked by a stimulus in neurons of primary visual cortex are dominated by low spatial frequency information in the image, whereas finer spatial scales dominate later in the response. Such phenomena could arise from the dynamics of receptive field (RF) size at early stages of cortical processing. We measured changes in RF size in simple cells recorded from the primary visual cortex of anesthetized macaques by measuring their first-order spatio-temporal kernels and fitting them with two-dimensional Gabor functions at different time slices. We found that the width and length of the RF envelope and the period of the carrier tend to decrease during the time-course of the response. The most pronounced changes are seen in the width and spatial period of the RFs, which decrease by 15% during the central 20 ms of the response. These results show a novel form of spatio-temporal inseparability in simple cells and are consistent with the notion of a coarse-to-fine processing of information in early visual cortex.

Classification Capacity of a Modular Neural Network Implementing Neurally Inspired Architecture and Training Rules

A three-layer neural network (NN) with novel adaptive architecture has been developed. The hidden layer of the network consists of slabs of single neuron models, where neurons within a slab--but not between slabs--have the same type of activation function. The network activation functions in all three layers have adaptable parameters. The network was trained using a biologically inspired, guided-annealing learning rule on a variety of medical data. Good training/testing classification performance was obtained on all data sets tested. The performance achieved was comparable to that of SVM classifiers. It was shown that the adaptive network architecture, inspired from the modular organization often encountered in the mammalian cerebral cortex, can benefit classification performance.

Dynamic approximation of spatiotemporal receptive fields in nonlinear neural field models

This article presents an approximation method to reduce the spatiotemporal behavior of localized activation peaks (also called "bumps") in non-linear neural field equations to a set of coupled ordinary differential equations (ODEs) for only the amplitudes and tuning widths of these peaks. This enables a simplified analysis of steady-state receptive fields and their stability, as well as spatiotemporal point spread functions and dynamic tuning properties. A lowest-order approximation for peak amplitudes alone shows that much of the well-studied behavior of small neural systems (e.g., the Wilson-Cowan oscillator) should carry over to localized solutions in neural fields. Full spatiotemporal response profiles can further be reconstructed from this low-dimensional approximation. The method is applied to two standard neural field models: a one-layer model with difference-of-gaussians connectivity kernel and a two-layer excitatory-inhibitory network. Similar models have been previously employed in numerical studies addressing orientation tuning of cortical simple cells. Explicit formulas for tuning properties, instabilities, and oscillation frequencies are given, and exemplary spatiotemporal response functions, reconstructed from the low-dimensional approximation, are compared with full network simulations.

Orientation tuning properties of simple cells in area V1 derived from an approximate analysis of nonlinear neural field models

We present a general approximation method for the mathematical analysis of spatially localized steady-state solutions in nonlinear neural field models. These models comprise several layers of excitatory and inhibitory cells. Coupling kernels between and inside layers are assumed to be gaussian shaped. In response to spatially localized (i.e., tuned) inputs, such networks typically reveal stationary localized activity profiles in the different layers. Qualitative properties of these solutions, like response amplitudes and tuning widths, are approximated for a whole class of nonlinear rate functions that obey a power law above some threshold and that are zero below. A special case of these functions is the semilinear function, which is commonly used in neural field models. The method is then applied to models for orientation tuning in cortical simple cells: first, to the one-layer model with "difference of gaussians" connectivity kernel developed by Carandini and Ringach (1997) as an abstraction of the biologically detailed simulations of Somers, Nelson, and Sur (1995); second, to a two-field model comprising excitatory and inhibitory cells in two separate layers. Under certain conditions, both models have the same steady states. Comparing simulations of the field models and results derived from the approximation method, we find that the approximation well predicts the tuning behavior of the full model. Moreover, explicit formulas for approximate amplitudes and tuning widths in response to changing input strength are given and checked numerically. Comparing the network behavior for different nonlinearities, we find that the only rate function (from the class of functions under study) that leads to constant tuning widths and a linear increase of firing rates in response to increasing input is the semilinear function. For other nonlinearities, the qualitative network response depends on whether the model neurons operate in a convex (e.g., x(2)) or concave (e.g., sqrt(x)) regime of their rate function. In the first case, tuning gradually changes from input driven at low input strength (broad tuning strongly depending on the input and roughly linear amplitudes in response to input strength) to recurrently driven at moderate input strength (sharp tuning, supralinear increase of amplitudes in response to input strength). For concave rate functions, the network reveals stable hysteresis between a state at low firing rates and a tuned state at high rates. This means that the network can "memorize" tuning properties of a previously shown stimulus. Sigmoid rate functions can combine both effects. In contrast to the Carandini-Ringach model, the two-field model further reveals oscillations with typical frequencies in the beta and gamma range, when the excitatory and inhibitory connections are relatively strong. This suggests a rhythmic modulation of tuning properties during cortical oscillations.

Functional periodic intracortical couplings induced by structured lateral inhibition in a linear cortical network

The spatial organization of cortical axon and dendritic fields could be an interesting structural paradigm to obtain a functional specificity without postulating highly specific feedforward connections. In this article, we investigate the functional implications of recurrent intracortical inhibition when it occurs through clustered medium-range interconnection schemes (Wörgötter & Koch, 1991; Somogyi, 1989; Kritzer, Cowey, & Somogyi, 1992). Moreover, the interaction between the inhibitory schemes and visual orientation maps is explored. Assuming linearity, we show that clustered inhibitory mechanisms can trigger a propagation process that allows the development of extra (i.e., induced) interactions among the cortical sites involved in the recurrent loops. In addition, we point out how these interactions functionally modify the response of cortical simple cells and yield to highly structured Gabor-like receptive fields. This study should be considered not as a realistic biological model of the primary visual cortex but as an attempt to explain possible computational principles related to intracortical connectivity and to the underlying single-cell properties.