Anatomical constraints on lateral competition in columnar cortical architectures

Inhibitory stabilized network behaviour in a balanced neural mass model of a cortical column

Strong inhibitory recurrent connections can reduce the tendency for a neural network to become unstable. This is known as inhibitory stabilization; networks that are unstable in the absence of strong inhibitory feedback because of their unstable excitatory recurrent connections are known as Inhibition Stabilized Networks (ISNs). One of the characteristics of ISNs is their "paradoxical response", where perturbing the inhibitory neurons with additional excitatory input results in a decrease in their activity after a temporal delay instead of increasing their activity. Here, we develop a model of populations of neurons across different layers of cortex. Within each layer, there is one population of inhibitory neurons and one population of excitatory neurons. The connectivity weights across different populations in the model are derived from a synaptic physiology database provided by the Allen Institute. The model shows a gradient of excitation-inhibition balance across different layers in the cortex, where superficial layers are more inhibitory dominated compared to deeper layers. To investigate the presence of ISNs across different layers, we measured the membrane potentials of neural populations in the model after perturbing inhibitory populations. The results show that layer 2/3 in the model does not operate in the ISN regime but layers 4 and 5 do operate in the ISN regime. These results accord with neurophysiological findings that explored the presence of ISNs across different layers in the cortex. The results show that there may be a systematic macroscopic gradient of inhibitory stabilization across different layers in the cortex that depends on the level of excitation-inhibition balance, and that the strength of the paradoxical response increases as the model moves closer to bifurcation points.

Hybrid Circuit of Memristor and Complementary Metal-Oxide-Semiconductor for Defect-Tolerant Spatial Pooling with Boost-Factor Adjustment

Hierarchical Temporal Memory (HTM) has been known as a software framework to model the brain's neocortical operation. However, mimicking the brain's neocortical operation by not software but hardware is more desirable, because the hardware can not only describe the neocortical operation, but can also employ the brain's architectural advantages. To develop a hybrid circuit of memristor and Complementary Metal-Oxide-Semiconductor (CMOS) for realizing HTM's spatial pooler (SP) by hardware, memristor defects such as stuck-at-faults and variations should be considered. For solving the defect problem, we first show that the boost-factor adjustment can make HTM's SP defect-tolerant, because the false activation of defective columns are suppressed. Second, we propose a memristor-CMOS hybrid circuit with the boost-factor adjustment to realize this defect-tolerant SP by hardware. The proposed circuit does not rely on the conventional defect-aware mapping scheme, which cannot avoid the false activation of defective columns. For the Modified subset of National Institute of Standards and Technology (MNIST) vectors, the boost-factor adjusted crossbar with defects = 10% shows a rate loss of only ~0.6%, compared to the ideal crossbar with defects = 0%. On the contrary, the defect-aware mapping without the boost-factor adjustment demonstrates a significant rate loss of ~21.0%. The energy overhead of the boost-factor adjustment is only ~0.05% of the programming energy of memristor synapse crossbar.

Memristor-CMOS Hybrid Circuit for Temporal-Pooling of Sensory and Hippocampal Responses of Cortical Neurons

As a software framework, Hierarchical Temporal Memory (HTM) has been developed to perform the brain’s neocortical functions, such as spatial and temporal pooling. However, it should be realized with hardware not software not only to mimic the neocortical function but also to exploit its architectural benefit. To do so, we propose a new memristor-CMOS (Complementary Metal-Oxide-Semiconductor) hybrid circuit of temporal-pooling here, which is composed of the input-layer and output-layer neurons mimicking the neocortex. In the hybrid circuit, the input-layer neurons have the proximal and basal/distal dendrites to combine sensory information with the temporal/location information from the brain’s hippocampus. Using the same crossbar architecture, the output-layer neurons can perform a prediction by integrating the temporal information on the basal/distal dendrites. For training the proposed circuit, we used only simple Hebbian learning, not the complicated backpropagation algorithm. Due to the simple hardware of Hebbian learning, the proposed hybrid circuit can be very suitable to online learning. The proposed memristor-CMOS hybrid circuit has been verified by the circuit simulation using the real memristor model. The proposed circuit has been verified to predict both the ordinal and out-of-order sequences. In addition, the proposed circuit has been tested with the external noise and memristance variation.

A Computational Model of Dual Competition between the Basal Ganglia and the Cortex

We propose a model that includes interactions between the cortex, the basal ganglia (BG), and the thalamus based on a dual competition. We hypothesize that the striatum, the subthalamic nucleus (STN), the internal globus pallidus (GPi), the thalamus, and the cortex are involved in closed feedback loops through the hyperdirect and direct pathways. These loops support a competition process that results in the ability of BG to make a cognitive decision followed by a motor one. Considering lateral cortical interactions, another competition takes place inside the cortex allowing the latter to make a cognitive and a motor decision. We show how this dual competition endows the model with two regimes. One is driven by reinforcement learning and the other by Hebbian learning. The final decision is made according to a combination of these two mechanisms with a gradual transfer from the former to the latter. We confirmed these theoretical results on primates (Macaca mulatta) using a novel paradigm predicted by the model.

Assessing the Role of Inhibition in Stabilizing Neocortical Networks Requires Large-Scale Perturbation of the Inhibitory Population

Neurons within cortical microcircuits are interconnected with recurrent excitatory synaptic connections that are thought to amplify signals (Douglas and Martin, 2007), form selective subnetworks (Ko et al., 2011), and aid feature discrimination. Strong inhibition (Haider et al., 2013) counterbalances excitation, enabling sensory features to be sharpened and represented by sparse codes (Willmore et al., 2011). This balance between excitation and inhibition makes it difficult to assess the strength, or gain, of recurrent excitatory connections within cortical networks, which is key to understanding their operational regime and the computations that they perform. Networks that combine an unstable high-gain excitatory population with stabilizing inhibitory feedback are known as inhibition-stabilized networks (ISNs) (Tsodyks et al., 1997). Theoretical studies using reduced network models predict that ISNs produce paradoxical responses to perturbation, but experimental perturbations failed to find evidence for ISNs in cortex (Atallah et al., 2012). Here, we reexamined this question by investigating how cortical network models consisting of many neurons behave after perturbations and found that results obtained from reduced network models fail to predict responses to perturbations in more realistic networks. Our models predict that a large proportion of the inhibitory network must be perturbed to reliably detect an ISN regime robustly in cortex. We propose that wide-field optogenetic suppression of inhibition under promoters targeting a large fraction of inhibitory neurons may provide a perturbation of sufficient strength to reveal the operating regime of cortex. Our results suggest that detailed computational models of optogenetic perturbations are necessary to interpret the results of experimental paradigms.SIGNIFICANCE STATEMENT Many useful computational mechanisms proposed for cortex require local excitatory recurrence to be very strong, such that local inhibitory feedback is necessary to avoid epileptiform runaway activity (an "inhibition-stabilized network" or "ISN" regime). However, recent experimental results suggest that this regime may not exist in cortex. We simulated activity perturbations in cortical networks of increasing realism and found that, to detect ISN-like properties in cortex, large proportions of the inhibitory population must be perturbed. Current experimental methods for inhibitory perturbation are unlikely to satisfy this requirement, implying that existing experimental observations are inconclusive about the computational regime of cortex. Our results suggest that new experimental designs targeting a majority of inhibitory neurons may be able to resolve this question.

Specific excitatory connectivity for feature integration in mouse primary visual cortex

Local excitatory connections in mouse primary visual cortex (V1) are stronger and more prevalent between neurons that share similar functional response features. However, the details of how functional rules for local connectivity shape neuronal responses in V1 remain unknown. We hypothesised that complex responses to visual stimuli may arise as a consequence of rules for selective excitatory connectivity within the local network in the superficial layers of mouse V1. In mouse V1 many neurons respond to overlapping grating stimuli (plaid stimuli) with highly selective and facilitatory responses, which are not simply predicted by responses to single gratings presented alone. This complexity is surprising, since excitatory neurons in V1 are considered to be mainly tuned to single preferred orientations. Here we examined the consequences for visual processing of two alternative connectivity schemes: in the first case, local connections are aligned with visual properties inherited from feedforward input (a 'like-to-like' scheme specifically connecting neurons that share similar preferred orientations); in the second case, local connections group neurons into excitatory subnetworks that combine and amplify multiple feedforward visual properties (a 'feature binding' scheme). By comparing predictions from large scale computational models with in vivo recordings of visual representations in mouse V1, we found that responses to plaid stimuli were best explained by assuming feature binding connectivity. Unlike under the like-to-like scheme, selective amplification within feature-binding excitatory subnetworks replicated experimentally observed facilitatory responses to plaid stimuli; explained selective plaid responses not predicted by grating selectivity; and was consistent with broad anatomical selectivity observed in mouse V1. Our results show that visual feature binding can occur through local recurrent mechanisms without requiring feedforward convergence, and that such a mechanism is consistent with visual responses and cortical anatomy in mouse V1.

Rapid Long-Range Disynaptic Inhibition Explains the Formation of Cortical Orientation Maps

Competitive interactions are believed to underlie many types of cortical processing, ranging from memory formation, attention and development of cortical functional organization (e.g., development of orientation maps in primary visual cortex). In the latter case, the competitive interactions happen along the cortical surface, with local populations of neurons reinforcing each other, while competing with those displaced more distally. This specific configuration of lateral interactions is however in stark contrast with the known properties of the anatomical substrate, i.e., excitatory connections (mediating reinforcement) having longer reach than inhibitory ones (mediating competition). No satisfactory biologically plausible resolution of this conflict between anatomical measures, and assumed cortical function has been proposed. Recently a specific pattern of delays between different types of neurons in cat cortex has been discovered, where direct mono-synaptic excitation has approximately the same delay, as the combined delays of the disynaptic inhibitory interactions between excitatory neurons (i.e., the sum of delays from excitatory to inhibitory and from inhibitory to excitatory neurons). Here we show that this specific pattern of delays represents a biologically plausible explanation for how short-range inhibition can support competitive interactions that underlie the development of orientation maps in primary visual cortex. We demonstrate this statement analytically under simplifying conditions, and subsequently show using network simulations that development of orientation maps is preserved when long-range excitation, direct inhibitory to inhibitory interactions, and moderate inequality in the delays between excitatory and inhibitory pathways is added.

Formation and Dynamics of Waves in a Cortical Model of Cholinergic Modulation

Acetylcholine (ACh) is a regulator of neural excitability and one of the neurochemical substrates of sleep. Amongst the cellular effects induced by cholinergic modulation are a reduction in spike-frequency adaptation (SFA) and a shift in the phase response curve (PRC). We demonstrate in a biophysical model how changes in neural excitability and network structure interact to create three distinct functional regimes: localized asynchronous, traveling asynchronous, and traveling synchronous. Our results qualitatively match those observed experimentally. Cortical activity during slow wave sleep (SWS) differs from that during REM sleep or waking states. During SWS there are traveling patterns of activity in the cortex; in other states stationary patterns occur. Our model is a network composed of Hodgkin-Huxley type neurons with a M-current regulated by ACh. Regulation of ACh level can account for dynamical changes between functional regimes. Reduction of the magnitude of this current recreates the reduction in SFA the shift from a type 2 to a type 1 PRC observed in the presence of ACh. When SFA is minimal (in waking or REM sleep state, high ACh) patterns of activity are localized and easily pinned by network inhomogeneities. When SFA is present (decreasing ACh), traveling waves of activity naturally arise. A further decrease in ACh leads to a high degree of synchrony within traveling waves. We also show that the level of ACh determines how sensitive network activity is to synaptic heterogeneity. These regimes may have a profound functional significance as stationary patterns may play a role in the proper encoding of external input as memory and traveling waves could lead to synaptic regularization, giving unique insights into the role and significance of ACh in determining patterns of cortical activity and functional differences arising from the patterns.

Eigenspectrum bounds for semirandom matrices with modular and spatial structure for neural networks

The eigenvalue spectrum of the matrix of directed weights defining a neural network model is informative of several stability and dynamical properties of network activity. Existing results for eigenspectra of sparse asymmetric random matrices neglect spatial or other constraints in determining entries in these matrices, and so are of partial applicability to cortical-like architectures. Here we examine a parameterized class of networks that are defined by sparse connectivity, with connection weighting modulated by physical proximity (i.e., asymmetric Euclidean random matrices), modular network partitioning, and functional specificity within the excitatory population. We present a set of analytical constraints that apply to the eigenvalue spectra of associated weight matrices, highlighting the relationship between connectivity rules and classes of network dynamics.