Dynamics of winner-take-all competition in recurrent neural networks with lateral inhibition

Multi-Stimuli-Responsive Synapse Based on Vertical van der Waals Heterostructures

Brain-inspired intelligent systems demand diverse neuromorphic devices beyond simple functionalities. Merging biomimetic sensing with weight-updating capabilities in artificial synaptic devices represents one of the key research focuses. Here, we report a multiresponsive synapse device that integrates synaptic and optical-sensing functions. The device adopts vertically stacked graphene/h-BN/WSe₂ heterostructures, including an ultrahigh-mobility readout layer, a weight-control layer, and a dual-stimuli-responsive layer. The unique structure endows synapse devices with excellent synaptic plasticity, short response time (3 μs), and excellent optical responsivity (10⁵ A/W). To demonstrate the application in neuromorphic computing, handwritten digit recognition was simulated based on an unsupervised spiking neural network (SNN) with a precision of 90.89%, well comparable with the state-of-the-art results. Furthermore, multiterminal neuromorphic devices are demonstrated to mimic dendritic integration and photoswitching logic. Different from other synaptic devices, the research work validates multifunctional integration in synaptic devices, supporting the potential fusion of sensing and self-learning in neuromorphic networks.

Trading mental effort for confidence in the metacognitive control of value-based decision-making

Why do we sometimes opt for actions or items that we do not value the most? Under current neurocomputational theories, such preference reversals are typically interpreted in terms of errors that arise from the unreliable signaling of value to brain decision systems. But, an alternative explanation is that people may change their mind because they are reassessing the value of alternative options while pondering the decision. So, why do we carefully ponder some decisions, but not others? In this work, we derive a computational model of the metacognitive control of decisions or MCD. In brief, we assume that fast and automatic processes first provide initial (and largely uncertain) representations of options' values, yielding prior estimates of decision difficulty. These uncertain value representations are then refined by deploying cognitive (e.g., attentional, mnesic) resources, the allocation of which is controlled by an effort-confidence tradeoff. Importantly, the anticipated benefit of allocating resources varies in a decision-by-decision manner according to the prior estimate of decision difficulty. The ensuing MCD model predicts response time, subjective feeling of effort, choice confidence, changes of mind, as well as choice-induced preference change and certainty gain. We test these predictions in a systematic manner, using a dedicated behavioral paradigm. Our results provide a quantitative link between mental effort, choice confidence, and preference reversals, which could inform interpretations of related neuroimaging findings.

Artificial Intelligence Applications in Military Systems and Their Influence on Sense of Security of Citizens

The paper presents an overview of current and expected prospects for the development of artificial intelligence algorithms, especially in military applications, and conducted research regarding applications in the area of civilian life. Attention was paid mainly to the use of AI algorithms in cybersecurity, object detection, military logistics and robotics. It discusses the problems connected with the present solutions and how artificial intelligence can help solve them. It briefly presents also mathematical structures and descriptions for ART, CNN and SVM networks as well as Expectation–Maximization and Gaussian Mixture Model algorithms that are used in solving of discussed problems. The third chapter discusses the attitude of society towards the use of neural network algorithms in military applications. The basic problems related to ethics in the application of artificial intelligence and issues of responsibility for errors made by autonomous systems are discussed.

Robust parallel decision-making in neural circuits with nonlinear inhibition

Animals frequently need to choose the best alternative from a set of possibilities, whether it is which direction to swim in or which food source to favor. How long should a network of neurons take to choose the best of N options? Theoretical results suggest that the optimal time grows as log(N), if the values of each option are imperfectly perceived. However, standard self-terminating neural network models of decision-making cannot achieve this optimal behavior. We show how using certain additional nonlinear response properties in neurons, which are ignored in standard models, results in a decision-making architecture that both achieves the optimal scaling of decision time and accounts for multiple experimentally observed features of neural decision-making.

An elemental computation in the brain is to identify the best in a set of options and report its value. It is required for inference, decision-making, optimization, action selection, consensus, and foraging. Neural computing is considered powerful because of its parallelism; however, it is unclear whether neurons can perform this max-finding operation in a way that improves upon the prohibitively slow optimal serial max-finding computation (which takes ∼N⁡log(N) time for N noisy candidate options) by a factor of N, the benchmark for parallel computation. Biologically plausible architectures for this task are winner-take-all (WTA) networks, where individual neurons inhibit each other so only those with the largest input remain active. We show that conventional WTA networks fail the parallelism benchmark and, worse, in the presence of noise, altogether fail to produce a winner when N is large. We introduce the nWTA network, in which neurons are equipped with a second nonlinearity that prevents weakly active neurons from contributing inhibition. Without parameter fine-tuning or rescaling as N varies, the nWTA network achieves the parallelism benchmark. The network reproduces experimentally observed phenomena like Hick’s law without needing an additional readout stage or adaptive N-dependent thresholds. Our work bridges scales by linking cellular nonlinearities to circuit-level decision-making, establishes that distributed computation saturating the parallelism benchmark is possible in networks of noisy, finite-memory neurons, and shows that Hick’s law may be a symptom of near-optimal parallel decision-making with noisy input.

Spike-Based Winner-Take-All Computation: Fundamental Limits and Order-Optimal Circuits

Winner-take-all (WTA) refers to the neural operation that selects a (typically small) group of neurons from a large neuron pool. It is conjectured to underlie many of the brain's fundamental computational abilities. However, not much is known about the robustness of a spike-based WTA network to the inherent randomness of the input spike trains. In this work, we consider a spike-based k-WTA model wherein n randomly generated input spike trains compete with each other based on their underlying firing rates and k winners are supposed to be selected. We slot the time evenly with each time slot of length 1 ms and model the n input spike trains as n independent Bernoulli processes. We analytically characterize the minimum waiting time needed so that a target minimax decision accuracy (success probability) can be reached. We first derive an information-theoretic lower bound on the waiting time. We show that to guarantee a (minimax) decision error ≤δ (where δ∈(0,1)), the waiting time of any WTA circuit is at least [Formula: see text]where R⊆(0,1) is a finite set of rates and TR is a difficulty parameter of a WTA task with respect to set R for independent input spike trains. Additionally, TR is independent of δ, n, and k. We then design a simple WTA circuit whose waiting time is [Formula: see text]provided that the local memory of each output neuron is sufficiently long. It turns out that for any fixed δ, this decision time is order-optimal (i.e., it matches the above lower bound up to a multiplicative constant factor) in terms of its scaling in n, k, and TR.

Network structure and input integration in competing firing rate models for decision-making

Making a decision among numerous alternatives is a pervasive and central undertaking encountered by mammals in natural settings. While decision making for two-option tasks has been studied extensively both experimentally and theoretically, characterizing decision making in the face of a large set of alternatives remains challenging. We explore this issue by formulating a scalable mechanistic network model for decision making and analyzing the dynamics evoked given various potential network structures. In the case of a fully-connected network, we provide an analytical characterization of the model fixed points and their stability with respect to winner-take-all behavior for fair tasks. We compare several means of input integration, demonstrating a more gradual sigmoidal transfer function is likely evolutionarily advantageous relative to binary gain commonly utilized in engineered systems. We show via asymptotic analysis and numerical simulation that sigmoidal transfer functions with smaller steepness yield faster response times but depreciation in accuracy. However, in the presence of noise or degradation of connections, a sigmoidal transfer function garners significantly more robust and accurate decision-making dynamics. For fair tasks and sigmoidal gain, our model network also exhibits a stable parameter regime that produces high accuracy and persists across tasks with diverse numbers of alternatives and difficulties, satisfying physiological energetic constraints. In the case of more sparse and structured network topologies, including random, regular, and small-world connectivity, we show the high-accuracy parameter regime persists for biologically realistic connection densities. Our work shows how neural system architecture is potentially optimal in making economic, reliable, and advantageous decisions across tasks.

From statistical inference to a differential learning rule for stochastic neural networks

Stochastic neural networks are a prototypical computational device able to build a probabilistic representation of an ensemble of external stimuli. Building on the relationship between inference and learning, we derive a synaptic plasticity rule that relies only on delayed activity correlations, and that shows a number of remarkable features. Our delayed-correlations matching (DCM) rule satisfies some basic requirements for biological feasibility: finite and noisy afferent signals, Dale’s principle and asymmetry of synaptic connections, locality of the weight update computations. Nevertheless, the DCM rule is capable of storing a large, extensive number of patterns as attractors in a stochastic recurrent neural network, under general scenarios without requiring any modification: it can deal with correlated patterns, a broad range of architectures (with or without hidden neuronal states), one-shot learning with the palimpsest property, all the while avoiding the proliferation of spurious attractors. When hidden units are present, our learning rule can be employed to construct Boltzmann machine-like generative models, exploiting the addition of hidden neurons in feature extraction and classification tasks.

The Striatum Organizes 3D Behavior via Moment-to-Moment Action Selection

Many naturalistic behaviors are built from modular components that are expressed sequentially. Although striatal circuits have been implicated in action selection and implementation, the neural mechanisms that compose behavior in unrestrained animals are not well understood. Here, we record bulk and cellular neural activity in the direct and indirect pathways of dorsolateral striatum (DLS) as mice spontaneously express action sequences. These experiments reveal that DLS neurons systematically encode information about the identity and ordering of sub-second 3D behavioral motifs; this encoding is facilitated by fast-timescale decorrelations between the direct and indirect pathways. Furthermore, lesioning the DLS prevents appropriate sequence assembly during exploratory or odor-evoked behaviors. By characterizing naturalistic behavior at neural timescales, these experiments identify a code for elemental 3D pose dynamics built from complementary pathway dynamics, support a role for DLS in constructing meaningful behavioral sequences, and suggest models for how actions are sculpted over time.

Winner-take-all in a phase oscillator system with adaptation

We consider a system of generalized phase oscillators with a central element and radial connections. In contrast to conventional phase oscillators of the Kuramoto type, the dynamic variables in our system include not only the phase of each oscillator but also the natural frequency of the central oscillator, and the connection strengths from the peripheral oscillators to the central oscillator. With appropriate parameter values the system demonstrates winner-take-all behavior in terms of the competition between peripheral oscillators for the synchronization with the central oscillator. Conditions for the winner-take-all regime are derived for stationary and non-stationary types of system dynamics. Bifurcation analysis of the transition from stationary to non-stationary winner-take-all dynamics is presented. A new bifurcation type called a Saddle Node on Invariant Torus (SNIT) bifurcation was observed and is described in detail. Computer simulations of the system allow an optimal choice of parameters for winner-take-all implementation.

Structured Pyramidal Neural Networks

The Pyramidal Neural Networks (PNN) are an example of a successful recently proposed model inspired by the human visual system and deep learning theory. PNNs are applied to computer vision and based on the concept of receptive fields. This paper proposes a variation of PNN, named here as Structured Pyramidal Neural Network (SPNN). SPNN has self-adaptive variable receptive fields, while the original PNNs rely on the same size for the fields of all neurons, which limits the model since it is not possible to put more computing resources in a particular region of the image. Another limitation of the original approach is the need to define values for a reasonable number of parameters, which can turn difficult the application of PNNs in contexts in which the user does not have experience. On the other hand, SPNN has a fewer number of parameters. Its structure is determined using a novel method with Delaunay Triangulation and k-means clustering. SPNN achieved better results than PNNs and similar performance when compared to Convolutional Neural Network (CNN) and Support Vector Machine (SVM), but using lower memory capacity and processing time.

Intrinsic Plasticity for Natural Competition in Koniocortex-Like Neural Networks

In this paper, we use the neural property known as intrinsic plasticity to develop neural network models that resemble the koniocortex, the fourth layer of sensory cortices. These models evolved from a very basic two-layered neural network to a complex associative koniocortex network. In the initial network, intrinsic and synaptic plasticity govern the shifting of the activation function, and the modification of synaptic weights, respectively. In this first version, competition is forced, so that the most activated neuron is arbitrarily set to one and the others to zero, while in the second, competition occurs naturally due to inhibition between second layer neurons. In the third version of the network, whose architecture is similar to the koniocortex, competition also occurs naturally owing to the interplay between inhibitory interneurons and synaptic and intrinsic plasticity. A more complex associative neural network was developed based on this basic koniocortex-like neural network, capable of dealing with incomplete patterns and ideally suited to operating similarly to a learning vector quantization network. We also discuss the biological plausibility of the networks and their role in a more complex thalamocortical model.

Solving nonlinear equality constrained multiobjective optimization problems using neural networks

This paper develops a neural network architecture and a new processing method for solving in real time, the nonlinear equality constrained multiobjective optimization problem (NECMOP), where several nonlinear objective functions must be optimized in a conflicting situation. In this processing method, the NECMOP is converted to an equivalent scalar optimization problem (SOP). The SOP is then decomposed into several-separable subproblems processable in parallel and in a reasonable time by multiplexing switched capacitor circuits. The approach which we propose makes use of a decomposition-coordination principle that allows nonlinearity to be treated at a local level and where coordination is achieved through the use of Lagrange multipliers. The modularity and the regularity of the neural networks architecture herein proposed make it suitable for very large scale integration implementation. An application to the resolution of a physical problem is given to show that the approach used here possesses some advantages of the point of algorithmic view, and provides processes of resolution often simpler than the usual techniques.

Neuronal networks with NMDARs and lateral inhibition implement winner-takes-all

A neural circuit that relies on the electrical properties of NMDA synaptic receptors is shown by numerical and theoretical analysis to be capable of realizing the winner-takes-all function, a powerful computational primitive that is often attributed to biological nervous systems. This biophysically-plausible model employs global lateral inhibition in a simple feedback arrangement. As its inputs increase, high-gain and then bi- or multi-stable equilibrium states may be assumed in which there is significant depolarization of a single neuron and hyperpolarization or very weak depolarization of other neurons in the network. The state of the winning neuron conveys analog information about its input. The winner-takes-all characteristic depends on the nonmonotonic current-voltage relation of NMDA receptor ion channels, as well as neural thresholding, and the gain and nature of the inhibitory feedback. Dynamical regimes vary with input strength. Fixed points may become unstable as the network enters a winner-takes-all regime, which can lead to entrained oscillations. Under some conditions, oscillatory behavior can be interpreted as winner-takes-all in nature. Stable winner-takes-all behavior is typically recovered as inputs increase further, but with still larger inputs, the winner-takes-all characteristic is ultimately lost. Network stability may be enhanced by biologically plausible mechanisms.

Constructive autoassociative neural network for facial recognition

Autoassociative artificial neural networks have been used in many different computer vision applications. However, it is difficult to define the most suitable neural network architecture because this definition is based on previous knowledge and depends on the problem domain. To address this problem, we propose a constructive autoassociative neural network called CANet (Constructive Autoassociative Neural Network). CANet integrates the concepts of receptive fields and autoassociative memory in a dynamic architecture that changes the configuration of the receptive fields by adding new neurons in the hidden layer, while a pruning algorithm removes neurons from the output layer. Neurons in the CANet output layer present lateral inhibitory connections that improve the recognition rate. Experiments in face recognition and facial expression recognition show that the CANet outperforms other methods presented in the literature.

Transient and steady-state selection in the striatal microcircuit

Although the basal ganglia have been widely studied and implicated in signal processing and action selection, little information is known about the active role the striatal microcircuit plays in action selection in the basal ganglia-thalamo-cortical loops. To address this knowledge gap we use a large scale three dimensional spiking model of the striatum, combined with a rate coded model of the basal ganglia-thalamo-cortical loop, to asses the computational role the striatum plays in action selection. We identify a robust transient phenomena generated by the striatal microcircuit, which temporarily enhances the difference between two competing cortical inputs. We show that this transient is sufficient to modulate decision making in the basal ganglia-thalamo-cortical circuit. We also find that the transient selection originates from a novel adaptation effect in single striatal projection neurons, which is amenable to experimental testing. Finally, we compared transient selection with models implementing classical steady-state selection. We challenged both forms of model to account for recent reports of paradoxically enhanced response selection in Huntington's disease patients. We found that steady-state selection was uniformly impaired under all simulated Huntington's conditions, but transient selection was enhanced given a sufficient Huntington's-like increase in NMDA receptor sensitivity. Thus our models provide an intriguing hypothesis for the mechanisms underlying the paradoxical cognitive improvements in manifest Huntington's patients.

Lateral inhibition pyramidal neural network for image classification

The human visual system is one of the most fascinating and complex mechanisms of the central nervous system that enables our capacity to see. It is through the visual system that we are able to accomplish from the most simple task such as object recognition to the most complex visual interpretation, understanding and perception. Inspired by this sophisticated system, two models based on the properties of the human visual system are proposed. These models are designed based on the concepts of receptive and inhibitory fields. The first model is a pyramidal neural network with lateral inhibition, called lateral inhibition pyramidal neural network. The second proposed model is a supervised image segmentation system, called segmentation and classification based on receptive fields. This work shows that the combination of these two models is beneficial, and the results obtained are better than that of other state-of-the-art methods.

Transition from amplitude to oscillation death via Turing bifurcation

Coupled oscillators are shown to experience two structurally different oscillation quenching types: amplitude death (AD) and oscillation death (OD). We demonstrate that both AD and OD can occur in one system and find that the transition between them underlies a classical, Turing-type bifurcation, providing a clear classification of these significantly different dynamical regimes. The implications of obtaining a homogeneous (AD) or inhomogeneous (OD) steady state, as well as their significance for physical and biological applications and control studies, are also pointed out.

Feedforward inhibition and synaptic scaling--two sides of the same coin?

Feedforward inhibition and synaptic scaling are important adaptive processes that control the total input a neuron can receive from its afferents. While often studied in isolation, the two have been reported to co-occur in various brain regions. The functional implications of their interactions remain unclear, however. Based on a probabilistic modeling approach, we show here that fast feedforward inhibition and synaptic scaling interact synergistically during unsupervised learning. In technical terms, we model the input to a neural circuit using a normalized mixture model with Poisson noise. We demonstrate analytically and numerically that, in the presence of lateral inhibition introducing competition between different neurons, Hebbian plasticity and synaptic scaling approximate the optimal maximum likelihood solutions for this model. Our results suggest that, beyond its conventional use as a mechanism to remove undesired pattern variations, input normalization can make typical neural interaction and learning rules optimal on the stimulus subspace defined through feedforward inhibition. Furthermore, learning within this subspace is more efficient in practice, as it helps avoid locally optimal solutions. Our results suggest a close connection between feedforward inhibition and synaptic scaling which may have important functional implications for general cortical processing.

The inputs a neuron receives from its presynaptic partners strongly fluctuate as a result of either varying sensory information or ongoing intrinsic activity. To represent this wide range of signals effectively, neurons use various mechanisms that regulate the total input they receive. On the one hand, feedforward inhibition adjusts the relative contribution of individual inputs inversely proportional to the total number of active afferents, implementing a form of input normalization. On the other hand, synaptic scaling uniformly rescales the efficacy of incoming synapses to stabilize the neuron's firing rate after learning-induced changes in drive. Given that these mechanisms often act on the same neurons, we ask here if there are any benefits in combining the two. We show that the interaction between the two has important computational consequences, beyond their traditional role in maintaining network homeostasis. When combined with lateral inhibition, synaptic scaling and fast feedforward inhibition allow the circuit to learn efficiently from noisy, ambiguous inputs. For inputs not normalized by feed-forward inhibition, learning is less efficient. Given that feed-forward inhibition and synaptic scaling have been reported in various systems, our results suggest that they could generally facilitate learning in neural circuits. More broadly, our work emphasizes the importance of studying the interaction between different plasticity mechanisms for understanding circuit function.

Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D(1) alone for appetitive eating but D(1) and D(2) together for fear

The medial shell of nucleus accumbens (NAc) and its mesolimbic dopamine inputs mediate forms of fearful as well as of incentive motivation. For example, either appetitive and/or actively fearful behaviors are generated in a keyboard pattern by localized glutamate disruptions in NAc (via microinjection of the AMPA receptor antagonist DNQX) at different anatomical locations along a rostrocaudal gradient within the medial shell of rats. Rostral glutamate disruptions produce intense increases in eating, but more caudally placed disruptions produce increasingly fearful behaviors: distress vocalizations and escape attempts to human touch, and a spontaneous and directed antipredator response called defensive treading/burying. Local endogenous dopamine is required for either intense motivation to be generated by AMPA disruptions. Here we report that only endogenous local signaling at D(1) dopamine receptors is needed for rostral generation of excessive eating, potentially implicating a direct output pathway contribution. In contrast, fear generation at caudal sites requires both D(1) and D(2) signaling simultaneously, potentially implicating an indirect output pathway contribution. Finally, when motivation valence generated by AMPA disruptions at intermediate sites was flipped by manipulating environmental ambience, from mostly appetitive in a comfortable home environment to mostly fearful in a stressful environment, the roles of local D(1) and D(2) signaling in dopamine/glutamate interaction at microinjection sites also switched dynamically to match the motivation valence generated at the moment. Thus, NAc D(1) and D(2) receptors, and their associated neuronal circuits, play different and dynamic roles in enabling desire and dread to be generated by localized NAc glutamate disruptions in medial shell.

Recurrent antitopographic inhibition mediates competitive stimulus selection in an attention network

Topographically organized neurons represent multiple stimuli within complex visual scenes and compete for subsequent processing in higher visual centers. The underlying neural mechanisms of this process have long been elusive. We investigate an experimentally constrained model of a midbrain structure: the optic tectum and the reciprocally connected nucleus isthmi. We show that a recurrent antitopographic inhibition mediates the competitive stimulus selection between distant sensory inputs in this visual pathway. This recurrent antitopographic inhibition is fundamentally different from surround inhibition in that it projects on all locations of its input layer, except to the locus from which it receives input. At a larger scale, the model shows how a focal top-down input from a forebrain region, the arcopallial gaze field, biases the competitive stimulus selection via the combined activation of a local excitation and the recurrent antitopographic inhibition. Our findings reveal circuit mechanisms of competitive stimulus selection and should motivate a search for anatomical implementations of these mechanisms in a range of vertebrate attentional systems.

Taste-specific cell assemblies in a biologically informed model of the nucleus of the solitary tract

Although the cellular organization of many primary sensory nuclei has been well characterized, questions remain about the functional architecture of the first central relay for gustation, the rostral nucleus of the solitary tract (NTS). Here we used electrophysiological data recorded from single cells in the NTS to inform a network model of taste processing. Previous studies showed that electrical stimulation of the chorda tympani (CT) nerve initiates two types of inhibitory influences with different time courses in separate groups of NTS cells. Each type of inhibition targeted cells with distinct taste response properties. Further analyses of these data identified three NTS cell types differentiated by their latency of evoked response, time course of CT evoked inhibition, and degree of selectivity across taste qualities. Based on these results, we designed a model of the NTS consisting of discrete, reciprocally connected, stimulus-specific "cell" assemblies. Input to the network of integrate-and-fire model neurons was based on electrophysiological recordings from the CT nerve. Following successful simulation of paired-pulse CT stimulation, the network was tested for its ability to discriminate between two "taste" stimuli. Network dynamics of the model produced biologically plausible responses from each unit type and enhanced discrimination between taste qualities. We propose that an interactive network of taste quality specific cell assemblies, similar to our model, may account for the coherence in across-neuron patterns of NTS responses between similar tastants.

Nontrivial global attractors in 2-D multistable attractor neural networks

Attractor dynamics is a crucial problem for attractor neural networks, as it is the underling computational mechanism for memory storage and retrieval in neural systems. This brief studies a class of attractor network consisting of linearized threshold neurons, and analyzes global attractors based on a parameterized 2-D model. On the basis of previous results on nondegenerate and degenerate equilibria in mathematics, we further elucidate all possible nontrivial global attractors. Our theoretical result provides precise descriptions on how the changes of network parameters affect the attractors' distribution and landscape, and it may give a feasible solution towards specifying attractors by specifying weights. Simulations are presented to illustrate the theoretical results.

Spatio-temporal adaptation in the unsupervised development of networked visual neurons

There have been many computational models mimicking the visual cortex that are based on spatial adaptations of unsupervised neural networks. In this paper, we present a new model called neuronal cluster which includes spatial as well as temporal weights in its unified adaptation scheme. The "in-place" nature of the model is based on two biologically plausible learning rules, Hebbian rule and lateral inhibition. We present the mathematical demonstration that the temporal weights are derived from the delay in lateral inhibition. By training with the natural videos, this model can develop spatio-temporal features such as orientation selective cells, motion sensitive cells, and spatio-temporal complex cells. The unified nature of the adaptation scheme allows us to construct a multilayered and task-independent attention selection network which uses the same learning rule for edge, motion, and color detection, and we can use this network to engage in attention selection in both static and dynamic scenes.

Analysis of continuous attractors for 2-D linear threshold neural networks

This brief investigates continuous attractors of the well-developed model in visual cortex, i.e., the linear threshold (LT) neural networks, based on a parameterized 2-D model. On the basis of existing results on nondegenerate equilibria in mathematics, we further discuss degenerate equilibria for such networks and present properties and distributions of the equilibria, which enables us to draw the coexistence conditions of nondegenerate and degenerate equilibria (e.g., singular lines). Our theoretical results provide a useful framework for precise tuning on the network parameters, e.g., the feedbacks and visual inputs. Simulations are also presented to illustrate the theoretical findings.

Equilibria and their bifurcations in a recurrent neural network involving iterates of a transcendental function

Some practical models contain so complicated mathematical expressions that it is hard to determine the number and distribution of all equilibria, not mentioning the qualitative properties and bifurcations of those equilibria. The three-node recurrent neural network system with two free weight parameters, originally introduced by Ruiz, Owens, and Townley in 1997, is such a system, for which the equation of equilibria involves transcendental function tanh(x) and its iterates. Not computing coordinates of its equilibria, in this paper, we display an effective technique to determine the number and distribution of its equilibria. Without full information about equilibria, our method enables to further study qualitative properties of those equilibria and discuss their saddle node, pitchfork, and Hopf bifurcations by approximating center manifolds.

Multiperiodicity and attractivity of delayed recurrent neural networks with unsaturating piecewise linear transfer functions

This paper studies multiperiodicity and attractivity for a class of recurrent neural networks (RNNs) with unsaturating piecewise linear transfer functions and variable delays. Using local inhibition, conditions for boundedness and global attractivity are established. These conditions allow coexistence of stable and unstable trajectories. Moreover, multiperiodicity of the network is investigated by using local invariant sets. It shows that under some interesting conditions, there exists one periodic trajectory in each invariant set which exponentially attracts all trajectories in that region correspondingly. Simulations are carried out to illustrate the theories.

Mechanisms for Frequency Control in Neuronal Competition Models

We investigate analytically a firing rate model for a two-population network based on mutual inhibition and slow negative feedback in the form of spike frequency adaptation. Both neuronal populations receive external constant input whose strength determines the system's dynamical state-a steady state of identical activity levels or periodic oscillations or a winner-take-all state of bistability. We prove that oscillations appear in the system through supercritical Hopf bifurcations and that they are antiphase. The period of oscillations depends on the input strength in a nonmonotonic fashion, and we show that the increasing branch of the period versus input curve corresponds to a release mechanism and the decreasing branch to an escape mechanism. In the limiting case of infinitely slow feedback we characterize the conditions for release, escape, and occurrence of the winner-take-all behavior. Some extensions of the model are also discussed.