Properties and Performance of Imperfect Dual Neural Network-Based kWTA Networks

Influence of Imperfections on the Operational Correctness of DNN-kWTA Model

The dual neural network (DNN)-based k -winner-take-all (WTA) model is able to identify the k largest numbers from its m input numbers. When there are imperfections, such as non-ideal step function and Gaussian input noise, in the realization, the model may not output the correct result. This brief analyzes the influence of the imperfections on the operational correctness of the model. Due to the imperfections, it is not efficient to use the original DNN- k WTA dynamics for analyzing the influence. In this regard, this brief first derives an equivalent model to describe the dynamics of the model under the imperfections. From the equivalent model, we derive a sufficient condition for which the model outputs the correct result. Thus, we apply the sufficient condition to design an efficiently estimation method for the probability of the model outputting the correct result. Furthermore, for the inputs with uniform distribution, a closed form expression for the probability value is derived. Finally, we extend our analysis for handling non-Gaussian input noise. Simulation results are provided to validate our theoretical results.

Robust k-WTA Network Generation, Analysis, and Applications to Multiagent Coordination

In this article, a robust k -winner-take-all ( k -WTA) neural network employing the saturation-allowed activation functions is designed and investigated to perform a k -WTA operation, and is shown to possess enhanced robustness to disturbance compared to existing k -WTA neural networks. Global convergence and robustness of the proposed k -WTA neural network are demonstrated through analysis and simulations. An application studied in detail is competitive multiagent coordination and dynamic task allocation, in which k active agents [among ] are allocated to execute a tracking task with the static m-k ones. This is implemented by adopting a distributed k -WTA network with limited communication, aided with a consensus filter. Simulation results demonstrating the system's efficacy and feasibility are presented.

DNN-kWTA With Bounded Random Offset Voltage Drifts in Threshold Logic Units

The dual neural network-based k -winner-take-all (DNN- k WTA) is an analog neural model that is used to identify the k largest numbers from n inputs. Since threshold logic units (TLUs) are key elements in the model, offset voltage drifts in TLUs may affect the operational correctness of a DNN- k WTA network. Previous studies assume that drifts in TLUs follow some particular distributions. This brief considers that only the drift range, given by [-∆, ∆] , is available. We consider two drift cases: time-invariant and time-varying. For the time-invariant case, we show that the state of a DNN- k WTA network converges. The sufficient condition to make a network with the correct operation is given. Furthermore, for uniformly distributed inputs, we prove that the probability that a DNN- k WTA network operates properly is greater than (1-2∆)ⁿ . The aforementioned results are generalized for the time-varying case. In addition, for the time-invariant case, we derive a method to compute the exact convergence time for a given data set. For uniformly distributed inputs, we further derive the mean and variance of the convergence time. The convergence time results give us an idea about the operational speed of the DNN- k WTA model. Finally, simulation experiments have been conducted to validate those theoretical results.

Design of a K-Winners-Take-All Model With a Binary Spike Train

A continuous-time K -winners-take-all (KWTA) neural model that can identify the largest K of N inputs, where command signal is described. The model is given by a differential equation where the spike train is a sum of delta functions. A functional block-diagram of the model includes N feed-forward hard-limiting neurons and one feedback neuron, used to handle input dynamics. The existence and uniqueness of the model steady states are analyzed, the convergence analysis of the state variable trajectories to the KWTA operation is proven, the convergence time and number of spikes required are derived, as well as the processing of time-varying inputs and perturbations of the model nonlinearities are analyzed. The main advantage of the model is that it is not subject to the intrinsic convergence of speed limitations of comparable designs. The model also has an arbitrary finite resolution determined by a given parameter, low complexity, and initial condition independence. Applications of the model for parallel sorting and parallel rank-order filtering are presented. Theoretical results are derived and illustrated with computer-simulated examples that demonstrate the model's performance.

On Wang $k$ WTA With Input Noise, Output Node Stochastic, and Recurrent State Noise

In this paper, the effect of input noise, output node stochastic, and recurrent state noise on the Wang $k$ WTA is analyzed. Here, we assume that noise exists at the recurrent state $y(t)$ and it can either be additive or multiplicative. Besides, its dynamical change (i.e., $dy/dt$ ) is corrupted by noise as well. In sequel, we model the dynamics of $y(t)$ as a stochastic differential equation and show that the stochastic behavior of $y(t)$ is equivalent to an Ito diffusion. Its stationary distribution is a Gibbs distribution, whose modality depends on the noise condition. With moderate input noise and very small recurrent state noise, the distribution is single modal and hence $y(\infty )$ has high probability varying within the input values of the $k$ and $k+1$ winners (i.e., correct output). With small input noise and large recurrent state noise, the distribution could be multimodal and hence $y(\infty )$ could have probability varying outside the input values of the $k$ and $k+1$ winners (i.e., incorrect output). In this regard, we further derive the conditions that the $k$ WTA has high probability giving correct output. Our results reveal that recurrent state noise could have severe effect on Wang $k$ WTA. But, input noise and output node stochastic could alleviate such an effect.

Robustness Analysis on Dual Neural Network-based $k$ WTA With Input Noise

This paper studies the effects of uniform input noise and Gaussian input noise on the dual neural network-based WTA (DNN- WTA) model. We show that the state of the network (under either uniform input noise or Gaussian input noise) converges to one of the equilibrium points. We then derive a formula to check if the network produce correct outputs or not. Furthermore, for the uniformly distributed inputs, two lower bounds (one for each type of input noise) on the probability that the network produces the correct outputs are presented. Besides, when the minimum separation amongst inputs is given, we derive the condition for the network producing the correct outputs. Finally, experimental results are presented to verify our theoretical results. Since random drift in the comparators can be considered as input noise, our results can be applied to the random drift situation.

A New Discrete-Time Multi-Constrained $K$-Winner-Take-All Recurrent Network and Its Application to Prioritized Scheduling

In this paper, we propose a novel discrete-time recurrent neural network aiming to resolve a new class of multi-constrained K-winner-take-all (K-WTA) problems. By facilitating specially designed asymmetric neuron weights, the proposed model is capable of operating in a fully parallel manner, thereby allowing true digital implementation. This paper also provides theorems that delineate the theoretical upper bound of the convergence latency, which is merely O(K). Importantly, via simulations, the average convergence time is close to O(1) in most general cases. Moreover, as the multi-constrained K-WTA problem degenerates to a traditional single-constrained problem, the upper bound becomes exactly two parallel iterations, which significantly outperforms the existing K-WTA models. By associating the neurons and neuron weights with routing paths and path priorities, respectively, we then apply the model to a prioritized flow scheduler for the data center networks. Through extensive simulations, we demonstrate that the proposed scheduler converges to the equilibrium state within near-constant time for different scales of networks while achieving maximal throughput, quality-of-service priority differentiation, and minimum energy consumption, subject to the flow contention-free constraints.