Design of oscillatory neural networks by machine learning

We demonstrate the utility of machine learning algorithms for the design of oscillatory neural networks (ONNs). After constructing a circuit model of the oscillators in a machine-learning-enabled simulator and performing Backpropagation through time (BPTT) for determining the coupling resistances between the ring oscillators, we demonstrate the design of associative memories and multi-layered ONN classifiers. The machine-learning-designed ONNs show superior performance compared to other design methods (such as Hebbian learning), and they also enable significant simplifications in the circuit topology. We also demonstrate the design of multi-layered ONNs that show superior performance compared to single-layer ones. We argue that machine learning can be a valuable tool to unlock the true computing potential of ONNs hardware.

Experimental demonstration of coupled differential oscillator networks for versatile applications

Oscillatory neural networks (ONNs) exhibit a high potential for energy-efficient computing. In ONNs, neurons are implemented with oscillators and synapses with resistive and/or capacitive coupling between pairs of oscillators. Computing is carried out on the basis of the rich, complex, non-linear synchronization dynamics of a system of coupled oscillators. The exploited synchronization phenomena in ONNs are an example of fully parallel collective computing. A fast system's convergence to stable states, which correspond to the desired processed information, enables an energy-efficient solution if small area and low-power oscillators are used, specifically when they are built on the basis of the hysteresis exhibited by phase-transition materials such as VO2. In recent years, there have been numerous studies on ONNs using VO2. Most of them report simulation results. Although in some cases experimental results are also shown, they do not implement the design techniques that other works on electrical simulations report that allow to improve the behavior of the ONNs. Experimental validation of these approaches is necessary. Therefore, in this study, we describe an ONN realized in a commercial CMOS technology in which the oscillators are built using a circuit that we have developed to emulate the VO2 device. The purpose is to be able to study in-depth the synchronization dynamics of relaxation oscillators similar to those that can be performed with VO2 devices. The fabricated circuit is very flexible. It allows programming the synapses to implement different ONNs, calibrating the frequency of the oscillators, or controlling their initialization. It uses differential oscillators and resistive synapses, equivalent to the use of memristors. In this article, the designed and fabricated circuits are described in detail, and experimental results are shown. Specifically, its satisfactory operation as an associative memory is demonstrated. The experiments carried out allow us to conclude that the ONN must be operated according to the type of computational task to be solved, and guidelines are extracted in this regard.

Oscillatory neural network learning for pattern recognition: an on-chip learning perspective and implementation

In the human brain, learning is continuous, while currently in AI, learning algorithms are pre-trained, making the model non-evolutive and predetermined. However, even in AI models, environment and input data change over time. Thus, there is a need to study continual learning algorithms. In particular, there is a need to investigate how to implement such continual learning algorithms on-chip. In this work, we focus on Oscillatory Neural Networks (ONNs), a neuromorphic computing paradigm performing auto-associative memory tasks, like Hopfield Neural Networks (HNNs). We study the adaptability of the HNN unsupervised learning rules to on-chip learning with ONN. In addition, we propose a first solution to implement unsupervised on-chip learning using a digital ONN design. We show that the architecture enables efficient ONN on-chip learning with Hebbian and Storkey learning rules in hundreds of microseconds for networks with up to 35 fully-connected digital oscillators.

Digital Implementation of Oscillatory Neural Network for Image Recognition Applications

Computing paradigm based on von Neuman architectures cannot keep up with the ever-increasing data growth (also called “data deluge gap”). This has resulted in investigating novel computing paradigms and design approaches at all levels from materials to system-level implementations and applications. An alternative computing approach based on artificial neural networks uses oscillators to compute or Oscillatory Neural Networks (ONNs). ONNs can perform computations efficiently and can be used to build a more extensive neuromorphic system. Here, we address a fundamental problem: can we efficiently perform artificial intelligence applications with ONNs? We present a digital ONN implementation to show a proof-of-concept of the ONN approach of “computing-in-phase” for pattern recognition applications. To the best of our knowledge, this is the first attempt to implement an FPGA-based fully-digital ONN. We report ONN accuracy, training, inference, memory capacity, operating frequency, hardware resources based on simulations and implementations of 5 × 3 and 10 × 6 ONNs. We present the digital ONN implementation on FPGA for pattern recognition applications such as performing digits recognition from a camera stream. We discuss practical challenges and future directions in implementing digital ONN.

Hardware Implementation of Differential Oscillatory Neural Networks Using VO 2-Based Oscillators and Memristor-Bridge Circuits

Oscillatory Neural Networks (ONNs) are currently arousing interest in the research community for their potential to implement very fast, ultra-low-power computing tasks by exploiting specific emerging technologies. From the architectural point of view, ONNs are based on the synchronization of oscillatory neurons in cognitive processing, as occurs in the human brain. As emerging technologies, VO₂ and memristive devices show promising potential for the efficient implementation of ONNs. Abundant literature is now becoming available pertaining to the study and building of ONNs based on VO₂ devices and resistive coupling, such as memristors. One drawback of direct resistive coupling is that physical resistances cannot be negative, but from the architectural and computational perspective this would be a powerful advantage when interconnecting weights in ONNs. Here we solve the problem by proposing a hardware implementation technique based on differential oscillatory neurons for ONNs (DONNs) with VO₂-based oscillators and memristor-bridge circuits. Each differential oscillatory neuron is made of a pair of VO₂ oscillators operating in anti-phase. This way, the neurons provide a pair of differential output signals in opposite phase. The memristor-bridge circuit is used as an adjustable coupling function that is compatible with differential structures and capable of providing both positive and negative weights. By combining differential oscillatory neurons and memristor-bridge circuits, we propose the hardware implementation of a fully connected differential ONN (DONN) and use it as an associative memory. The standard Hebbian rule is used for training, and the weights are then mapped to the memristor-bridge circuit through a proposed mapping rule. The paper also introduces some functional and hardware specifications to evaluate the design. Evaluation is performed by circuit-level electrical simulations and shows that the retrieval accuracy of the proposed design is comparable to that of classic Hopfield Neural Networks.