Oscillatory neural network learning for pattern recognition: an on-chip learning perspective and implementation

In the human brain, learning is continuous, while currently in AI, learning algorithms are pre-trained, making the model non-evolutive and predetermined. However, even in AI models, environment and input data change over time. Thus, there is a need to study continual learning algorithms. In particular, there is a need to investigate how to implement such continual learning algorithms on-chip. In this work, we focus on Oscillatory Neural Networks (ONNs), a neuromorphic computing paradigm performing auto-associative memory tasks, like Hopfield Neural Networks (HNNs). We study the adaptability of the HNN unsupervised learning rules to on-chip learning with ONN. In addition, we propose a first solution to implement unsupervised on-chip learning using a digital ONN design. We show that the architecture enables efficient ONN on-chip learning with Hebbian and Storkey learning rules in hundreds of microseconds for networks with up to 35 fully-connected digital oscillators.

Energy-Performance Assessment of Oscillatory Neural Networks Based on VO 2 Devices for Future Edge AI Computing

Oscillatory neural network (ONN) is an emerging neuromorphic architecture composed of oscillators that implement neurons and are coupled by synapses. ONNs exhibit rich dynamics and associative properties, which can be used to solve problems in the analog domain according to the paradigm let physics compute. For example, compact oscillators made of VO 2 material are good candidates for building low-power ONN architectures dedicated to AI applications at the edge, like pattern recognition. However, little is known about the ONN scalability and its performance when implemented in hardware. Before deploying ONN, it is necessary to assess its computation time, energy consumption, performance, and accuracy for a given application. Here, we consider a VO 2 -oscillator as an ONN building block and perform circuit-level simulations to evaluate the ONN performances at the architecture level. Notably, we investigate how the ONN computation time, energy, and memory capacity scale with the number of oscillators. It appears that the ONN energy grows linearly when scaling up the network, making it suitable for large-scale integration at the edge. Furthermore, we investigate the design knobs for minimizing the ONN energy. Assisted by technology computer-aided design (TCAD) simulations, we report on scaling down the dimensions of VO 2 devices in crossbar (CB) geometry to decrease the oscillator voltage and energy. We benchmark ONN versus state-of-the-art architectures and observe that the ONN paradigm is a competitive energy-efficient solution for scaled VO 2 devices oscillating above 100 MHz. Finally, we present how ONN can efficiently detect edges in images captured on low-power edge devices and compare the results with Sobel and Canny edge detectors.

How Frequency Injection Locking Can Train Oscillatory Neural Networks to Compute in Phase

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model-information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.

Digital Implementation of Oscillatory Neural Network for Image Recognition Applications

Computing paradigm based on von Neuman architectures cannot keep up with the ever-increasing data growth (also called “data deluge gap”). This has resulted in investigating novel computing paradigms and design approaches at all levels from materials to system-level implementations and applications. An alternative computing approach based on artificial neural networks uses oscillators to compute or Oscillatory Neural Networks (ONNs). ONNs can perform computations efficiently and can be used to build a more extensive neuromorphic system. Here, we address a fundamental problem: can we efficiently perform artificial intelligence applications with ONNs? We present a digital ONN implementation to show a proof-of-concept of the ONN approach of “computing-in-phase” for pattern recognition applications. To the best of our knowledge, this is the first attempt to implement an FPGA-based fully-digital ONN. We report ONN accuracy, training, inference, memory capacity, operating frequency, hardware resources based on simulations and implementations of 5 × 3 and 10 × 6 ONNs. We present the digital ONN implementation on FPGA for pattern recognition applications such as performing digits recognition from a camera stream. We discuss practical challenges and future directions in implementing digital ONN.