Insect-inspired neuromorphic computing

Insect-Inspired, Spike-Based, in-Sensor, and Night-Time Collision Detector Based on Atomically Thin and Light-Sensitive Memtransistors

Detecting a potential collision at night is a challenging task owing to the lack of discernible features that can be extracted from the available visual stimuli. To alert the driver or, alternatively, the maneuvering system of an autonomous vehicle, current technologies utilize resource draining and expensive solutions such as light detection and ranging (LiDAR) or image sensors coupled with extensive software running sophisticated algorithms. In contrast, insects perform the same task of collision detection with frugal neural resources. Even though the general architecture of separate sensing and processing modules is the same in insects and in image-sensor-based collision detectors, task-specific obstacle avoidance algorithms allow insects to reap substantial benefits in terms of size and energy. Here, we show that insect-inspired collision detection algorithms, when implemented in conjunction with in-sensor processing and enabled by innovative optoelectronic integrated circuits based on atomically thin and photosensitive memtransistor technology, can greatly simplify collision detection at night. The proposed collision detector eliminates the need for image capture and image processing yet demonstrates timely escape responses for cars on collision courses under various real-life scenarios at night. The collision detector also has a small footprint of ∼40 μm2 and consumes only a few hundred picojoules of energy. We strongly believe that the proposed collision detectors can augment existing sensors necessary for ensuring autonomous vehicular safety.

Neuromorphic object localization using resistive memories and ultrasonic transducers

Real-world sensory-processing applications require compact, low-latency, and low-power computing systems. Enabled by their in-memory event-driven computing abilities, hybrid memristive-Complementary Metal-Oxide Semiconductor neuromorphic architectures provide an ideal hardware substrate for such tasks. To demonstrate the full potential of such systems, we propose and experimentally demonstrate an end-to-end sensory processing solution for a real-world object localization application. Drawing inspiration from the barn owl’s neuroanatomy, we developed a bio-inspired, event-driven object localization system that couples state-of-the-art piezoelectric micromachined ultrasound transducer sensors to a neuromorphic resistive memories-based computational map. We present measurement results from the fabricated system comprising resistive memories-based coincidence detectors, delay line circuits, and a full-custom ultrasound sensor. We use these experimental results to calibrate our system-level simulations. These simulations are then used to estimate the angular resolution and energy efficiency of the object localization model. The results reveal the potential of our approach, evaluated in orders of magnitude greater energy efficiency than a microcontroller performing the same task.

The real-world object localization application needs a low-latency and power efficient computing system. Here, Moro et al. demonstrate a neuromorphic in-memory event driven system, inspired by the barn owl’s neuroanatomy, which is orders of magnitude more energy efficient than microcontrollers.

Simulation for a Mems-Based CTRNN Ultra-Low Power Implementation of Human Activity Recognition

This paper presents an energy-efficient classification framework that performs human activity recognition (HAR). Typically, HAR classification tasks require a computational platform that includes a processor and memory along with sensors and their interfaces, all of which consume significant power. The presented framework employs microelectromechanical systems (MEMS) based Continuous Time Recurrent Neural Network (CTRNN) to perform HAR tasks very efficiently. In a real physical implementation, we show that the MEMS-CTRNN nodes can perform computing while consuming power on a nano-watts scale compared to the micro-watts state-of-the-art hardware. We also confirm that this huge power reduction doesn't come at the expense of reduced performance by evaluating its accuracy to classify the highly cited human activity recognition dataset (HAPT). Our simulation results show that the HAR framework that consists of a training module, and a network of MEMS-based CTRNN nodes, provides HAR classification accuracy for the HAPT that is comparable to traditional CTRNN and other Recurrent Neural Network (RNN) implantations. For example, we show that the MEMS-based CTRNN model average accuracy for the worst-case scenario of not using pre-processing techniques, such as quantization, to classify 5 different activities is 77.94% compared to 78.48% using the traditional CTRNN.

Exploiting Pull-In/Pull-Out Hysteresis in Electrostatic MEMS Sensor Networks to Realize a Novel Sensing Continuous-Time Recurrent Neural Network

The goal of this paper is to provide a novel computing approach that can be used to reduce the power consumption, size, and cost of wearable electronics. To achieve this goal, the use of microelectromechanical systems (MEMS) sensors for simultaneous sensing and computing is introduced. Specifically, by enabling sensing and computing locally at the MEMS sensor node and utilizing the usually unwanted pull in/out hysteresis, we may eliminate the need for cloud computing and reduce the use of analog-to-digital converters, sampling circuits, and digital processors. As a proof of concept, we show that a simulation model of a network of three commercially available MEMS accelerometers can classify a train of square and triangular acceleration signals inherently using pull-in and release hysteresis. Furthermore, we develop and fabricate a network with finger arrays of parallel plate actuators to facilitate coupling between MEMS devices in the network using actuating assemblies and biasing assemblies, thus bypassing the previously reported coupling challenge in MEMS neural networks.

Drosophila reward system - A summary of current knowledge

The fruit fly Drosophila melanogaster brain is the most extensively investigated model of a reward system in insects. Drosophila can discriminate between rewarding and punishing environmental stimuli and consequently undergo associative learning. Functional models, especially those modelling mushroom bodies, are constantly being developed using newly discovered information, adding to the complexity of creating a simple model of the reward system. This review aims to clarify whether its reward system also includes a hedonic component. Neurochemical systems that mediate the 'wanting' component of reward in the Drosophila brain are well documented, however, the systems that mediate the pleasure component of reward in mammals, including those involving the endogenous opioid and endocannabinoid systems, are unlikely to be present in insects. The mushroom body components exhibit differential developmental age and different functional processes. We propose a hypothetical hierarchy of the levels of reinforcement processing in response to particular stimuli, and the parallel processes that take place concurrently. The possible presence of activity-silencing and meta-satiety inducing levels in Drosophila should be further investigated.

Ear-Bot: Locust Ear-on-a-Chip Bio-Hybrid Platform

During hundreds of millions of years of evolution, insects have evolved some of the most efficient and robust sensing organs, often far more sensitive than their man-made equivalents. In this study, we demonstrate a hybrid bio-technological approach, integrating a locust tympanic ear with a robotic platform. Using an Ear-on-a-Chip method, we manage to create a long-lasting miniature sensory device that operates as part of a bio-hybrid robot. The neural signals recorded from the ear in response to sound pulses, are processed and used to control the robot’s motion. This work is a proof of concept, demonstrating the use of biological ears for robotic sensing and control.

Artificial fly visual joint perception neural network inspired by multiple-regional collision detection

The biological visual system includes multiple types of motion sensitive neurons which preferentially respond to specific perceptual regions. However, it still keeps open how to borrow such neurons to construct bio-inspired computational models for multiple-regional collision detection. To fill this gap, this work proposes a visual joint perception neural network with two subnetworks - presynaptic and postsynaptic neural networks, inspired by the preferentialperception characteristics of three horizontal and vertical motion sensitive neurons. Related to the neural network and three hazard detection mechanisms, an artificial fly visual synthesized collision detection model for multiple-regional collision detection is originally developed to monitor possible danger occurrence in the case where one or more moving objects appear in the whole field of view. The experiments can clearly draw two conclusions: (i) the acquired neural network can effectively display the characteristics of visual movement, and (ii) the collision detection model, which outperforms the compared models, can effectively perform multiple-regional collision detection at a high success rate, and only takes about 0.24s to complete the process of collision detection for each virtual or actual image frame with resolution 110×60.

Using virtual worlds to understand insect navigation for bio-inspired systems

Insects perform a wide array of intricate behaviors over large spatial and temporal scales in complex natural environments. A mechanistic understanding of insect cognition has direct implications on how brains integrate multimodal information and can inspire bio-based solutions for autonomous robots. Virtual Reality (VR) offers an opportunity assess insect neuroethology while presenting complex, yet controlled, stimuli. Here, we discuss the use of insects as inspiration for artificial systems, recent advances in different VR technologies, current knowledge gaps, and the potential for application of insect VR research to bio-inspired robots. Finally, we advocate the need to diversify our model organisms, behavioral paradigms, and embrace the complexity of the natural world. This will help us to uncover the proximate and ultimate basis of brain and behavior and extract general principles for common challenging problems.

An On-chip Spiking Neural Network for Estimation of the Head Pose of the iCub Robot

In this work, we present a neuromorphic architecture for head pose estimation and scene representation for the humanoid iCub robot. The spiking neuronal network is fully realized in Intel's neuromorphic research chip, Loihi, and precisely integrates the issued motor commands to estimate the iCub's head pose in a neuronal path-integration process. The neuromorphic vision system of the iCub is used to correct for drift in the pose estimation. Positions of objects in front of the robot are memorized using on-chip synaptic plasticity. We present real-time robotic experiments using 2 degrees of freedom (DoF) of the robot's head and show precise path integration, visual reset, and object position learning on-chip. We discuss the requirements for integrating the robotic system and neuromorphic hardware with current technologies.

Synaptic Delays for Insect-Inspired Temporal Feature Detection in Dynamic Neuromorphic Processors

Spiking neural networks are well-suited for spatiotemporal feature detection and learning, and naturally involve dynamic delay mechanisms in the synapses, dendrites, and axons. Dedicated delay neurons and axonal delay circuits have been considered when implementing such pattern recognition networks in dynamic neuromorphic processors. Inspired by an auditory feature detection circuit in crickets, featuring a delayed excitation by post-inhibitory rebound, we investigate disynaptic delay elements formed by inhibitory-excitatory pairs of dynamic synapses. We configured such disynaptic delay elements in the DYNAP-SE neuromorphic processor and characterized the distribution of delayed excitations resulting from device mismatch. Interestingly, we found that the disynaptic delay elements can be configured such that the timing and magnitude of the delayed excitation depend mainly on the efficacy of the inhibitory and excitatory synapses, respectively, and that a neuron with multiple delay elements can be tuned to respond selectively to a specific pattern. Furthermore, we present a network with one disynaptic delay element that mimics the auditory feature detection circuit of crickets, and we demonstrate how varying synaptic weights, input noise and processor temperature affect the circuit. Dynamic delay elements of this kind open up for synapse level temporal feature tuning with configurable delays of up to 100 ms.