Putting a bug in ML: The moth olfactory network learns to read MNIST

Brain inspired neuronal silencing mechanism to enable reliable sequence identification

Real-time sequence identification is a core use-case of artificial neural networks (ANNs), ranging from recognizing temporal events to identifying verification codes. Existing methods apply recurrent neural networks, which suffer from training difficulties; however, performing this function without feedback loops remains a challenge. Here, we present an experimental neuronal long-term plasticity mechanism for high-precision feedforward sequence identification networks (ID-nets) without feedback loops, wherein input objects have a given order and timing. This mechanism temporarily silences neurons following their recent spiking activity. Therefore, transitory objects act on different dynamically created feedforward sub-networks. ID-nets are demonstrated to reliably identify 10 handwritten digit sequences, and are generalized to deep convolutional ANNs with continuous activation nodes trained on image sequences. Counterintuitively, their classification performance, even with a limited number of training examples, is high for sequences but low for individual objects. ID-nets are also implemented for writer-dependent recognition, and suggested as a cryptographic tool for encrypted authentication. The presented mechanism opens new horizons for advanced ANN algorithms.

Neural architectures in the light of comparative connectomics

Since the Cambrian, animals diversified from a few body forms or bauplans, into many extinct and all extant species. A characteristic neural architecture serves each bauplan. How the connectome of each animal differs from that of closely related species or whether it converged into an optimal architecture shared with more distant ones is unknown. Recent technological innovations in molecular biology, microscopy, digital data storage and processing, and computational neuroscience have lowered the barriers for whole-brain connectomics. Comparative connectomics of suitable, relatively small, representative species across the phylogenetic tree can infer the archetypal neural architecture of each bauplan and identify any circuits that possibly converged onto a shared and potentially optimal, structure.

Performing arithmetic using a neural network trained on images of digit permutation pairs

In this paper, a neural network is trained to perform simple arithmetic using images of concatenated handwritten digit pairs. A convolutional neural network was trained with images consisting of two side-by-side handwritten digits, where the image’s label is the summation of the two digits contained in the combined image. Crucially, the network was tested on permutation pairs that were not present during training in an effort to see if the network could learn the task of addition, as opposed to simply mapping images to labels. A dataset was generated for all possible permutation pairs of length 2 for the digits 0–9 using MNIST as a basis for the images, with one thousand samples generated for each permutation pair. For testing the network, samples generated from previously unseen permutation pairs were fed into the trained network, and its predictions measured. Results were encouraging, with the network achieving an accuracy of over 90% on some permutation train/test splits. This suggests that the network learned at first digit recognition, and subsequently the further task of addition based on the two recognised digits. As far as the authors are aware, no previous work has concentrated on learning a mathematical operation in this way. This paper is an attempt to demonstrate that a network can learn more than a direct mapping from image to label, but is learning to analyse two separate regions of an image and combining what was recognised to produce the final output label.

Power-law scaling to assist with key challenges in artificial intelligence

Power-law scaling, a central concept in critical phenomena, is found to be useful in deep learning, where optimized test errors on handwritten digit examples converge as a power-law to zero with database size. For rapid decision making with one training epoch, each example is presented only once to the trained network, the power-law exponent increased with the number of hidden layers. For the largest dataset, the obtained test error was estimated to be in the proximity of state-of-the-art algorithms for large epoch numbers. Power-law scaling assists with key challenges found in current artificial intelligence applications and facilitates an a priori dataset size estimation to achieve a desired test accuracy. It establishes a benchmark for measuring training complexity and a quantitative hierarchy of machine learning tasks and algorithms.

A spiking neural program for sensorimotor control during foraging in flying insects

Foraging is a vital behavioral task for living organisms. Behavioral strategies and abstract mathematical models thereof have been described in detail for various species. To explore the link between underlying neural circuits and computational principles, we present how a biologically detailed neural circuit model of the insect mushroom body implements sensory processing, learning, and motor control. We focus on cast and surge strategies employed by flying insects when foraging within turbulent odor plumes. Using a spike-based plasticity rule, the model rapidly learns to associate individual olfactory sensory cues paired with food in a classical conditioning paradigm. We show that, without retraining, the system dynamically recalls memories to detect relevant cues in complex sensory scenes. Accumulation of this sensory evidence on short time scales generates cast-and-surge motor commands. Our generic systems approach predicts that population sparseness facilitates learning, while temporal sparseness is required for dynamic memory recall and precise behavioral control. Our work successfully combines biological computational principles with spike-based machine learning. It shows how knowledge transfer from static to arbitrary complex dynamic conditions can be achieved by foraging insects and may serve as inspiration for agent-based machine learning.

Surface-Enhanced Raman Scattering-Based Odor Compass: Locating Multiple Chemical Sources and Pathogens

Olfaction is important for identifying and avoiding toxic substances in living systems. Many efforts have been made to realize artificial olfaction systems that reflect the capacity of biological systems. A sophisticated example of an artificial olfaction device is the odor compass which uses chemical sensor data to identify odor source direction. Successful odor compass designs often rely on plume-based detection and mobile robots, where active, mechanical motion of the sensor platform is employed. Passive, diffusion-based odor compasses remain elusive as detection of low analyte concentrations and quantification of small concentration gradients from within the sensor platform are necessary. Further, simultaneously identifying multiple odor sources using an odor compass remains an ongoing challenge, especially for similar analytes. Here, we show that surface-enhanced Raman scattering (SERS) sensors overcome these challenges, and we present the first SERS odor compass. Using a grid array of SERS sensors, machine learning analysis enables reliable identification of multiple odor sources arising from diffusion of analytes from one or two localized sources. Specifically, convolutional neural network and support vector machine classifier models achieve over 90% accuracy for a multiple odor source problem. This system is then used to identify the location of an Escherichia coli biofilm via its complex signature of volatile organic compounds. Thus, the fabricated SERS chemical sensors have the needed limit of detection and quantification for diffusion-based odor compasses. Solving the multiple odor source problem with a passive platform opens a path toward an Internet of things approach to monitor toxic gases and indoor pathogens.