Generalized Key-Value Memory to Flexibly Adjust Redundancy in Memory-Augmented Networks

Memory-augmented neural networks enhance a neural network with an external key-value (KV) memory whose complexity is typically dominated by the number of support vectors in the key memory. We propose a generalized KV memory that decouples its dimension from the number of support vectors by introducing a free parameter that can arbitrarily add or remove redundancy to the key memory representation. In effect, it provides an additional degree of freedom to flexibly control the tradeoff between robustness and the resources required to store and compute the generalized KV memory. This is particularly useful for realizing the key memory on in-memory computing hardware where it exploits nonideal, but extremely efficient nonvolatile memory devices for dense storage and computation. Experimental results show that adapting this parameter on demand effectively mitigates up to 44% nonidealities, at equal accuracy and number of devices, without any need for neural network retraining.

Vector Symbolic Architectures as a Computing Framework for Emerging Hardware

This article reviews recent progress in the development of the computing framework Vector Symbolic Architectures (also known as Hyperdimensional Computing). This framework is well suited for implementation in stochastic, emerging hardware and it naturally expresses the types of cognitive operations required for Artificial Intelligence (AI). We demonstrate in this article that the field-like algebraic structure of Vector Symbolic Architectures offers simple but powerful operations on high-dimensional vectors that can support all data structures and manipulations relevant to modern computing. In addition, we illustrate the distinguishing feature of Vector Symbolic Architectures, "computing in superposition," which sets it apart from conventional computing. It also opens the door to efficient solutions to the difficult combinatorial search problems inherent in AI applications. We sketch ways of demonstrating that Vector Symbolic Architectures are computationally universal. We see them acting as a framework for computing with distributed representations that can play a role of an abstraction layer for emerging computing hardware. This article serves as a reference for computer architects by illustrating the philosophy behind Vector Symbolic Architectures, techniques of distributed computing with them, and their relevance to emerging computing hardware, such as neuromorphic computing.

A Highly Energy-Efficient Hyperdimensional Computing Processor for Biosignal Classification

Hyperdimensional computing (HDC) is a brain-inspired computing paradigm that operates on pseudo-random hypervectors to perform high-accuracy classifications for biomedical applications. The energy efficiency of prior HDC processors for this computationally minimal algorithm is dominated by costly hypervector memory storage, which grows linearly with the number of sensors. To address this, the memory is replaced with a light-weight cellular automaton for on-the-fly hypervector generation. The use of this technique is explored in conjunction with vector folding for various real-time classification latencies in post-layout simulation on an emotion recognition dataset with 200 channels. The proposed architecture achieves 39.1 nJ/prediction; a 4.9× energy efficiency improvement, 9.5× per channel, over the state-of-the-art HDC processor. At maximum throughput, the architecture achieves a 10.7× improvement, 33.5× per channel. An optimized support vector machine (SVM) processor is designed in this work for the same use-case. HDC is 9.5× more energy-efficient than the SVM, paving the way for it to become the paradigm of choice for high-accuracy, on-board biosignal classification.

Efficient emotion recognition using hyperdimensional computing with combinatorial channel encoding and cellular automata

In this paper, a hardware-optimized approach to emotion recognition based on the efficient brain-inspired hyperdimensional computing (HDC) paradigm is proposed. Emotion recognition provides valuable information for human–computer interactions; however, the large number of input channels (> 200) and modalities (> 3 ) involved in emotion recognition are significantly expensive from a memory perspective. To address this, methods for memory reduction and optimization are proposed, including a novel approach that takes advantage of the combinatorial nature of the encoding process, and an elementary cellular automaton. HDC with early sensor fusion is implemented alongside the proposed techniques achieving two-class multi-modal classification accuracies of > 76% for valence and > 73% for arousal on the multi-modal AMIGOS and DEAP data sets, almost always better than state of the art. The required vector storage is seamlessly reduced by 98% and the frequency of vector requests by at least 1/5. The results demonstrate the potential of efficient hyperdimensional computing for low-power, multi-channeled emotion recognition tasks.

Adaptive Body Area Networks Using Kinematics and Biosignals

The increasing penetration of wearable and implantable devices necessitates energy-efficient and robust ways of connecting them to each other and to the cloud. However, the wireless channel around the human body poses unique challenges such as a high and variable path-loss caused by frequent changes in the relative node positions as well as the surrounding environment. An adaptive wireless body area network (WBAN) scheme is presented that reconfigures the network by learning from body kinematics and biosignals. It has very low overhead since these signals are already captured by the WBAN sensor nodes to support their basic functionality. Periodic channel fluctuations in activities like walking can be exploited by reusing accelerometer data and scheduling packet transmissions at optimal times. Network states can be predicted based on changes in observed biosignals to reconfigure the network parameters in real time. A realistic body channel emulator that evaluates the path-loss for everyday human activities was developed to assess the efficacy of the proposed techniques. Simulation results show up to 41% improvement in packet delivery ratio (PDR) and up to 27% reduction in power consumption by intelligent scheduling at lower transmission power levels. Moreover, experimental results on a custom test-bed demonstrate an average PDR increase of 20% and 18% when using our adaptive EMG- and heart-rate-based transmission power control methods, respectively. The channel emulator and simulation code is made publicly available at https://github.com/a-moin/wban-pathloss.

Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping Characteristics

Hyperdimensional (HD) computing is a promising paradigm for future intelligent electronic appliances operating at low power. This paper discusses tradeoffs of selecting parameters of binary HD representations when applied to pattern recognition tasks. Particular design choices include density of representations and strategies for mapping data from the original representation. It is demonstrated that for the considered pattern recognition tasks (using synthetic and real-world data) both sparse and dense representations behave nearly identically. This paper also discusses implementation peculiarities which may favor one type of representations over the other. Finally, the capacity of representations of various densities is discussed.

A Comparative Study of On-Body Radio-Frequency Links in the 420 MHz⁻2.4 GHz Range

While there exists a wide variety of radio frequency (RF) technologies amenable for usage in Wireless Body Area Networks (WBANs), which have been studied separately before, it is currently still unclear how their performance compares in true on-body scenarios. In this paper, a single reference on-body scenario-that is, propagation along the arm-is used to experimentally compare six distinct RF technologies (between 420 MHz and 2.4 GHz) in terms of path loss. To further quantify on-body path loss, measurements for five different on-body scenarios are presented as well. To compensate for the effect of often large path losses, two mitigation strategies to (dynamically) improve on-body links are introduced and experimentally verified: beam steering using a phased array, and usage of on-body RF repeaters. The results of this study can serve as a tool for WBAN designers to aid in the selection of the right RF frequency and technology for their application.

Ultrasonic beamforming system for interrogating multiple implantable sensors

In this paper, we present an ultrasonic beamforming system capable of interrogating individual implantable sensors via backscatter in a distributed, ultrasound-based recording platform known as Neural Dust [1]. A custom ASIC drives a 7 × 2 PZT transducer array with 3 cycles of 32V square wave with a specific programmable time delay to focus the beam at the 800mm neural dust mote placed 50mm away. The measured acoustic-to-electrical conversion efficiency of the receive mote in water is 0.12% and the overall system delivers 26.3% of the power from the 1.8V power supply to the transducer drive output, consumes 0.75μJ in each transmit phase, and has a 0.5% change in the backscatter per volt applied to the input of the backscatter circuit. Further miniaturization of both the transmit array and the receive mote can pave the way for a wearable, chronic sensing and neuromodulation system.

Powering and communication for OMNI: A distributed and modular closed-loop neuromodulation device

A distributed, modular, intelligent, and efficient neuromodulation device, called OMNI, is presented. It supports closed-loop recording and stimulation on 256 channels from up to 4 physically distinct neuromodulation modules placed in any configuration around the brain, hence offering the capability of addressing neural disorders that are presented at the network level. The specific focus of this paper is the communication and power distribution network that enables the modular and distributed nature of the device.

Beamforming approaches for untethered, ultrasonic neural dust motes for cortical recording: a simulation study

In this paper, we examine the use of beamforming techniques to interrogate a multitude of neural implants in a distributed, ultrasound-based intra-cortical recording platform known as Neural Dust. We propose a general framework to analyze system design tradeoffs in the ultrasonic beamformer that extracts neural signals from modulated ultrasound waves that are backscattered by free-floating neural dust (ND) motes. Simulations indicate that high-resolution linearly-constrained minimum variance beamforming sufficiently suppresses interference from unselected ND motes and can be incorporated into the ND-based cortical recording system.

Equalization for intracortical microstimulation artifact reduction

We present a method for decreasing the duration of artifacts present during intra-cortical microstimulation (ICMS) recordings by using techniques developed for digital communications. We replace the traditional monophasic or biphasic current stimulation pulse with a patterned pulse stream produced by a Zero Forcing Equalizer (ZFE) filter after characterizing the artifact as a communications channel. The results find that using the ZFE stimulus has the potential to reduce artifact width by more than 70%. Considerations for the hardware implementation of the equalizer are presented.

Physical principles for scalable neural recording

Simultaneously measuring the activities of all neurons in a mammalian brain at millisecond resolution is a challenge beyond the limits of existing techniques in neuroscience. Entirely new approaches may be required, motivating an analysis of the fundamental physical constraints on the problem. We outline the physical principles governing brain activity mapping using optical, electrical, magnetic resonance, and molecular modalities of neural recording. Focusing on the mouse brain, we analyze the scalability of each method, concentrating on the limitations imposed by spatiotemporal resolution, energy dissipation, and volume displacement. Based on this analysis, all existing approaches require orders of magnitude improvement in key parameters. Electrical recording is limited by the low multiplexing capacity of electrodes and their lack of intrinsic spatial resolution, optical methods are constrained by the scattering of visible light in brain tissue, magnetic resonance is hindered by the diffusion and relaxation timescales of water protons, and the implementation of molecular recording is complicated by the stochastic kinetics of enzymes. Understanding the physical limits of brain activity mapping may provide insight into opportunities for novel solutions. For example, unconventional methods for delivering electrodes may enable unprecedented numbers of recording sites, embedded optical devices could allow optical detectors to be placed within a few scattering lengths of the measured neurons, and new classes of molecularly engineered sensors might obviate cumbersome hardware architectures. We also study the physics of powering and communicating with microscale devices embedded in brain tissue and find that, while radio-frequency electromagnetic data transmission suffers from a severe power-bandwidth tradeoff, communication via infrared light or ultrasound may allow high data rates due to the possibility of spatial multiplexing. The use of embedded local recording and wireless data transmission would only be viable, however, given major improvements to the power efficiency of microelectronic devices.

Wireless channel characterization for mm-size neural implants

This paper discusses an approach to modeling and characterizing wireless channel properties for mm-size neural implants. Full-wave electromagnetic simulation was employed to model signal propagation characteristics in biological materials. Animal tests were carried out, proving the validity of the simulation model over a wide range of frequency from 100MHz to 6GHz. Finally, effects of variability and uncertainty in human anatomy and dielectric properties of tissues on these radio links are explored.

A study of energy consumption and reliability in a multi-hop sensor network

For a moderate-size, multi-hop, sensor network, we present experimental measurements of radio energy consumption and packet reliability. We categorize the energy measurements by energy consumed in each radio state and for each traffic type. Packet reliability results are presented from a network and link perspective, whereas prior work only considered the former. We introduce a novel technique of application-aware radio duty cycling called on-demand spatial TDMA. When compared to the non-cycling case, this technique can achieve greater than an order of magnitude reduction in idle energy consumption, while not sacrificing reliability. We show end-to-end packet loss rates as low as 0.04 when averaged over the network. Even with substantial idle energy savings, we identify radio idling as the dominate energy consumer and overhearing as the dominate traffic type.