Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy

High-speed wide-field fluorescence microscopy has the potential to capture biological processes with exceptional spatiotemporal resolution. However, conventional cameras suffer from low signal-to-noise ratio at high frame rates, limiting their ability to detect faint fluorescent events. Here, we introduce an image sensor where each pixel has individually programmable sampling speed and phase, so that pixels can be arranged to simultaneously sample at high speed with a high signal-to-noise ratio. In high-speed voltage imaging experiments, our image sensor significantly increases the output signal-to-noise ratio compared to a low-noise scientific CMOS camera (~2-3 folds). This signal-to-noise ratio gain enables the detection of weak neuronal action potentials and subthreshold activities missed by the standard scientific CMOS cameras. Our camera with flexible pixel exposure configurations offers versatile sampling strategies to improve signal quality in various experimental conditions.

The influence of stereopsis on visual saliency in a proto-object based model of selective attention

Some animals including humans use stereoscopic vision which reconstructs spatial information about the environment from the disparity between images captured by eyes in two separate adjacent locations. Like other sensory information, such stereoscopic information is expected to influence attentional selection. We develop a biologically plausible model of binocular vision to study its effect on bottom-up visual attention, i.e., visual saliency. In our model, the scene is organized in terms of proto-objects on which attention acts, rather than on unbound sets of elementary features. We show that taking into account the stereoscopic information improves the performance of the model in the prediction of human eye movements with statistically significant differences.

Neuromorphic applications in medicine

In recent years, there has been a growing demand for miniaturization, low power consumption, quick treatments, and non-invasive clinical strategies in the healthcare industry. To meet these demands, healthcare professionals are seeking new technological paradigms that can improve diagnostic accuracy while ensuring patient compliance. Neuromorphic engineering, which uses neural models in hardware and software to replicate brain-like behaviors, can help usher in a new era of medicine by delivering low power, low latency, small footprint, and high bandwidth solutions. This paper provides an overview of recent neuromorphic advancements in medicine, including medical imaging and cancer diagnosis, processing of biosignals for diagnosis, and biomedical interfaces, such as motor, cognitive, and perception prostheses. For each section, we provide examples of how brain-inspired models can successfully compete with conventional artificial intelligence algorithms, demonstrating the potential of neuromorphic engineering to meet demands and improve patient outcomes. Lastly, we discuss current struggles in fitting neuromorphic hardware with non-neuromorphic technologies and propose potential solutions for future bottlenecks in hardware compatibility.

First Organoid Intelligence (OI) workshop to form an OI community

The brain is arguably the most powerful computation system known. It is extremely efficient in processing large amounts of information and can discern signals from noise, adapt, and filter faulty information all while running on only 20 watts of power. The human brain's processing efficiency, progressive learning, and plasticity are unmatched by any computer system. Recent advances in stem cell technology have elevated the field of cell culture to higher levels of complexity, such as the development of three-dimensional (3D) brain organoids that recapitulate human brain functionality better than traditional monolayer cell systems. Organoid Intelligence (OI) aims to harness the innate biological capabilities of brain organoids for biocomputing and synthetic intelligence by interfacing them with computer technology. With the latest strides in stem cell technology, bioengineering, and machine learning, we can explore the ability of brain organoids to compute, and store given information (input), execute a task (output), and study how this affects the structural and functional connections in the organoids themselves. Furthermore, understanding how learning generates and changes patterns of connectivity in organoids can shed light on the early stages of cognition in the human brain. Investigating and understanding these concepts is an enormous, multidisciplinary endeavor that necessitates the engagement of both the scientific community and the public. Thus, on Feb 22-24 of 2022, the Johns Hopkins University held the first Organoid Intelligence Workshop to form an OI Community and to lay out the groundwork for the establishment of OI as a new scientific discipline. The potential of OI to revolutionize computing, neurological research, and drug development was discussed, along with a vision and roadmap for its development over the coming decade.

Analysis and Design Methodology of RF Energy Harvesting Rectifier Circuit for Ultra-Low Power Applications

This paper reviews and analyses the design of popular radio frequency energy harvesting systems and proposes a method to qualitatively and quantitatively analyze their circuit architectures using new square-wave approximation method. This approach helps in simplifying design analysis. Using this analysis, we can establish no load output voltage characteristics, upper limit on rectifier efficiency, and maximum power characteristics of a rectifier. This paper will help guide the design of RF energy harvesting rectifier circuits for radio frequency identification (RFIDs), the Internet of Things (IoTs), wearable, and implantable medical device applications. Different application scenarios are explained in the context of design challenges, and corresponding design considerations are discussed in order to evaluate their performance. The pros and cons of different rectifier topologies are also investigated. In addition to presenting the popular rectifier topologies, new measurement results of these energy harvester topologies, fabricated in 65nm, 130nm and 180nm CMOS technologies are also presented.

Decoding accelerometry for classification and prediction of critically ill patients with severe brain injury

Our goal is to explore quantitative motor features in critically ill patients with severe brain injury (SBI). We hypothesized that computational decoding of these features would yield information on underlying neurological states and outcomes. Using wearable microsensors placed on all extremities, we recorded a median 24.1 (IQR: 22.8-25.1) hours of high-frequency accelerometry data per patient from a prospective cohort (n = 69) admitted to the ICU with SBI. Models were trained using time-, frequency-, and wavelet-domain features and levels of responsiveness and outcome as labels. The two primary tasks were detection of levels of responsiveness, assessed by motor sub-score of the Glasgow Coma Scale (GCSm), and prediction of functional outcome at discharge, measured with the Glasgow Outcome Scale-Extended (GOSE). Detection models achieved significant (AUC: 0.70 [95% CI: 0.53-0.85]) and consistent (observation windows: 12 min-9 h) discrimination of SBI patients capable of purposeful movement (GCSm > 4). Prediction models accurately discriminated patients of upper moderate disability or better (GOSE > 5) with 2-6 h of observation (AUC: 0.82 [95% CI: 0.75-0.90]). Results suggest that time series analysis of motor activity yields clinically relevant insights on underlying functional states and short-term outcomes in patients with SBI.

A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model With a Hybrid FPGA Implementation

Computing and attending to salient regions of a visual scene is an innate and necessary preprocessing step for both biological and engineered systems performing high-level visual tasks including object detection, tracking, and classification. Computational bandwidth and speed are improved by preferentially devoting computational resources to salient regions of the visual field. The human brain computes saliency effortlessly, but modeling this task in engineered systems is challenging. We first present a neuromorphic dynamic saliency model, which is bottom-up, feed-forward, and based on the notion of proto-objects with neurophysiological spatio-temporal features requiring no training. Our neuromorphic model outperforms state-of-the-art dynamic visual saliency models in predicting human eye fixations (i.e., ground truth saliency). Secondly, we present a hybrid FPGA implementation of the model for real-time applications, capable of processing 112×84 resolution frames at 18.71 Hz running at a 100 MHz clock rate - a 23.77× speedup from the software implementation. Additionally, our fixed-point model of the FPGA implementation yields comparable results to the software implementation.

A syndromic surveillance tool to detect anomalous clusters of COVID-19 symptoms in the United States

Coronavirus SARS-COV-2 infections continue to spread across the world, yet effective large-scale disease detection and prediction remain limited. COVID Control: A Johns Hopkins University Study, is a novel syndromic surveillance approach, which collects body temperature and COVID-like illness (CLI) symptoms across the US using a smartphone app and applies spatio-temporal clustering techniques and cross-correlation analysis to create maps of abnormal symptomatology incidence that are made publicly available. The results of the cross-correlation analysis identify optimal temporal lags between symptoms and a range of COVID-19 outcomes, with new taste/smell loss showing the highest correlations. We also identified temporal clusters of change in taste/smell entries and confirmed COVID-19 incidence in Baltimore City and County. Further, we utilized an extended simulated dataset to showcase our analytics in Maryland. The resulting clusters can serve as indicators of emerging COVID-19 outbreaks, and support syndromic surveillance as an early warning system for disease prevention and control.

A syndromic surveillance tool to detect anomalous clusters of COVID-19 symptoms in the United States

Coronavirus SARS-COV-2 infections continue to spread across the world, yet effective large-scale disease detection and prediction remain limited. COVID Control: A Johns Hopkins University Study, is a novel syndromic surveillance approach, which collects body temperature and COVID-like illness (CLI) symptoms across the US using a smartphone app and applies spatio-temporal clustering techniques and cross-correlation analysis to create maps of abnormal symptomatology incidence that are made publicly available. The results of the cross-correlation analysis identify optimal temporal lags between symptoms and a range of COVID-19 outcomes, with new taste/smell loss showing the highest correlations. We also identified temporal clusters of change in taste/smell entries and confirmed COVID-19 incidence in Baltimore City and County. Further, we utilized an extended simulated dataset to showcase our analytics in Maryland. The resulting clusters can serve as indicators of emerging COVID-19 outbreaks, and support syndromic surveillance as an early warning system for disease prevention and control.

A Miniature Wireless Silicon-on-Insulator Image Sensor for Brain Fluorescence Imaging

Optical recording of genetically encoded calcium indicator (GECI) allows neuroscientists to study the activity of genetically labeled neuron populations, but our current tools lack the resolution, stability and are often too invasive. Here we present the design concepts, prototypes, and preliminary measurement results of a super-miniaturized wireless image sensor built using a 32nm Silicon-on-Insulator process. SOI process is optimal for wireless applications, and we can further thin the substrate to reduce overall device thickness to ~25μm and operate the pixels using back-side illumination. The proposed device is 300μm × 300μm. Our prototype is built on a 3 × 3mm die.

Proto-Object Based Saliency Model With Texture Detection Channel

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

A Compact Free-Floating Device for Passive Charge-Balanced Neural Stimulation using PEDOT/CNT microelectrodes

Wirelessly powered implants are increasingly being developed as free-floating single-channel devices to interface with neurons directly at stimulation sites. In order to stimulate neurons in a manner that is safe to both the electrode and the surrounding tissue, charge accumulation over time needs to be avoided. The implementation of conventional charge balancing methods often leads to an increase in system complexity, power consumption or area, all of which are critical parameters in ultra-small wireless devices. The proposed charge balancing method described in this work, which relies on bipolar capacitive integrated electrodes, does not increase these parameters. The standalone wirelessly powered stimulating implant is implemented in a 130nm CMOS technology and measures 0.009 mm³.

A closed-loop, all-electronic pixel-wise adaptive imaging system for high dynamic range videography

Digital cameras expose and readout all pixels in accordance with a global sample clock. This rigid global control of exposure and sampling is problematic for capturing scenes with large variance in brightness and motion, and may cause regions of motion blur, under- and overexposure. To address these issues, we developed a CMOS imaging system that automatically adjusts each pixel's exposure and sampling rate to fit local motion and brightness. This system consists of an image sensor with pixel-addressable exposure configurability in combination with a real-time, per-pixel exposure controller. It operates in a closed-loop to sample, detect and optimize each pixel's exposure and sampling rate for optimal acquisition. Per-pixel exposure control is implemented using all-integrated electronics without external optical modulation. This reduces system complexity and power consumption compared to existing solutions. Implemented using standard 130nm CMOS process, the chip has 256 × 256 pixels and consumes 7.31mW. To evaluate performance, we used this system to capture scenes with complex lighting and motion conditions that would lead to loss of information for globally-exposed cameras. These results demonstrate the advantage of pixel-wise adaptive imaging for a range of computer vision tasks such as segmentation, motion estimation and object recognition.

Closed-loop bioelectronic medicine for diabetes management

Modulation of the nervous system by delivering electrical or pharmaceutical agents has contributed to the development of novel treatments to serious health disorders. Recent advances in multidisciplinary research has enabled the emergence of a new powerful therapeutic approach called bioelectronic medicine. Bioelectronic medicine exploits the fact that every organ in our bodies is neurally innervated and thus electrical interfacing with peripheral nerves can be a potential pathway for diagnosing or treating diseases such as diabetes. In this context, a plethora of studies have confirmed the important role of the nervous system in maintaining a tight regulation of glucose homeostasis. This has initiated new research exploring the opportunities of bioelectronic medicine for improving glucose control in people with diabetes, including regulation of gastric emptying, insulin sensitivity, and secretion of pancreatic hormones. Moreover, the development of novel closed-loop strategies aims to provide effective, specific and safe interfacing with the nervous system, and thereby targeting the organ of interest. This is especially valuable in the context of chronic diseases such as diabetes, where closed-loop bioelectronic medicine promises to provide real-time, autonomous and patient-specific therapies. In this article, we present an overview of the state-of-the-art for closed-loop neuromodulation systems in relation to diabetes and discuss future related opportunities for management of this chronic disease.

The Microbead: A 0.009 mm3 Implantable Wireless Neural Stimulator

Wirelessly powered implants are increasingly being developed to interface with neurons in the brain. They often rely on microelectrode arrays, which are limited by their ability to cover large cortical surface areas and long-term stability because of their physical size and rigid configuration. Yet some clinical and research applications prioritize a distributed neural interface over one that offers high channel count. One solution to make large scale, fully specifiable, electrical stimulation/recording possible, is to disconnect the electrodes from the base, so that they can be arbitrarily placed freely in the nervous system. In this work, a wirelessly powered stimulating implant is miniaturized using a novel electrode integration technique, and its implanted depth maximized using new optimization design methods for the transmitter and receiver coils. The stimulating device is implemented in a 130 nm CMOS technology with the following characteristics: 300 μm × 300 μm × 80 μm size; optimized two-coil inductive link; and integrated circuit, electrodes and coil. The wireless and stimulation capability of the implant is demonstrated in a conductive medium, as well as in-vivo. To the best of our knowledge, the fabricated free-floating miniaturized implant has the best depth-to-volume ratio making it an excellent tool for minimally-invasive distributed neural interface, and thus could eventually complement or replace the rigid arrays that are currently the state-of-the-art in brain set-ups.

Deep Learning-Based Target Tracking and Classification for Low Quality Videos Using Coded Aperture Cameras

Compressive sensing has seen many applications in recent years. One type of compressive sensing device is the Pixel-wise Code Exposure (PCE) camera, which has low power consumption and individual control of pixel exposure time. In order to use PCE cameras for practical applications, a time consuming and lossy process is needed to reconstruct the original frames. In this paper, we present a deep learning approach that directly performs target tracking and classification in the compressive measurement domain without any frame reconstruction. In particular, we propose to apply You Only Look Once (YOLO) to detect and track targets in the frames and we propose to apply Residual Network (ResNet) for classification. Extensive simulations using low quality optical and mid-wave infrared (MWIR) videos in the SENSIAC database demonstrated the efficacy of our proposed approach.

An Unsupervised Compressed Sensing Algorithm for Multi-Channel Neural Recording and Spike Sorting

We propose an unsupervised compressed sensing (CS)-based framework to compress, recover, and cluster neural action potentials. This framework can be easily integrated into high-density multi-electrode neural recording VLSI systems. Embedding spectral clustering and group structures in dictionary learning, we extend the proposed framework to unsupervised spike sorting without prior label information. Additionally, we incorporate group sparsity concepts in the dictionary learning to enable the framework for multi-channel neural recordings, as in tetrodes. To further improve spike sorting success rates in the CS framework, we embed template matching in sparse coding to jointly predict clusters of spikes. Our experimental results demonstrate that the proposed CS-based framework can achieve a high compression ratio (8:1 to 20:1), with a high quality reconstruction performance (>8 dB) and a high spike sorting accuracy (>90%).

Large-Scale Neuromorphic Spiking Array Processors: A Quest to Mimic the Brain

Neuromorphic engineering (NE) encompasses a diverse range of approaches to information processing that are inspired by neurobiological systems, and this feature distinguishes neuromorphic systems from conventional computing systems. The brain has evolved over billions of years to solve difficult engineering problems by using efficient, parallel, low-power computation. The goal of NE is to design systems capable of brain-like computation. Numerous large-scale neuromorphic projects have emerged recently. This interdisciplinary field was listed among the top 10 technology breakthroughs of 2014 by the MIT Technology Review and among the top 10 emerging technologies of 2015 by the World Economic Forum. NE has two-way goals: one, a scientific goal to understand the computational properties of biological neural systems by using models implemented in integrated circuits (ICs); second, an engineering goal to exploit the known properties of biological systems to design and implement efficient devices for engineering applications. Building hardware neural emulators can be extremely useful for simulating large-scale neural models to explain how intelligent behavior arises in the brain. The principal advantages of neuromorphic emulators are that they are highly energy efficient, parallel and distributed, and require a small silicon area. Thus, compared to conventional CPUs, these neuromorphic emulators are beneficial in many engineering applications such as for the porting of deep learning algorithms for various recognitions tasks. In this review article, we describe some of the most significant neuromorphic spiking emulators, compare the different architectures and approaches used by them, illustrate their advantages and drawbacks, and highlight the capabilities that each can deliver to neural modelers. This article focuses on the discussion of large-scale emulators and is a continuation of a previous review of various neural and synapse circuits (Indiveri et al., 2011). We also explore applications where these emulators have been used and discuss some of their promising future applications.

The Microbead: A Highly Miniaturized Wirelessly Powered Implantable Neural Stimulating System

An implant that can electrically stimulate neurons across different depths and regions of the brain currently does not exist as it poses a number of obstacles that need to be solved. In order to address the challenges, this paper presents the concept of "microbead," a fully integrated wirelessly powered neural device that allows for spatially selective activation of neural tissue. The prototype chip is fabricated in 130-nm CMOS technology and currently measures 200 μm × 200 μm, which represents the smallest remotely powered stimulator to date. The system is validated experimentally in a rat by stimulating the sciatic nerve with 195-μs current pulses. To power the ultrasmall on-silicon coil, 36-dBm source power is provided to a highly optimized transmitter (Tx) coil at a coupling distance of 5 mm. In order to satisfy the strict power limit for safe use in human subjects, a pulsed powering scheme is implemented that enables a significant decrease in the average power emitted from the Tx.

Active phantoms: a paradigm for ultrasound calibration using phantom feedback

In ultrasound (US)-guided medical procedures, accurate tracking of interventional tools is crucial to patient safety and clinical outcome. This requires a calibration procedure to recover the relationship between the US image and the tracking coordinate system. In literature, calibration has been performed on passive phantoms, which depend on image quality and parameters, such as frequency, depth, and beam-thickness as well as in-plane assumptions. In this work, we introduce an active phantom for US calibration. This phantom actively detects and responds to the US beams transmitted from the imaging probe. This active echo (AE) approach allows identification of the US image midplane independent of image quality. Both target localization and segmentation can be done automatically, minimizing user dependency. The AE phantom is compared with a crosswire phantom in a robotic US setup. An out-of-plane estimation US calibration method is also demonstrated through simulation and experiments to compensate for remaining elevational uncertainty. The results indicate that the AE calibration phantom can have more consistent results across experiments with varying image configurations. Automatic segmentation is also shown to have similar performance to manual segmentation.

Novel integration and packaging concepts of highly miniaturized inductively powered neural implants

This work proposes solutions to the current bulky packaged neural implants. We describe the next generation of miniaturized wirelessly powered neural interface that are distributed and free floating in the nervous system. This paper focuses on the microassembly, hermetic packaging and its effect on the inductive power link.

A CMOS Current Steering Neurostimulation Array With Integrated DAC Calibration and Charge Balancing

An 8-channel current steerable, multi-phasic neural stimulator with on-chip current DAC calibration and residue nulling for precise charge balancing is presented. Each channel consists of two sub-binary radix DACs followed by wide-swing, high output impedance current buffers providing time-multiplexed source and sink outputs for anodic and cathodic stimulation. A single integrator is shared among channels and serves to calibrate DAC coefficients and to closely match the anodic and cathodic stimulation phases. Following calibration, the differential non-linearity is within ±0.3 LSB at 8-bit resolution, and the two stimulation phases are matched within 0.3%. Individual control in digital programming of stimulation coefficients across the array allows altering the spatial profile of current stimulation for selection of stimulation targets by current steering. Combined with the self-calibration and current matching functions, the current steering capabilities integrated on-chip support use in fully implanted neural interfaces with autonomous operation for and adaptive stimulation under variations in electrode and tissue conditions. As a proof-of-concept we applied current steering stimulation through a multi-channel cuff electrode on the sciatic nerve of a rat.

A Bidirectional Neural Interface IC With Chopper Stabilized BioADC Array and Charge Balanced Stimulator

We present a bidirectional neural interface with a 4-channel biopotential analog-to-digital converter (bioADC) and a 4-channel current-mode stimulator in 180 nm CMOS. The bioADC directly transduces microvolt biopotentials into a digital representation without a voltage-amplification stage. Each bioADC channel comprises a continuous-time first-order ∆Σ modulator with a chopper-stabilized OTA input and current feedback, followed by a second-order comb-filter decimator with programmable oversampling ratio. Each stimulator channel contains two independent digital-to-analog converters for anodic and cathodic current generation. A shared calibration circuit matches the amplitude of the anodic and cathodic currents for charge balancing. Powered from a 1.5 V supply, the analog and digital circuits in each recording channel draw on average [Formula: see text] and [Formula: see text] of supply current, respectively. The bioADCs achieve an SNR of [Formula: see text] and a SFDR of [Formula: see text] , for better than 9-b ENOB. Intracranial EEG recordings from an anesthetized rat are shown and compared to simultaneous recordings from a commercial reference system to validate performance in-vivo . Additionally, we demonstrate bidirectional operation by recording cardiac modulation induced through vagus nerve stimulation, and closed-loop control of cardiac rhythm. The micropower operation, direct digital readout, and integration of electrical stimulation circuits make this interface ideally suited for closed-loop neuromodulation applications.

Intraspinal microstimulation produces over-ground walking in anesthetized cats

OBJECTIVE

Spinal cord injury causes a drastic loss of motor, sensory and autonomic function. The goal of this project was to investigate the use of intraspinal microstimulation (ISMS) for producing long distances of walking over ground. ISMS is an electrical stimulation method developed for restoring motor function by activating spinal networks below the level of an injury. It produces movements of the legs by stimulating the ventral horn of the lumbar enlargement using fine penetrating electrodes (≤50 μm diameter).

APPROACH

In each of five adult cats (4.2-5.5 kg), ISMS was applied through 16 electrodes implanted with tips targeting lamina IX in the ventral horn bilaterally. A desktop system implemented a physiologically-based control strategy that delivered different stimulation patterns through groups of electrodes to evoke walking movements with appropriate limb kinematics and forces corresponding to swing and stance. Each cat walked over an instrumented 2.9 m walkway and limb kinematics and forces were recorded.

MAIN RESULTS

Both propulsive and supportive forces were required for over-ground walking. Cumulative walking distances ranging from 609 to 835 m (longest tested) were achieved in three animals. In these three cats, the mean peak supportive force was 3.5 ± 0.6 N corresponding to full-weight-support of the hind legs, while the angular range of the hip, knee, and ankle joints were 23.1 ± 2.0°, 29.1 ± 0.2°, and 60.3 ± 5.2°, respectively. To further demonstrate the viability of ISMS for future clinical use, a prototype implantable module was successfully implemented in a subset of trials and produced comparable walking performance.

SIGNIFICANCE

By activating inherent locomotor networks within the lumbosacral spinal cord, ISMS was capable of producing bilaterally coordinated and functional over-ground walking with current amplitudes <100 μA. These exciting results suggest that ISMS may be an effective intervention for restoring functional walking after spinal cord injury.

A Fully Integrated Wireless Compressed Sensing Neural Signal Acquisition System for Chronic Recording and Brain Machine Interface

Reliable, multi-channel neural recording is critical to the neuroscience research and clinical treatment. However, most hardware development of fully integrated, multi-channel wireless neural recorders to-date, is still in the proof-of-concept stage. To be ready for practical use, the trade-offs between performance, power consumption, device size, robustness, and compatibility need to be carefully taken into account. This paper presents an optimized wireless compressed sensing neural signal recording system. The system takes advantages of both custom integrated circuits and universal compatible wireless solutions. The proposed system includes an implantable wireless system-on-chip (SoC) and an external wireless relay. The SoC integrates 16-channel low-noise neural amplifiers, programmable filters and gain stages, a SAR ADC, a real-time compressed sensing module, and a near field wireless power and data transmission link. The external relay integrates a 32 bit low-power microcontroller with Bluetooth 4.0 wireless module, a programming interface, and an inductive charging unit. The SoC achieves high signal recording quality with minimized power consumption, while reducing the risk of infection from through-skin connectors. The external relay maximizes the compatibility and programmability. The proposed compressed sensing module is highly configurable, featuring a SNDR of 9.78 dB with a compression ratio of 8×. The SoC has been fabricated in a 180 nm standard CMOS technology, occupying 2.1 mm × 0.6 mm silicon area. A pre-implantable system has been assembled to demonstrate the proposed paradigm. The developed system has been successfully used for long-term wireless neural recording in freely behaving rhesus monkey.

A Mixed-Signal VLSI System for Producing Temporally Adapting Intraspinal Microstimulation Patterns for Locomotion

Neural pathways can be artificially activated through the use of electrical stimulation. For individuals with a spinal cord injury, intraspinal microstimulation, using electrical currents on the order of 125 μ A, can produce muscle contractions and joint torques in the lower extremities suitable for restoring walking. The work presented here demonstrates an integrated circuit implementing a state-based control strategy where sensory feedback and intrinsic feed forward control shape the stimulation waveforms produced on-chip. Fabricated in a 0.5 μ m process, the device was successfully used in vivo to produce walking movements in a model of spinal cord injury. This work represents progress towards an implantable solution to be used for restoring walking in individuals with spinal cord injuries.

Compact all-CMOS spatiotemporal compressive sensing video camera with pixel-wise coded exposure

We present a low power all-CMOS implementation of temporal compressive sensing with pixel-wise coded exposure. This image sensor can increase video pixel resolution and frame rate simultaneously while reducing data readout speed. Compared to previous architectures, this system modulates pixel exposure at the individual photo-diode electronically without external optical components. Thus, the system provides reduction in size and power compare to previous optics based implementations. The prototype image sensor (127 × 90 pixels) can reconstruct 100 fps videos from coded images sampled at 5 fps. With 20× reduction in readout speed, our CMOS image sensor only consumes 14μW to provide 100 fps videos.

HFirst: A Temporal Approach to Object Recognition

This paper introduces a spiking hierarchical model for object recognition which utilizes the precise timing information inherently present in the output of biologically inspired asynchronous address event representation (AER) vision sensors. The asynchronous nature of these systems frees computation and communication from the rigid predetermined timing enforced by system clocks in conventional systems. Freedom from rigid timing constraints opens the possibility of using true timing to our advantage in computation. We show not only how timing can be used in object recognition, but also how it can in fact simplify computation. Specifically, we rely on a simple temporal-winner-take-all rather than more computationally intensive synchronous operations typically used in biologically inspired neural networks for object recognition. This approach to visual computation represents a major paradigm shift from conventional clocked systems and can find application in other sensory modalities and computational tasks. We showcase effectiveness of the approach by achieving the highest reported accuracy to date (97.5% ± 3.5%) for a previously published four class card pip recognition task and an accuracy of 84.9% ± 1.9% for a new more difficult 36 class character recognition task.

A closed-loop compressive-sensing-based neural recording system

OBJECTIVE

This paper describes a low power closed-loop compressive sensing (CS) based neural recording system. This system provides an efficient method to reduce data transmission bandwidth for implantable neural recording devices. By doing so, this technique reduces a majority of system power consumption which is dissipated at data readout interface. The design of the system is scalable and is a viable option for large scale integration of electrodes or recording sites onto a single device.

APPROACH

The entire system consists of an application-specific integrated circuit (ASIC) with 4 recording readout channels with CS circuits, a real time off-chip CS recovery block and a recovery quality evaluation block that provides a closed feedback to adaptively adjust compression rate. Since CS performance is strongly signal dependent, the ASIC has been tested in vivo and with standard public neural databases.

MAIN RESULTS

Implemented using efficient digital circuit, this system is able to achieve >10 times data compression on the entire neural spike band (500-6KHz) while consuming only 0.83uW (0.53 V voltage supply) additional digital power per electrode. When only the spikes are desired, the system is able to further compress the detected spikes by around 16 times. Unlike other similar systems, the characteristic spikes and inter-spike data can both be recovered which guarantes a >95% spike classification success rate. The compression circuit occupied 0.11mm(2)/electrode in a 180nm CMOS process. The complete signal processing circuit consumes <16uW/electrode.

SIGNIFICANCE

Power and area efficiency demonstrated by the system make it an ideal candidate for integration into large recording arrays containing thousands of electrode. Closed-loop recording and reconstruction performance evaluation further improves the robustness of the compression method, thus making the system more practical for long term recording.

Energy-efficient multi-mode compressed sensing system for implantable neural recordings

Widely utilized in the field of Neuroscience, implantable neural recording devices could capture neuron activities with an acquisition rate on the order of megabytes per second. In order to efficiently transmit neural signals through wireless channels, these devices require compression methods that reduce power consumption. Although recent Compressed Sensing (CS) approaches have successfully demonstrated their power, their full potential is yet to be explored. Built upon our previous on-chip CS implementation, we propose an energy efficient multi-mode CS framework that focuses on improving the off-chip components, including (i) a two-stage sensing strategy, (ii) a sparsifying dictionary directly using data, (iii) enhanced compression performance from Full Signal CS mode and Spike Restoration mode to Spike CS + Restoration mode and; (iv) extension of our framework to the Tetrode CS recovery using joint sparsity. This new framework achieves energy efficiency, implementation simplicity and system flexibility simultaneously. Extensive experiments are performed on simulation and real datasets. For our Spike CS + Restoration mode, we achieve a compression ratio of 6% with a reconstruction SNDR > 10 dB and a classification accuracy > 95% for synthetic datasets. For real datasets, we get a 10% compression ratio with  ∼  10 dB for Spike CS + Restoration mode.

An efficient and compact compressed sensing microsystem for implantable neural recordings

Multi-Electrode Arrays (MEA) have been widely used in neuroscience experiments. However, the reduction of their wireless transmission power consumption remains a major challenge. To resolve this challenge, an efficient on-chip signal compression method is essential. In this paper, we first introduce a signal-dependent Compressed Sensing (CS) approach that outperforms previous works in terms of compression rate and reconstruction quality. Using a publicly available database, our simulation results show that the proposed system is able to achieve a signal compression rate of 8 to 16 while guaranteeing almost perfect spike classification rate. Finally, we demonstrate power consumption measurements and area estimation of a test structure implemented using TSMC 0.18 μm process. We estimate the proposed system would occupy an area of around 200 μm ×300 μm per recording channel, and consumes 0.27 μ W operating at 20 KHz .

A model of proto-object based saliency

Organisms use the process of selective attention to optimally allocate their computational resources to the instantaneously most relevant subsets of a visual scene, ensuring that they can parse the scene in real time. Many models of bottom-up attentional selection assume that elementary image features, like intensity, color and orientation, attract attention. Gestalt psychologists, however, argue that humans perceive whole objects before they analyze individual features. This is supported by recent psychophysical studies that show that objects predict eye-fixations better than features. In this report we present a neurally inspired algorithm of object based, bottom-up attention. The model rivals the performance of state of the art non-biologically plausible feature based algorithms (and outperforms biologically plausible feature based algorithms) in its ability to predict perceptual saliency (eye fixations and subjective interest points) in natural scenes. The model achieves this by computing saliency as a function of proto-objects that establish the perceptual organization of the scene. All computational mechanisms of the algorithm have direct neural correlates, and our results provide evidence for the interface theory of attention.

Fast neuromimetic object recognition using FPGA outperforms GPU implementations

Recognition of objects in still images has traditionally been regarded as a difficult computational problem. Although modern automated methods for visual object recognition have achieved steadily increasing recognition accuracy, even the most advanced computational vision approaches are unable to obtain performance equal to that of humans. This has led to the creation of many biologically inspired models of visual object recognition, among them the hierarchical model and X (HMAX) model. HMAX is traditionally known to achieve high accuracy in visual object recognition tasks at the expense of significant computational complexity. Increasing complexity, in turn, increases computation time, reducing the number of images that can be processed per unit time. In this paper we describe how the computationally intensive and biologically inspired HMAX model for visual object recognition can be modified for implementation on a commercial field-programmable aate Array, specifically the Xilinx Virtex 6 ML605 evaluation board with XC6VLX240T FPGA. We show that with minor modifications to the traditional HMAX model we can perform recognition on images of size 128 × 128 pixels at a rate of 190 images per second with a less than 1% loss in recognition accuracy in both binary and multiclass visual object recognition tasks.

Prevention and diagnosis of retained foreign bodies through the years: past, present, and future technologies

Post operative retained foreign bodies are a rare but recalcitrant problem. We detail reports of interventions over the last two centuries and review the most current interventions using automated data identity capture and computer aided detection. This was one of earliest areas in which multidisciplinary collaboration was achieved in patient safety. This multidisciplinary collaboration was unique because most other initiatives had been internal: among the disciplines working in the OR i.e. surgeons, nurses and anesthesiologists; this collaboration, to achieve optimal patient safety at that point in time was between surgeons and radiologists to ensure a lack of post operative retained foreign bodies.

A miniaturized platform for laser speckle contrast imaging

Imaging the brain in animal models enables scientists to unravel new biological insights. Despite critical advancements in recent years, most laboratory imaging techniques comprise of bulky bench top apparatus that require the imaged animals to be anesthetized and immobilized. Thus, animals are imaged in their non-native state severely restricting the scope of behavioral experiments. To address this gap, we report a miniaturized microscope that can be mounted on a rat's head for imaging in awake and unrestrained conditions. The microscope uses laser speckle contrast imaging (LSCI), a high resolution yet wide field imaging modality for imaging blood vessels and perfusion. Design details of both the image formation and acquisition modules are presented. A Monte Carlo simulation was used to estimate the depth of tissue penetration achievable by the imaging system while the produced speckle Airy disc patterns were simulated using Fresnel's diffraction theory. The microscope system weighs only 7 g and occupies less than 5 cm³ and was successfully used to generate proof of concept LSCI images of rat brain vasculature. We validated the utility of the head-mountable system in an awake rat brain model by confirming no impairment to the rat's native behavior.

Sponges and incorrect sponge count: Minor contributions to the process of detecting retained foreign bodies

Background: Postoperative retained foreign bodies [RFBs] can be a serious event, but they are rare. The x-ray is the current gold standard to detect RFBs. There has been scant research on the process of detection as opposed to the consequence of RFBs. Surgical sponges incorporating automatic data identity capture technology (radiofrequency tags, barcodes) have been proposed to detect RFBs. Because resources in healthcare are scarce, careful consideration needs to be given to developing the right technology in order to maximize the process of RFB elimination. There have been few studies that identify factors contributing to the process of RFB detection. Study design: Our goal was to determine the frequency with which x-rays were ordered to detect abdominal surgery post operative RFBs and the indications for ordering them. We reviewed the Johns Hopkins Hospital's Department of Radiology database to retrospectively study the demographic and radiologic data on patients who underwent exploratory surgery for RFBs following abdominal procedures performed between April 2004 and April 2008. Results: Of the 13,335 portable abdominal x-rays taken during the period, 203 (1.5%) were ordered to assess patients for the presence of an RFB. Of these, 57 (28%) were taken because no RFB count was made (e.g., for emergency procedures), 57 (28%) were taken per procedure or protocol, 51 (25%) were taken because of an incorrect instrument count, and 39 (19%) were taken because of an incorrect sponge count. Of the 203 x-rays, 192 (95%) were negative for RFBs, 11 (5%) were positive or had suspicious findings, and of these 3 (2%) revealed more than 1 RFB. The 11 patients with positive or suspicious findings underwent exploratory procedures immediately during the same operation; of these, 8 (72%) actually had an RFB and 3 (28%) had a negative result at exploration. Conclusion: Multiple pathways lead to the decision to obtain X-rays for RFBs, of which sponges/Incorrect sponge counts make up only one in five. Therefore, technology that focuses on sponges alone may not majorly impact clinical outcome because x-rays will still be required in the majority of cases of suspected high risk.

Parameter estimation of a spiking silicon neuron

Spiking neuron models are used in a multitude of tasks ranging from understanding neural behavior at its most basic level to neuroprosthetics. Parameter estimation of a single neuron model, such that the model's output matches that of a biological neuron is an extremely important task. Hand tuning of parameters to obtain such behaviors is a difficult and time consuming process. This is further complicated when the neuron is instantiated in silicon (an attractive medium in which to implement these models) as fabrication imperfections make the task of parameter configuration more complex. In this paper we show two methods to automate the configuration of a silicon (hardware) neuron's parameters. First, we show how a Maximum Likelihood method can be applied to a leaky integrate and fire silicon neuron with spike induced currents to fit the neuron's output to desired spike times. We then show how a distance based method which approximates the negative log likelihood of the lognormal distribution can also be used to tune the neuron's parameters. We conclude that the distance based method is better suited for parameter configuration of silicon neurons due to its superior optimization speed.

Feed forward and feedback control for over-ground locomotion in anaesthetized cats

The biological central pattern generator (CPG) integrates open and closed loop control to produce over-ground walking. The goal of this study was to develop a physiologically based algorithm capable of mimicking the biological system to control multiple joints in the lower extremities for producing over-ground walking. The algorithm used state-based models of the step cycle each of which produced different stimulation patterns. Two configurations were implemented to restore over-ground walking in five adult anaesthetized cats using intramuscular stimulation (IMS) of the main hip, knee and ankle flexor and extensor muscles in the hind limbs. An open loop controller relied only on intrinsic timing while a hybrid-CPG controller added sensory feedback from force plates (representing limb loading), and accelerometers and gyroscopes (representing limb position). Stimulation applied to hind limb muscles caused extension or flexion in the hips, knees and ankles. A total of 113 walking trials were obtained across all experiments. Of these, 74 were successful in which the cats traversed 75% of the 3.5 m over-ground walkway. In these trials, the average peak step length decreased from 24.9 ± 8.4 to 21.8 ± 7.5 (normalized units) and the median number of steps per trial increased from 7 (Q1 = 6, Q3 = 9) to 9 (8, 11) with the hybrid-CPG controller. Moreover, within these trials, the hybrid-CPG controller produced more successful steps (step length ≤ 20 cm; ground reaction force ≥ 12.5% body weight) than the open loop controller: 372 of 544 steps (68%) versus 65 of 134 steps (49%), respectively. This supports our previous preliminary findings, and affirms that physiologically based hybrid-CPG approaches produce more successful stepping than open loop controllers. The algorithm provides the foundation for a neural prosthetic controller and a framework to implement more detailed control of locomotion in the future.

Sensor and display human factors based design constraints for head mounted and tele-operation systems

For mobile imaging systems in head mounted displays and tele-operation systems it is important to maximize the amount of visual information transmitted to the human visual system without exceeding its input capacity. This paper aims to describe the design constraints on the imager and display systems of head mounted devices and tele-operated systems based upon the capabilities of the human visual system. We also present the experimental results of methods to improve the amount of visual information conveyed to a user when trying to display a high dynamic range image on a low dynamic range display.

Modified map-seeking circuit: use of computer-aided detection in locating postoperative retained foreign bodies

BACKGROUND

More than 98% of intra-operative X-rays taken to search for postoperative retained foreign bodies (RFBs) have negative findings; in over 30% of cases of such X-rays, the finding is a false negative. Newer technologies created to find RFBs must not only reduce the false-negative rate, but also must not increase the burden of detecting RFBs. We have introduced the use of computer-aided detection (CAD) to facilitate the detection of RFBs on X-rays utilizing a modified version of map-seeking circuit (MSC) algorithm the referenced map-seeking circuit (RMSC), for our proof-of-concept study for detection of needles in plain abdominal X-rays.

METHODS

Images were obtained by using a portable cassette-based X-ray machine and a C-arm (digital) machine, both of which are commonly used in the operating room. The images obtained using these machines were divided into subimages of approximately 250 × 250 pixels each, for a total of 455 subimages from the cassette-based machine (A) and 365 from the digital machine (B) for use as test samples. Images obtained from A and B were analyzed separately using our modified MSC algorithm with a minimum (τ = 0) and a maximum threshold (τ = 0.5).

RESULTS

The automated detection rate (positive predictive value) was 86%, with a false positive/negative rate of 10% to 15% when τ was zero.

CONCLUSION

The CAD-based RMSC algorithm has the potential to improve the accuracy with which RFBs can be found in X-rays. Further research is needed to optimize the detection rate and to identify a wider range of RFBs.

Silicon modeling of the Mihalaş-Niebur neuron

There are a number of spiking and bursting neuron models with varying levels of complexity, ranging from the simple integrate-and-fire model to the more complex Hodgkin-Huxley model. The simpler models tend to be easily implemented in silicon but yet not biologically plausible. Conversely, the more complex models tend to occupy a large area although they are more biologically plausible. In this paper, we present the 0.5 μm complementary metal-oxide-semiconductor (CMOS) implementation of the Mihalaş-Niebur neuron model--a generalized model of the leaky integrate-and-fire neuron with adaptive threshold--that is able to produce most of the known spiking and bursting patterns that have been observed in biology. Our implementation modifies the original proposed model, making it more amenable to CMOS implementation and more biologically plausible. All but one of the spiking properties--tonic spiking, class 1 spiking, phasic spiking, hyperpolarized spiking, rebound spiking, spike frequency adaptation, accommodation, threshold variability, integrator and input bistability--are demonstrated in this model.

A CMOS In-Pixel CTIA High Sensitivity Fluorescence Imager

Traditionally, charge coupled device (CCD) based image sensors have held sway over the field of biomedical imaging. Complementary metal oxide semiconductor (CMOS) based imagers so far lack sensitivity leading to poor low-light imaging. Certain applications including our work on animal-mountable systems for imaging in awake and unrestrained rodents require the high sensitivity and image quality of CCDs and the low power consumption, flexibility and compactness of CMOS imagers. We present a 132×124 high sensitivity imager array with a 20.1 μm pixel pitch fabricated in a standard 0.5 μ CMOS process. The chip incorporates n-well/p-sub photodiodes, capacitive transimpedance amplifier (CTIA) based in-pixel amplification, pixel scanners and delta differencing circuits. The 5-transistor all-nMOS pixel interfaces with peripheral pMOS transistors for column-parallel CTIA. At 70 fps, the array has a minimum detectable signal of 4 nW/cm(2) at a wavelength of 450 nm while consuming 718 μA from a 3.3 V supply. Peak signal to noise ratio (SNR) was 44 dB at an incident intensity of 1 μW/cm(2). Implementing 4×4 binning allowed the frame rate to be increased to 675 fps. Alternately, sensitivity could be increased to detect about 0.8 nW/cm(2) while maintaining 70 fps. The chip was used to image single cell fluorescence at 28 fps with an average SNR of 32 dB. For comparison, a cooled CCD camera imaged the same cell at 20 fps with an average SNR of 33.2 dB under the same illumination while consuming over a watt.

Estimating parameters of generalized integrate-and-fire neurons from the maximum likelihood of spike trains

When a neuronal spike train is observed, what can we deduce from it about the properties of the neuron that generated it? A natural way to answer this question is to make an assumption about the type of neuron, select an appropriate model for this type, and then choose the model parameters as those that are most likely to generate the observed spike train. This is the maximum likelihood method. If the neuron obeys simple integrate-and-fire dynamics, Paninski, Pillow, and Simoncelli (2004) showed that its negative log-likelihood function is convex and that, at least in principle, its unique global minimum can thus be found by gradient descent techniques. Many biological neurons are, however, known to generate a richer repertoire of spiking behaviors than can be explained in a simple integrate-and-fire model. For instance, such a model retains only an implicit (through spike-induced currents), not an explicit, memory of its input; an example of a physiological situation that cannot be explained is the absence of firing if the input current is increased very slowly. Therefore, we use an expanded model (Mihalas & Niebur, 2009 ), which is capable of generating a large number of complex firing patterns while still being linear. Linearity is important because it maintains the distribution of the random variables and still allows maximum likelihood methods to be used. In this study, we show that although convexity of the negative log-likelihood function is not guaranteed for this model, the minimum of this function yields a good estimate for the model parameters, in particular if the noise level is treated as a free parameter. Furthermore, we show that a nonlinear function minimization method (r-algorithm with space dilation) usually reaches the global minimum.

Neuromorphic silicon neuron circuits

Hardware implementations of spiking neurons can be extremely useful for a large variety of applications, ranging from high-speed modeling of large-scale neural systems to real-time behaving systems, to bidirectional brain–machine interfaces. The specific circuit solutions used to implement silicon neurons depend on the application requirements. In this paper we describe the most common building blocks and techniques used to implement these circuits, and present an overview of a wide range of neuromorphic silicon neurons, which implement different computational models, ranging from biophysically realistic and conductance-based Hodgkin–Huxley models to bi-dimensional generalized adaptive integrate and fire models. We compare the different design methodologies used for each silicon neuron design described, and demonstrate their features with experimental results, measured from a wide range of fabricated VLSI chips.

An integrated imaging microscope for untethered cortical imaging in freely-moving animals

Imaging in awake, behaving animals is an emerging field that offers the advantage of being able to study physiological processes and structures in a more natural state than what is possible in tissue slices or even in anesthetized animals. To date, most imaging in awake animals has used optical fiber bundles or electrical cables to transfer signals to traditional imaging-system components. However, the fibers or cables tether the animal and greatly limit the kind and duration of animal behavior that can be studied using imaging methods. We present an integrated imaging microscope (IIM) that incorporates all aspects of an imaging system - illumination, optics and photodetection - into a small footprint device, occupying under 4 cm(3) and weighing 5.4 g, that can be attached to the skull for imaging the brain in mobile rats. Power supply and image storage sufficient for approximately 7 hour operation at 15 frames/s was implemented on a backpack weighing 11.5 g. We implemented several optical techniques including reflectance, spectroscopy, speckle and fluorescence with the IIM, imaged vessels down to 15-20 microm in diameter and obtained, to the best of our knowledge, the worlds first cortical images from an untethered, freely-moving rat.

Optimization methods for spiking neurons and networks

Spiking neurons and spiking neural circuits are finding uses in a multitude of tasks such as robotic locomotion control, neuroprosthetics, visual sensory processing, and audition. The desired neural output is achieved through the use of complex neuron models, or by combining multiple simple neurons into a network. In either case, a means for configuring the neuron or neural circuit is required. Manual manipulation of parameters is both time consuming and non-intuitive due to the nonlinear relationship between parameters and the neuron's output. The complexity rises even further as the neurons are networked and the systems often become mathematically intractable. In large circuits, the desired behavior and timing of action potential trains may be known but the timing of the individual action potentials is unknown and unimportant, whereas in single neuron systems the timing of individual action potentials is critical. In this paper, we automate the process of finding parameters. To configure a single neuron we derive a maximum likelihood method for configuring a neuron model, specifically the Mihalas-Niebur Neuron. Similarly, to configure neural circuits, we show how we use genetic algorithms (GAs) to configure parameters for a network of simple integrate and fire with adaptation neurons. The GA approach is demonstrated both in software simulation and hardware implementation on a reconfigurable custom very large scale integration chip.