Toward a Proprioceptive Neural Interface that Mimics Natural Cortical Activity

Biomimetic computer-to-brain communication enhancing naturalistic touch sensations via peripheral nerve stimulation

Artificial communication with the brain through peripheral nerve stimulation shows promising results in individuals with sensorimotor deficits. However, these efforts lack an intuitive and natural sensory experience. In this study, we design and test a biomimetic neurostimulation framework inspired by nature, capable of "writing" physiologically plausible information back into the peripheral nervous system. Starting from an in-silico model of mechanoreceptors, we develop biomimetic stimulation policies. We then experimentally assess them alongside mechanical touch and common linear neuromodulations. Neural responses resulting from biomimetic neuromodulation are consistently transmitted towards dorsal root ganglion and spinal cord of cats, and their spatio-temporal neural dynamics resemble those naturally induced. We implement these paradigms within the bionic device and test it with patients (ClinicalTrials.gov identifier NCT03350061). He we report that biomimetic neurostimulation improves mobility (primary outcome) and reduces mental effort (secondary outcome) compared to traditional approaches. The outcomes of this neuroscience-driven technology, inspired by the human body, may serve as a model for advancing assistive neurotechnologies.

Amplitude- and frequency-dependent activation of layer II/III neurons by intracortical microstimulation

Intracortical microstimulation (ICMS) has been used for the development of brain machine interfaces. However, further understanding about the spatiotemporal responses of neurons to different electrical stimulation parameters is necessary to inform the design of optimal therapies. In this study, we employed in vivo electrophysiological recording, two-photon calcium imaging, and electric field simulation to evaluate the acute effect of ICMS on layer II/III neurons. Our results show that stimulation frequency non-linearly modulates neuronal responses, whereas the magnitude of responses is linearly correlated to the electric field strength and stimulation amplitude before reaching a steady state. Temporal dynamics of neurons' responses depends more on stimulation frequency and their distance to the stimulation electrode. In addition, amplitude-dependent post-stimulation suppression was observed within ∼500 μm of the stimulation electrode, as evidenced by both calcium imaging and local field potentials. These findings provide insights for selecting stimulation parameters to achieve desirable spatiotemporal specificity of ICMS.

Restoration of sensory information via bionic hands

Individuals who have lost the use of their hands because of amputation or spinal cord injury can use prosthetic hands to restore their independence. A dexterous prosthesis requires the acquisition of control signals that drive the movements of the robotic hand, and the transmission of sensory signals to convey information to the user about the consequences of these movements. In this Review, we describe non-invasive and invasive technologies for conveying artificial sensory feedback through bionic hands, and evaluate the technologies' long-term prospects.

Symbiotic electroneural and musculoskeletal framework to encode proprioception via neurostimulation: ProprioStim

Peripheral nerve stimulation in amputees achieved the restoration of touch, but not proprioception, which is critical in locomotion. A plausible reason is the lack of means to artificially replicate the complex activity of proprioceptors. To uncover this, we coupled neuromuscular models from ten subjects and nerve histologies from two implanted amputees to develop ProprioStim: a framework to encode proprioception by electrical evoking neural activity in close agreement with natural proprioceptive activity. We demonstrated its feasibility through non-invasive stimulation on seven healthy subjects comparing it with standard linear charge encoding. Results showed that ProprioStim multichannel stimulation was felt more natural, and hold promises for increasing accuracy in knee angle tracking, especially in future implantable solutions. Additionally, we quantified the importance of realistic 3D-nerve models against extruded models previously adopted for further design and validation of novel neurostimulation encoding strategies. ProprioStim provides clear guidelines for the development of neurostimulation policies restoring natural proprioception.

The science and engineering behind sensitized brain-controlled bionic hands

Advances in our understanding of brain function, along with the development of neural interfaces that allow for the monitoring and activation of neurons, have paved the way for brain-machine interfaces (BMIs), which harness neural signals to reanimate the limbs via electrical activation of the muscles or to control extracorporeal devices, thereby bypassing the muscles and senses altogether. BMIs consist of reading out motor intent from the neuronal responses monitored in motor regions of the brain and executing intended movements with bionic limbs, reanimated limbs, or exoskeletons. BMIs also allow for the restoration of the sense of touch by electrically activating neurons in somatosensory regions of the brain, thereby evoking vivid tactile sensations and conveying feedback about object interactions. In this review, we discuss the neural mechanisms of motor control and somatosensation in able-bodied individuals and describe approaches to use neuronal responses as control signals for movement restoration and to activate residual sensory pathways to restore touch. Although the focus of the review is on intracortical approaches, we also describe alternative signal sources for control and noninvasive strategies for sensory restoration.

Intrinsic somatosensory feedback supports motor control and learning to operate artificial body parts

Objective.Considerable resources are being invested to enhance the control and usability of artificial limbs through the delivery of unnatural forms of somatosensory feedback. Here, we investigated whether intrinsic somatosensory information from the body part(s) remotely controlling an artificial limb can be leveraged by the motor system to support control and skill learning.Approach.We used local anaesthetic to attenuate somatosensory inputs to the big toes while participants learned to operate through pressure sensors a toe-controlled and hand-worn robotic extra finger. Motor learning outcomes were compared against a control group who received sham anaesthetic and quantified in three different task scenarios: while operating in isolation from, in synchronous coordination, and collaboration with, the biological fingers.Main results.Both groups were able to learn to operate the robotic extra finger, presumably due to abundance of visual feedback and other relevant sensory cues. Importantly, the availability of displaced somatosensory cues from the distal bodily controllers facilitated the acquisition of isolated robotic finger movements, the retention and transfer of synchronous hand-robot coordination skills, and performance under cognitive load. Motor performance was not impaired by toes anaesthesia when tasks involved close collaboration with the biological fingers, indicating that the motor system can close the sensory feedback gap by dynamically integrating task-intrinsic somatosensory signals from multiple, and even distal, body-parts.Significance.Together, our findings demonstrate that there are multiple natural avenues to provide intrinsic surrogate somatosensory information to support motor control of an artificial body part, beyond artificial stimulation.

Charge injection characteristics of sputtered ruthenium oxide electrodes for neural stimulation and recording

We have studied the charge-injection characteristics and electrochemical impedance of sputtered ruthenium oxide (RuO_x ) films as electrode coatings for neural stimulation and recording electrodes. RuO_x films were deposited by reactive DC magnetron sputtering, using a combination of water vapor and oxygen gas as reactive plasma constituents. The cathodal charge storage capacity of planar RuO_x electrodes was found to be 54.6 ± 9.5 mC/cm² (mean ± SD, n = 12), and the charge-injection capacity in a 0.2-ms cathodal current pulse was found to be 7.1 ± 0.3 mC/cm² (mean ± SD, n = 15) at 0.6 V positive bias versus Ag|AgCl, in phosphate buffer saline at room temperature for ~250 nm thick films. In general, the RuO_x films exhibited high charge-injection capacities, with or without a positive interpulse bias, comparable to sputtered iridium oxide (SIROF) coatings. The charge-injection capacity increased monotonically with film thickness from 120 to 630 nm, and reached 11.30 ± 0.34 mC/cm² (mean ± SD, n = 5) at 0.6 V bias versus Ag|AgCl at 630 nm film thickness. In addition, RuO_x films showed minimal changes in electrochemical characteristics over 1.5 billion cycles of constant current pulsing at a charge density of 408 μC/cm² (8 nC/phase, 200 μs pulse width). The findings of low-impedance, high charge-injection capacity, and long-term pulsing stability suggest the suitability of RuO_x as a comparatively inexpensive and favorable choice of electrode material for neural stimulation and recording.

A comprehensive model-based framework for optimal design of biomimetic patterns of electrical stimulation for prosthetic sensation

OBJECTIVE

Touch and proprioception are essential to motor function as shown by the movement deficits that result from the loss of these senses, e.g. due to neuropathy of sensory nerves. To achieve a high-performance brain-controlled prosthetic arm/hand thus requires the restoration of somatosensation, perhaps through intracortical microstimulation (ICMS) of somatosensory cortex (S1). The challenge is to generate patterns of neuronal activation that evoke interpretable percepts. We present a framework to design optimal spatiotemporal patterns of ICMS (STIM) that evoke naturalistic patterns of neuronal activity and demonstrate performance superior to four previous approaches.

APPROACH

We recorded multiunit activity from S1 during a center-out reach task (from proprioceptive neurons in Brodmann's area 2) and during application of skin indentations (from cutaneous neurons in Brodmann's area 1). We implemented a computational model of a cortical hypercolumn and used a genetic algorithm to design STIM that evoked patterns of model neuron activity that mimicked their experimentally-measured counterparts. Finally, from the ICMS patterns, the evoked neuronal activity, and the stimulus parameters that gave rise to it, we trained a recurrent neural network (RNN) to learn the mapping function between the physical stimulus and the biomimetic stimulation pattern, i.e. the sensory encoder to be integrated into a neuroprosthetic device.

MAIN RESULTS

We identified ICMS patterns that evoked simulated responses that closely approximated the measured responses for neurons within 50 µm of the electrode tip. The RNN-based sensory encoder generalized well to untrained limb movements or skin indentations. STIM designed using the model-based optimization approach outperformed STIM designed using existing linear and nonlinear mappings.

SIGNIFICANCE

The proposed framework produces an encoder that converts limb state or patterns of pressure exerted onto the prosthetic hand into STIM that evoke naturalistic patterns of neuronal activation.

Sputtered ruthenium oxide coatings for neural stimulation and recording electrodes

We have investigated the deposition and electrochemical properties of sputtered ruthenium oxide coatings for neural stimulation and recording electrodes. A combination of oxygen and water vapor was used as a reactive gas mixture during DC magnetron sputtering from a ruthenium metal target. The sputtering plasma was monitored by optical emission spectroscopy to determine the reactive species present and confirm the control of plasma chemistry by reactive gas flow rates into the deposition chamber. The effect of the O₂ :H₂ O gas ratio on the microstructure and electrochemical properties of the ruthenium oxide were studied in detail. We employed a combination of surface characterization techniques, including scanning electron microscopy, x-ray diffraction, and x-ray photoelectron spectroscopy, to understand the relationship between plasma chemistry and the microstructure of the films produced under different gas flow conditions. Electrochemical characterization included cyclic voltammetry, electrochemical impedance spectroscopy, and voltage transient measurements, performed on planar ruthenium oxide electrodes with a geometric surface area of 1960 μm² . At an O₂ :H₂ O gas flow rate ratio of 1:3, a cathodal charge-storage capacity per unit film thickness of 228.7 mC cm^-2 μm^-1 (median, Q1 = 134.5, Q3 = 236.6, n = 15) and a charge-injection capacity (0.6 V anodal interpulse bias) of 7.4 mC cm^-2 (median, Q1 = 6.9, Q3 = 8.3, n = 15) were obtained in phosphate buffered saline. The charge-injection capacity of ruthenium oxide sputtered with water vapor in the reactive plasma is comparable with sputtered iridium oxide (SIROF) and higher than reported values for porous TiN, a commonly employed high-surface area stimulation electrode coating.

The combination of brain-computer interfaces and artificial intelligence: applications and challenges

Brain-computer interfaces (BCIs) have shown great prospects as real-time bidirectional links between living brains and actuators. Artificial intelligence (AI), which can advance the analysis and decoding of neural activity, has turbocharged the field of BCIs. Over the past decade, a wide range of BCI applications with AI assistance have emerged. These "smart" BCIs including motor and sensory BCIs have shown notable clinical success, improved the quality of paralyzed patients' lives, expanded the athletic ability of common people and accelerated the evolution of robots and neurophysiological discoveries. However, despite technological improvements, challenges remain with regard to the long training periods, real-time feedback, and monitoring of BCIs. In this article, the authors review the current state of AI as applied to BCIs and describe advances in BCI applications, their challenges and where they could be headed in the future.

Area 2 of primary somatosensory cortex encodes kinematics of the whole arm

Proprioception, the sense of body position, movement, and associated forces, remains poorly understood, despite its critical role in movement. Most studies of area 2, a proprioceptive area of somatosensory cortex, have simply compared neurons' activities to the movement of the hand through space. Using motion tracking, we sought to elaborate this relationship by characterizing how area 2 activity relates to whole arm movements. We found that a whole-arm model, unlike classic models, successfully predicted how features of neural activity changed as monkeys reached to targets in two workspaces. However, when we then evaluated this whole-arm model across active and passive movements, we found that many neurons did not consistently represent the whole arm over both conditions. These results suggest that 1) neural activity in area 2 includes representation of the whole arm during reaching and 2) many of these neurons represented limb state differently during active and passive movements.

Short reaction times in response to multi-electrode intracortical microstimulation may provide a basis for rapid movement-related feedback

OBJECTIVE

Tetraplegic patients using brain-machine interfaces can make visually guided reaches with robotic arms. However, restoring proprioceptive feedback to these patients will be critical, as evidenced by the movement deficit in patients with proprioceptive loss. Proprioception is critical in large part because it provides faster feedback than vision. Intracortical microstimulation (ICMS) is a promising approach, but the ICMS-evoked reaction time (RT) is typically slower than that to natural proprioceptive and often even visual cues, implying that ICMS feedback may not be fast enough to guide movement.

APPROACH

For most sensory modalities, RT decreases with increased stimulus intensity. Thus, it may be that stimulation intensities beyond what has previously been used will result in faster RTs. To test this, we compared the RT to ICMS applied through multi-electrode arrays in area 2 of somatosensory cortex to that of mechanical and visual cues.

MAIN RESULTS

We found that the RT to single-electrode ICMS decreased with increased current, frequency, and train length. For 100 µA, 330 Hz stimulation, the highest single-electrode intensity we tested routinely, most electrodes resulted in RTs slower than the mechanical cue but slightly faster than the visual cue. While increasing the current beyond 100 µA resulted in faster RTs, sustained stimulation at this level may damage tissue. Alternatively, by stimulating through multiple electrodes (mICMS), a large amount of current can be injected while keeping that through each electrode at a safe level. We found that stimulation with at least 480 µA equally distributed over 16 electrodes could produce RTs as much as 20 ms faster than the mechanical cue, roughly the conduction delay to cortex from the periphery.

SIGNIFICANCE

These results suggest that mICMS may provide a means to supply rapid, movement-related feedback. Future neuroprosthetics may need spatiotemporally patterned mICMS to convey useful somatosensory information. Novelty & Significance Intracortical microstimulation (ICMS) is a promising approach for providing artificial somatosensation to patients with spinal cord injury or limb amputation, but in prior experiments, subjects have been unable to respond as quickly to it as to natural cues. We have investigated the use of multi-electrode stimulation (mICMS) and discovered that it can produce reaction times as fast or faster even than natural mechanical cues. Although our stimulus trains were not modulated in time, this result opens the door to more complex spatiotemporal patterns of mICMS that might be used to rapidly write in complex somatosensory information to the CNS.

Neuromusculoskeletal Modeling-Based Prostheses for Recovery After Spinal Cord Injury

Concurrent stimulation and reinforcement of motor and sensory pathways has been proposed as an effective approach to restoring function after developmental or acquired neurotrauma. This can be achieved by applying multimodal rehabilitation regimens, such as thought-controlled exoskeletons or epidural electrical stimulation to recover motor pattern generation in individuals with spinal cord injury (SCI). However, the human neuromusculoskeletal (NMS) system has often been oversimplified in designing rehabilitative and assistive devices. As a result, the neuromechanics of the muscles is seldom considered when modeling the relationship between electrical stimulation, mechanical assistance from exoskeletons, and final joint movement. A powerful way to enhance current neurorehabilitation is to develop the next generation prostheses incorporating personalized NMS models of patients. This strategy will enable an individual voluntary interfacing with multiple electromechanical rehabilitation devices targeting key afferent and efferent systems for functional improvement. This narrative review discusses how real-time NMS models can be integrated with finite element (FE) of musculoskeletal tissues and interface multiple assistive and robotic devices with individuals with SCI to promote neural restoration. In particular, the utility of NMS models for optimizing muscle stimulation patterns, tracking functional improvement, monitoring safety, and providing augmented feedback during exercise-based rehabilitation are discussed.

Illusory movement perception improves motor control for prosthetic hands

To effortlessly complete an intentional movement, the brain needs feedback from the body regarding the movement's progress. This largely nonconscious kinesthetic sense helps the brain to learn relationships between motor commands and outcomes to correct movement errors. Prosthetic systems for restoring function have predominantly focused on controlling motorized joint movement. Without the kinesthetic sense, however, these devices do not become intuitively controllable. We report a method for endowing human amputees with a kinesthetic perception of dexterous robotic hands. Vibrating the muscles used for prosthetic control via a neural-machine interface produced the illusory perception of complex grip movements. Within minutes, three amputees integrated this kinesthetic feedback and improved movement control. Combining intent, kinesthesia, and vision instilled participants with a sense of agency over the robotic movements. This feedback approach for closed-loop control opens a pathway to seamless integration of minds and machines.

Towards neural co-processors for the brain: combining decoding and encoding in brain-computer interfaces

The field of brain-computer interfaces is poised to advance from the traditional goal of controlling prosthetic devices using brain signals to combining neural decoding and encoding within a single neuroprosthetic device. Such a device acts as a 'co-processor' for the brain, with applications ranging from inducing Hebbian plasticity for rehabilitation after brain injury to reanimating paralyzed limbs and enhancing memory. We review recent progress in simultaneous decoding and encoding for closed-loop control and plasticity induction. To address the challenge of multi-channel decoding and encoding, we introduce a unifying framework for developing brain co-processors based on artificial neural networks and deep learning. These 'neural co-processors' can be used to jointly optimize cost functions with the nervous system to achieve desired behaviors ranging from targeted neuro-rehabilitation to augmentation of brain function.

Neural Basis of Touch and Proprioception in Primate Cortex

The sense of proprioception allows us to keep track of our limb posture and movements and the sense of touch provides us with information about objects with which we come into contact. In both senses, mechanoreceptors convert the deformation of tissues-skin, muscles, tendons, ligaments, or joints-into neural signals. Tactile and proprioceptive signals are then relayed by the peripheral nerves to the central nervous system, where they are processed to give rise to percepts of objects and of the state of our body. In this review, we first examine briefly the receptors that mediate touch and proprioception, their associated nerve fibers, and pathways they follow to the cerebral cortex. We then provide an overview of the different cortical areas that process tactile and proprioceptive information. Next, we discuss how various features of objects-their shape, motion, and texture, for example-are encoded in the various cortical fields, and the susceptibility of these neural codes to attention and other forms of higher-order modulation. Finally, we summarize recent efforts to restore the senses of touch and proprioception by electrically stimulating somatosensory cortex. © 2018 American Physiological Society. Compr Physiol 8:1575-1602, 2018.

Proprioceptive and cutaneous sensations in humans elicited by intracortical microstimulation

Pioneering work with nonhuman primates and recent human studies established intracortical microstimulation (ICMS) in primary somatosensory cortex (S1) as a method of inducing discriminable artificial sensation. However, these artificial sensations do not yet provide the breadth of cutaneous and proprioceptive percepts available through natural stimulation. In a tetraplegic human with two microelectrode arrays implanted in S1, we report replicable elicitations of sensations in both the cutaneous and proprioceptive modalities localized to the contralateral arm, dependent on both amplitude and frequency of stimulation. Furthermore, we found a subset of electrodes that exhibited multimodal properties, and that proprioceptive percepts on these electrodes were associated with higher amplitudes, irrespective of the frequency. These novel results demonstrate the ability to provide naturalistic percepts through ICMS that can more closely mimic the body’s natural physiological capabilities. Furthermore, delivering both cutaneous and proprioceptive sensations through artificial somatosensory feedback could improve performance and embodiment in brain-machine interfaces.

Nerves throughout the body send information about touch, temperature, body position and pain through the spinal cord to the brain. A part of the brain called the somatosensory cortex processes this information. Spinal cord injuries disrupt these messages. Even though the somatosensory cortex has not been damaged, sensation is lost for the affected body areas. No treatment exists to repair the spinal cord so the loss of sensation is permanent.

Applying electricity to the somatosensory cortex can produce artificial sensations. Scientists are testing this approach to restore a sense of touch for people with spinal cord injury. Early experiments show that using different patterns of electrical stimulation generates unnatural sensations in different body parts. People receiving the stimulation describe it as tingling or shocks. Scientists wonder if they can improve the technique to mimic feelings like touch or body position to make it easier for people with a spinal injury to move or use prostheses.

Now, Armenta Salas et al. generated more natural sensations in a person with a spinal cord injury. Instead of taking the usual approach of delivering large currents to the surface of cortex, they inserted small electrodes into the inside of the cortex to stimulate it with small currents. In the experiments, electrodes were implanted in the somatosensory cortex of a volunteer who had lost the use of his limbs and torso because of a spinal injury. Armenta Salas et al. applied different patterns of electrical stimuli and the volunteer reported what they felt like. The patient described sensations like a pinch or squeeze in the forearm or upper arm with certain patterns. In some cases, the patient reported the sensation of the arm moving with stronger electrical currents.

The experiments show that electrical stimulation of the brain can recreate some natural sensations. These sensations could help patients using robotic or prosthetic arms become more dexterous. It might also help patients view artificial limbs as part of their bodies, which could improve their sense of wellbeing.

A Highly Selective 3D Spiked Ultraflexible Neural (SUN) Interface for Decoding Peripheral Nerve Sensory Information

Artificial sensors on the skin are proposed as a way to capture information that can be used in intracortical microstimulation or peripheral intraneural stimulation to restore sensory feedback to persons with tetraplegia. However, the ability of these artificial sensors to replicate the density and complexity of the natural mechanoreceptors is limited. One relatively unexplored approach is to make use of the signals from surviving tactile and proprioceptive receptors in existing limbs by recording from their transmitting axons within the primary sensory nerves. Here, a novel spiked ultraflexible neural (SUN) interface that is implanted into the peripheral nervous system to capture sensory information from these mechanoreceptors in acute rat experiments is described. The novel 3D design, which integrates spiked structures for intrafascicular nerve recording with an ultraflexible substrate, enables a unique conformal interface to the target nerve. With the high-quality recording (average signal-to-noise-ratio of 1.4) provided by the electrode, tactile from proprioceptive stimuli can be differentiated in terms of the firing rate. In toe pinching experiments, high spatial resolution classification can be achieved with support vector machine classifier. Further work remains to be done to assess the chronic recording capability of the SUN interface.

Sensory percepts induced by microwire array and DBS microstimulation in human sensory thalamus

BACKGROUND

Microstimulation in human sensory thalamus (ventrocaudal, VC) results in focal sensory percepts in the hand and arm which may provide an alternative target site (to somatosensory cortex) for the input of prosthetic sensory information. Sensory feedback to facilitate motor function may require simultaneous or timed responses across separate digits to recreate perceptions of slip as well as encoding of intensity variations in pressure or touch.

OBJECTIVES

To determine the feasibility of evoking sensory percepts on separate digits with variable intensity through either a microwire array or deep brain stimulation (DBS) electrode, recreating "natural" and scalable percepts relating to the arm and hand.

METHODS

We compared microstimulation within ventrocaudal sensory thalamus through either a 16-channel microwire array (∼400 kΩ per channel) or a 4-channel DBS electrode (∼1.2 kΩ per contact) for percept location, size, intensity, and quality sensation, during thalamic DBS electrode placement in patients with essential tremor.

RESULTS

Percepts in small hand or finger regions were evoked by microstimulation through individual microwires and in 5/6 patients sensation on different digits could be perceived from stimulation through separate microwires. Microstimulation through DBS electrode contacts evoked sensations over larger areas in 5/5 patients, and the apparent intensity of the perceived response could be modulated with stimulation amplitude. The perceived naturalness of the sensation depended both on the pattern of stimulation as well as intensity of the stimulation.

CONCLUSIONS

Producing consistent evoked perceptions across separate digits within sensory thalamus is a feasible concept and a compact alternative to somatosensory cortex microstimulation for prosthetic sensory feedback. This approach will require a multi-element low impedance electrode with a sufficient stimulation range to evoke variable intensities of perception and a predictable spread of contacts to engage separate digits.