Pavlovian control of intraspinal microstimulation to produce over-ground walking

In vitro biocompatibility evaluation of functional electrically stimulating microelectrodes on primary glia

Neural interfacing devices interact with the central nervous system to alleviate functional deficits arising from disease or injury. This often entails the use of invasive microelectrode implants that elicit inflammatory responses from glial cells and leads to loss of device function. Previous work focused on improving implant biocompatibility by modifying electrode composition; here, we investigated the direct effects of electrical stimulation on glial cells at the electrode interface. A high-throughput in vitro system that assesses primary glial cell response to biphasic stimulation waveforms at 0 mA, 0.15 mA, and 1.5 mA was developed and optimized. Primary mixed glial cell cultures were generated from heterozygous CX3CR-1+/EGFP mice, electrically stimulated for 4 h/day over 3 days using 75 μm platinum-iridium microelectrodes, and biomarker immunofluorescence was measured. Electrodes were then imaged on a scanning electron microscope to assess sustained electrode damage. Fluorescence and electron microscopy analyses suggest varying degrees of localized responses for each biomarker assayed (Hoescht, EGFP, GFAP, and IL-1β), a result that expands on comparable in vivo models. This system allows for the comparison of a breadth of electrical stimulation parameters, and opens another avenue through which neural interfacing device developers can improve biocompatibility and longevity of electrodes in tissue.

Generation of locomotor‑like activity using monopolar intraspinal electrical microstimulation in rats

Severe spinal cord injury (SCI) affects the ability of functional standing and walking. As the locomotor central pattern generator (CPG) in the lumbosacral spinal cord can generate a regulatory signal for movement, it is feasible to activate CPG neural network using intra-spinal micro-stimulation (ISMS) to induce alternating patterns. The present study identified two special sites with the ability to activate the CPG neural network that are symmetrical about the posterior median sulcus in the lumbosacral spinal cord by ISMS in adult rats. A reversal of flexion and extension can occur in an attempt to generate a stepping movement of the bilateral hindlimb by either reversing the pulse polarity of the stimulus or changing the special site. Therefore, locomotor-like activity can be restored with monopolar intraspinal electrical stimulation on either special site. To verify the motor function regeneration of the paralyzed hindlimbs, a four-week locomotor training with ISMS applied to the special site in the SCI + ISMS group (n=12) was performed. Evaluations of motor function recovery using behavior, kinematics and physiological analyses, were used to assess hindlimb function and the results showed the stimulation at one special site can promote significant functional recovery of the bilateral hindlimbs (P<0.05). The present study suggested that motor function of paralyzed bilateral hindlimbs can be restored with monopolar ISMS.

Fast inference of spinal neuromodulation for motor control using amortized neural networks

Objective.Epidural electrical stimulation (EES) has emerged as an approach to restore motor function following spinal cord injury (SCI). However, identifying optimal EES parameters presents a significant challenge due to the complex and stochastic nature of muscle control and the combinatorial explosion of possible parameter configurations. Here, we describe a machine-learning approach that leverages modern deep neural networks to learn bidirectional mappings between the space of permissible EES parameters and target motor outputs.Approach.We collected data from four sheep implanted with two 24-contact EES electrode arrays on the lumbosacral spinal cord. Muscle activity was recorded from four bilateral hindlimb electromyography (EMG) sensors. We introduce a general learning framework to identify EES parameters capable of generating desired patterns of EMG activity. Specifically, we first amortize spinal sensorimotor computations in a forward neural network model that learns to predict motor outputs based on EES parameters. Then, we employ a second neural network as an inverse model, which reuses the amortized knowledge learned by the forward model to guide the selection of EES parameters.Main results.We found that neural networks can functionally approximate spinal sensorimotor computations by accurately predicting EMG outputs based on EES parameters. The generalization capability of the forward model critically benefited our inverse model. We successfully identified novel EES parameters, in under 20 min, capable of producing desired target EMG recruitment duringin vivotesting. Furthermore, we discovered potential functional redundancies within the spinal sensorimotor networks by identifying unique EES parameters that result in similar motor outcomes. Together, these results suggest that our framework is well-suited to probe spinal circuitry and control muscle recruitment in a completely data-driven manner.Significance.We successfully identify novel EES parameters within minutes, capable of producing desired EMG recruitment. Our approach is data-driven, subject-agnostic, automated, and orders of magnitude faster than manual approaches.

Machine-learned Adaptive Switching in Voluntary Lower-limb Exoskeleton Control: Preliminary Results

Lower-limb exoskeletons utilize fixed control strategies and are not adaptable to user's intention. To this end, the goal of this study was to investigate the potential of using temporal-difference learning and general value functions for predicting the next possible walking mode that will be selected by users wearing exoskeletons in order to reduce the effort and cognitive load while switching between different modes of walking. Experiments were performed with a user wearing the Indego exoskeleton and given the authority to switch between five walking modes that were different in terms of speed and turn direction. The user's switching preferences were learned and predicted from device-centric and room-centric measurements by considering similarities in the movements being performed. A switching list was updated to show the most probable future next modes to be selected by the user. In contrast to other approaches that either can only predict a single time-step or require intensive offline training, this work used a computationally inexpensive method for learning and has the potential of providing temporally extended sets of predictions in real-time. Comparing the number of required manual switches between the machine-learned switching list and the best possible static lists showed an average decrease of 42.44% in the required switches for the machine-learned adaptive strategy. These promising results will facilitate the path for real-time application of this technique.

Prediction, Knowledge, and Explainability: Examining the Use of General Value Functions in Machine Knowledge

Within computational reinforcement learning, a growing body of work seeks to express an agent's knowledge of its world through large collections of predictions. While systems that encode predictions as General Value Functions (GVFs) have seen numerous developments in both theory and application, whether such approaches are explainable is unexplored. In this perspective piece, we explore GVFs as a form of explainable AI. To do so, we articulate a subjective agent-centric approach to explainability in sequential decision-making tasks. We propose that prior to explaining its decisions to others, an self-supervised agent must be able to introspectively explain decisions to itself. To clarify this point, we review prior applications of GVFs that involve human-agent collaboration. In doing so, we demonstrate that by making their subjective explanations public, predictive knowledge agents can improve the clarity of their operation in collaborative tasks.

A wireless spinal stimulation system for ventral activation of the rat cervical spinal cord

Electrical stimulation of the cervical spinal cord is gaining traction as a therapy following spinal cord injury; however, it is difficult to target the cervical motor region in a rodent using a non-penetrating stimulus compared with direct placement of intraspinal wire electrodes. Penetrating wire electrodes have been explored in rodent and pig models and, while they have proven beneficial in the injured spinal cord, the negative aspects of spinal parenchymal penetration (e.g., gliosis, neural tissue damage, and obdurate inflammation) are of concern when considering therapeutic potential. We therefore designed a novel approach for epidural stimulation of the rat spinal cord using a wireless stimulation system and ventral electrode array. Our approach allowed for preservation of mobility following surgery and was suitable for long term stimulation strategies in awake, freely functioning animals. Further, electrophysiology mapping of the ventral spinal cord revealed the ventral approach was suitable to target muscle groups of the rat forelimb and, at a single electrode lead position, different stimulation protocols could be applied to achieve unique activation patterns of the muscles of the forelimb.

Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge Architectures

Predictions and predictive knowledge have seen recent success in improving not only robot control but also other applications ranging from industrial process control to rehabilitation. A property that makes these predictive approaches well-suited for robotics is that they can be learned online and incrementally through interaction with the environment. However, a remaining challenge for many prediction-learning approaches is an appropriate choice of prediction-learning parameters, especially parameters that control the magnitude of a learning machine's updates to its predictions (the learning rates or step sizes). Typically, these parameters are chosen based on an extensive parameter search—an approach that neither scales well nor is well-suited for tasks that require changing step sizes due to non-stationarity. To begin to address this challenge, we examine the use of online step-size adaptation using the Modular Prosthetic Limb: a sensor-rich robotic arm intended for use by persons with amputations. Our method of choice, Temporal-Difference Incremental Delta-Bar-Delta (TIDBD), learns and adapts step sizes on a feature level; importantly, TIDBD allows step-size tuning and representation learning to occur at the same time. As a first contribution, we show that TIDBD is a practical alternative for classic Temporal-Difference (TD) learning via an extensive parameter search. Both approaches perform comparably in terms of predicting future aspects of a robotic data stream, but TD only achieves comparable performance with a carefully hand-tuned learning rate, while TIDBD uses a robust meta-parameter and tunes its own learning rates. Secondly, our results show that for this particular application TIDBD allows the system to automatically detect patterns characteristic of sensor failures common to a number of robotic applications. As a third contribution, we investigate the sensitivity of classic TD and TIDBD with respect to the initial step-size values on our robotic data set, reaffirming the robustness of TIDBD as shown in previous papers. Together, these results promise to improve the ability of robotic devices to learn from interactions with their environments in a robust way, providing key capabilities for autonomous agents and robots.