Biomimetic multi-channel microstimulation of somatosensory cortex conveys high resolution force feedback for bionic hands

Sensory integration for neuroprostheses: from functional benefits to neural correlates

In the field of sensory neuroprostheses, one ultimate goal is for individuals to perceive artificial somatosensory information and use the prosthesis with high complexity that resembles an intact system. To this end, research has shown that stimulation-elicited somatosensory information improves prosthesis perception and task performance. While studies strive to achieve sensory integration, a crucial phenomenon that entails naturalistic interaction with the environment, this topic has not been commensurately reviewed. Therefore, here we present a perspective for understanding sensory integration in neuroprostheses. First, we review the engineering aspects and functional outcomes in sensory neuroprosthesis studies. In this context, we summarize studies that have suggested sensory integration. We focus on how they have used stimulation-elicited percepts to maximize and improve the reliability of somatosensory information. Next, we review studies that have suggested multisensory integration. These works have demonstrated that congruent and simultaneous multisensory inputs provided cognitive benefits such that an individual experiences a greater sense of authority over prosthesis movements (i.e., agency) and perceives the prosthesis as part of their own (i.e., ownership). Thereafter, we present the theoretical and neuroscience framework of sensory integration. We investigate how behavioral models and neural recordings have been applied in the context of sensory integration. Sensory integration models developed from intact-limb individuals have led the way to sensory neuroprosthesis studies to demonstrate multisensory integration. Neural recordings have been used to show how multisensory inputs are processed across cortical areas. Lastly, we discuss some ongoing research and challenges in achieving and understanding sensory integration in sensory neuroprostheses. Resolving these challenges would help to develop future strategies to improve the sensory feedback of a neuroprosthetic system.

A roadmap for implanting microelectrode arrays to evoke tactile sensations through intracortical microstimulation

Intracortical microstimulation (ICMS) is a method for restoring sensation to people with paralysis as part of a bidirectional brain-computer interface to restore upper limb function. Evoking tactile sensations of the hand through ICMS requires precise targeting of implanted electrodes. Here we describe the presurgical imaging procedures used to generate functional maps of the hand area of the somatosensory cortex and subsequent planning that guided the implantation of intracortical microelectrode arrays. In five participants with cervical spinal cord injury, across two study locations, this procedure successfully enabled ICMS-evoked sensations localized to at least the first four digits of the hand. The imaging and planning procedures developed through this clinical trial provide a roadmap for other brain-computer interface studies to ensure successful placement of stimulation electrodes.

Intracortical microstimulation of human somatosensory cortex is sufficient to induce perceptual biases

Time-order error, a psychophysical phenomenon in which the duration in between successive stimuli alters perception, has been studied for decades by neuroscientists and psychologists. To date, however, the locus of these effects is unknown. We use intracortical microstimulation of somatosensory cortex in humans as a tool to bypass initial stages of processing and restrict the possible locations that signals could be modified. We find that, using both amplitude discrimination and magnitude estimation paradigms, intracortical microstimulation reliably evoked time-order errors across all participants. Points of subjective equality were symmetrically shifted during amplitude discrimination experiments and the intensity of a successive stimulus was perceived as being more intense when compared to single stimulus trials in magnitude estimation experiments. The error was reduced for both paradigms at longer inter-stimulus intervals. These results show that direct activation of primary somatosensory cortex is sufficient to induce time-order errors.

Activation and depression of neural and hemodynamic responses induced by the intracortical microstimulation and visual stimulation in the mouse visual cortex

Objective. Intracortical microstimulation (ICMS) can be an effective method for restoring sensory perception in contemporary brain-machine interfaces. However, the mechanisms underlying better control of neuronal responses remain poorly understood, as well as the relationship between neuronal activity and other concomitant phenomena occurring around the stimulation site.Approach. Different microstimulation frequencies were investigatedin vivoon Thy1-GCaMP6s mice using widefield and two-photon imaging to evaluate the evoked excitatory neural responses across multiple spatial scales as well as the induced hemodynamic responses. Specifically, we quantified stimulation-induced neuronal activation and depression in the mouse visual cortex and measured hemodynamic oxyhemoglobin and deoxyhemoglobin signals using mesoscopic-scale widefield imaging.Main results. Our calcium imaging findings revealed a preference for lower-frequency stimulation in driving stronger neuronal activation. A depressive response following the neural activation preferred a slightly higher frequency stimulation compared to the activation. Hemodynamic signals exhibited a comparable spatial spread to neural calcium signals. Oxyhemoglobin concentration around the stimulation site remained elevated during the post-activation (depression) period. Somatic and neuropil calcium responses measured by two-photon microscopy showed similar dependence on stimulation parameters, although the magnitudes measured in soma was greater than in neuropil. Furthermore, higher-frequency stimulation induced a more pronounced activation in soma compared to neuropil, while depression was predominantly induced in soma irrespective of stimulation frequencies.Significance. These results suggest that the mechanism underlying depression differs from activation, requiring ample oxygen supply, and affecting neurons. Our findings provide a novel understanding of evoked excitatory neuronal activity induced by ICMS and offer insights into neuro-devices that utilize both activation and depression phenomena to achieve desired neural responses.

Galvanic vs. pulsatile effects on decision-making networks: reshaping the neural activation landscape

Objective. Primarily due to safety concerns, biphasic pulsatile stimulation (PS) is the present standard for electrical excitation of neural tissue with a diverse set of applications. While pulses have been shown to be effective to achieve functional outcomes, they have well-known deficits. Due to recent technical advances, galvanic stimulation (GS), delivery of current for extended periods of time (>1 s), has re-emerged as an alternative to PS.Approach. In this paper, we use a winner-take-all decision-making cortical network model to investigate differences between pulsatile and GS in the context of a perceptual decision-making task.Main results. Based on previous work, we hypothesized that GS would produce more spatiotemporally distributed, network-sensitive neural responses, while PS would produce highly synchronized activation of a limited group of neurons. Our results in-silico support these hypotheses for low-amplitude GS but deviate when galvanic amplitudes are large enough to directly activate or block nearby neurons.Significance. We conclude that with careful parametrization, GS could overcome some limitations of PS to deliver more naturalistic firing patterns in the group of targeted neurons.

Biomimetic computer-to-brain communication enhancing naturalistic touch sensations via peripheral nerve stimulation

Artificial communication with the brain through peripheral nerve stimulation shows promising results in individuals with sensorimotor deficits. However, these efforts lack an intuitive and natural sensory experience. In this study, we design and test a biomimetic neurostimulation framework inspired by nature, capable of "writing" physiologically plausible information back into the peripheral nervous system. Starting from an in-silico model of mechanoreceptors, we develop biomimetic stimulation policies. We then experimentally assess them alongside mechanical touch and common linear neuromodulations. Neural responses resulting from biomimetic neuromodulation are consistently transmitted towards dorsal root ganglion and spinal cord of cats, and their spatio-temporal neural dynamics resemble those naturally induced. We implement these paradigms within the bionic device and test it with patients (ClinicalTrials.gov identifier NCT03350061). He we report that biomimetic neurostimulation improves mobility (primary outcome) and reduces mental effort (secondary outcome) compared to traditional approaches. The outcomes of this neuroscience-driven technology, inspired by the human body, may serve as a model for advancing assistive neurotechnologies.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Somatosensory cortex microstimulation modulates primary motor and ventral premotor cortex neurons with extensive spatial convergence and divergence

Intracortical microstimulation (ICMS) is known to affect distant neurons transynaptically, yet the extent to which ICMS pulses delivered in one cortical area modulate neurons in other cortical areas remains largely unknown. Here we assessed how the individual pulses of multi-channel ICMS trains delivered in the upper extremity representation of the macaque primary somatosensory area (S1) modulate neuron firing in the primary motor cortex (M1) and in the ventral premotor cortex (PMv). S1-ICMS pulses modulated the majority of units recorded both in the M1 upper extremity representation and in PMv, producing more inhibition than excitation. Effects converged on individual neurons in both M1 and PMv from extensive S1 territories. Conversely, effects of ICMS delivered in a small region of S1 diverged to wide territories in both M1 and PMv. The effects of this direct modulation of M1 and PMv neurons produced by multi-electrode S1-ICMS like that used here may need to be taken into account by bidirectional brain-computer interfaces that decode intended movements from neural activity in these cortical motor areas.

Significance Statement

Although ICMS is known to produce effects transynaptically, relatively little is known about how ICMS in one cortical area affects neurons in other cortical areas. We show that the effects of multi-channel ICMS in a small patch of S1 diverge to affect neurons distributed widely in both M1 and PMv, and conversely, individual neurons in each of these areas can be affected by ICMS converging from much of the S1 upper extremity representation. Such direct effects of ICMS may complicate the decoding of motor intent from M1 or PMv when artificial sensation is delivered via S1-ICMS in bidirectional brain-computer interfaces.

Microstimulation of human somatosensory cortex evokes task-dependent, spatially patterned responses in motor cortex

The primary motor (M1) and somatosensory (S1) cortices play critical roles in motor control but the signaling between these structures is poorly understood. To fill this gap, we recorded - in three participants in an ongoing human clinical trial (NCT01894802) for people with paralyzed hands - the responses evoked in the hand and arm representations of M1 during intracortical microstimulation (ICMS) in the hand representation of S1. We found that ICMS of S1 activated some M1 neurons at short, fixed latencies consistent with monosynaptic activation. Additionally, most of the ICMS-evoked responses in M1 were more variable in time, suggesting indirect effects of stimulation. The spatial pattern of M1 activation varied systematically: S1 electrodes that elicited percepts in a finger preferentially activated M1 neurons excited during that finger's movement. Moreover, the indirect effects of S1 ICMS on M1 were context dependent, such that the magnitude and even sign relative to baseline varied across tasks. We tested the implications of these effects for brain-control of a virtual hand, in which ICMS conveyed tactile feedback. While ICMS-evoked activation of M1 disrupted decoder performance, this disruption was minimized using biomimetic stimulation, which emphasizes contact transients at the onset and offset of grasp, and reduces sustained stimulation.

Tessellation of artificial touch via microstimulation of human somatosensory cortex

When we interact with objects, we rely on signals from the hand that convey information about the object and our interaction with it. A basic feature of these interactions, the locations of contacts between the hand and object, is often only available via the sense of touch. Information about locations of contact between a brain-controlled bionic hand and an object can be signaled via intracortical microstimulation (ICMS) of somatosensory cortex (S1), which evokes touch sensations that are localized to a specific patch of skin. To provide intuitive location information, tactile sensors on the robotic hand drive ICMS through electrodes that evoke sensations at skin locations matching sensor locations. This approach requires that ICMS-evoked sensations be focal, stable, and distributed over the hand. To systematically investigate the localization of ICMS-evoked sensations, we analyzed the projected fields (PFs) of ICMS-evoked sensations - their location and spatial extent - from reports obtained over multiple years from three participants implanted with microelectrode arrays in S1. First, we found that PFs vary widely in their size across electrodes, are highly stable within electrode, are distributed over large swaths of each participant's hand, and increase in size as the amplitude or frequency of ICMS increases. Second, while PF locations match the locations of the receptive fields (RFs) of the neurons near the stimulating electrode, PFs tend to be subsumed by the corresponding RFs. Third, multi-channel stimulation gives rise to a PF that reflects the conjunction of the PFs of the component channels. By stimulating through electrodes with largely overlapping PFs, then, we can evoke a sensation that is experienced primarily at the intersection of the component PFs. To assess the functional consequence of this phenomenon, we implemented multichannel ICMS-based feedback in a bionic hand and demonstrated that the resulting sensations are more localizable than are those evoked via single-channel ICMS.