Multisensory Integration in Bionics: Relevance and Perspectives

Sensory integration for neuroprostheses: from functional benefits to neural correlates

In the field of sensory neuroprostheses, one ultimate goal is for individuals to perceive artificial somatosensory information and use the prosthesis with high complexity that resembles an intact system. To this end, research has shown that stimulation-elicited somatosensory information improves prosthesis perception and task performance. While studies strive to achieve sensory integration, a crucial phenomenon that entails naturalistic interaction with the environment, this topic has not been commensurately reviewed. Therefore, here we present a perspective for understanding sensory integration in neuroprostheses. First, we review the engineering aspects and functional outcomes in sensory neuroprosthesis studies. In this context, we summarize studies that have suggested sensory integration. We focus on how they have used stimulation-elicited percepts to maximize and improve the reliability of somatosensory information. Next, we review studies that have suggested multisensory integration. These works have demonstrated that congruent and simultaneous multisensory inputs provided cognitive benefits such that an individual experiences a greater sense of authority over prosthesis movements (i.e., agency) and perceives the prosthesis as part of their own (i.e., ownership). Thereafter, we present the theoretical and neuroscience framework of sensory integration. We investigate how behavioral models and neural recordings have been applied in the context of sensory integration. Sensory integration models developed from intact-limb individuals have led the way to sensory neuroprosthesis studies to demonstrate multisensory integration. Neural recordings have been used to show how multisensory inputs are processed across cortical areas. Lastly, we discuss some ongoing research and challenges in achieving and understanding sensory integration in sensory neuroprostheses. Resolving these challenges would help to develop future strategies to improve the sensory feedback of a neuroprosthetic system.

Visual context affects the perceived timing of tactile sensations elicited through intra-cortical microstimulation

Intra-cortical microstimulation (ICMS) is a technique to provide tactile sensations for a somatosensory brain-machine interface (BMI). A viable BMI must function within the rich, multisensory environment of the real world, but how ICMS is integrated with other sensory modalities is poorly understood. To investigate how ICMS percepts are integrated with visual information, ICMS and visual stimuli were delivered at varying times relative to one another. Both visual context and ICMS current amplitude were found to bias the qualitative experience of ICMS. In two tetraplegic participants, ICMS and visual stimuli were more likely to be experienced as occurring simultaneously when visual stimuli were more realistic, demonstrating an effect of visual context on the temporal binding window. The peak of the temporal binding window varied but was consistently offset from zero, suggesting that multisensory integration with ICMS can suffer from temporal misalignment. Recordings from primary somatosensory cortex (S1) during catch trials where visual stimuli were delivered without ICMS demonstrated that S1 represents visual information related to ICMS across visual contexts.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Wearable High Voltage Compliant Current Stimulator for Restoring Sensory Feedback

Transcutaneous Electrical Nerve Stimulation (TENS) is a promising technique for eliciting referred tactile sensations in patients with limb amputation. Although several studies show the validity of this technique, its application in daily life and away from laboratories is limited by the need for more portable instrumentation that guarantees the necessary voltage and current requirements for proper sensory stimulation. This study proposes a low-cost, wearable high-voltage compliant current stimulator with four independent channels based on Components-Off-The-Shelf (COTS). This microcontroller-based system implements a voltage-current converter controllable through a digital-to-analog converter that delivers up to 25 mA to load up to 3.6 kΩ. The high-voltage compliance enables the system to adapt to variations in electrode-skin impedance, allowing it to stimulate loads over 10 kΩ with currents of 5 mA. The system was realized on a four-layer PCB (115.9 mm × 61 mm, 52 g). The functionality of the device was tested on resistive loads and on an equivalent skin-like RC circuit. Moreover, the possibility of implementing an amplitude modulation was demonstrated.

Peripheral neurostimulation for encoding artificial somatosensations

The direct neural stimulation of peripheral or central nervous systems has been shown as an effective tool to treat neurological conditions. The electrical activation of the nervous sensory pathway can be adopted to restore the artificial sense of touch and proprioception in people suffering from sensory-motor disorders. The modulation of the neural stimulation parameters has a direct effect on the electrically induced sensations, both when targeting the somatosensory cortex and the peripheral somatic nerves. The properties of the artificial sensations perceived, as their location, quality and intensity are strongly dependent on the direct modulation of pulse width, amplitude and frequency of the neural stimulation. Different sensory encoding schemes have been tested in patients showing distinct effects and outcomes according to their impact on the neural activation. Here, I reported the most adopted neural stimulation strategies to artificially encode somatosensation into the peripheral nervous system. The real-time implementation of these strategies in bionic devices is crucial to exploit the artificial sensory feedback in prosthetics. Thus, neural stimulation becomes a tool to directly communicate with the human nervous system. Given the importance of adding artificial sensory information to neuroprosthetic devices to improve their control and functionality, the choice of an optimal neural stimulation paradigm could increase the impact of prosthetic devices on the quality of life of people with sensorimotor disabilities.

Assess and rehabilitate body representations via (neuro)robotics: An emergent perspective

The perceptions of our own body (e.g., size and shape) do not always coincide with its real characteristics (e.g., dimension). To track the complexity of our perception, the concept of mental representations (model) of the body has been conceived. Body representations (BRs) are stored in the brain and are maintained and updated through multiple sensory information. Despite being altered in different clinical conditions and being tightly linked with self-consciousness, which is one of the most astonishing features of the human mind, the BRs and, especially, the underlying mechanisms and functions are still unclear. In this vein, here we suggest that (neuro)robotics can make an important contribution to the study of BRs. The first section of the study highlights the potential impact of robotics devices in investigating BRs. Far to be exhaustive, we illustrate major examples of its possible exploitation to further improve the assessment of motor, haptic, and multisensory information building up the BRs. In the second section, we review the main evidence showing the contribution of neurorobotics-based (multi)sensory stimulation in reducing BRs distortions in various clinical conditions (e.g., stroke, amputees). The present study illustrates an emergent multidisciplinary perspective combining the neuroscience of BRs and (neuro)robotics to understand and modulate the perception and experience of one's own body. We suggest that (neuro)robotics can enhance the study of BRs by improving experimental rigor and introducing new experimental conditions. Furthermore, it might pave the way for the rehabilitation of altered body perceptions.