Visuotactile synchrony of stimulation-induced sensation and natural somatosensation

Sensory integration for neuroprostheses: from functional benefits to neural correlates

In the field of sensory neuroprostheses, one ultimate goal is for individuals to perceive artificial somatosensory information and use the prosthesis with high complexity that resembles an intact system. To this end, research has shown that stimulation-elicited somatosensory information improves prosthesis perception and task performance. While studies strive to achieve sensory integration, a crucial phenomenon that entails naturalistic interaction with the environment, this topic has not been commensurately reviewed. Therefore, here we present a perspective for understanding sensory integration in neuroprostheses. First, we review the engineering aspects and functional outcomes in sensory neuroprosthesis studies. In this context, we summarize studies that have suggested sensory integration. We focus on how they have used stimulation-elicited percepts to maximize and improve the reliability of somatosensory information. Next, we review studies that have suggested multisensory integration. These works have demonstrated that congruent and simultaneous multisensory inputs provided cognitive benefits such that an individual experiences a greater sense of authority over prosthesis movements (i.e., agency) and perceives the prosthesis as part of their own (i.e., ownership). Thereafter, we present the theoretical and neuroscience framework of sensory integration. We investigate how behavioral models and neural recordings have been applied in the context of sensory integration. Sensory integration models developed from intact-limb individuals have led the way to sensory neuroprosthesis studies to demonstrate multisensory integration. Neural recordings have been used to show how multisensory inputs are processed across cortical areas. Lastly, we discuss some ongoing research and challenges in achieving and understanding sensory integration in sensory neuroprostheses. Resolving these challenges would help to develop future strategies to improve the sensory feedback of a neuroprosthetic system.

Visual context affects the perceived timing of tactile sensations elicited through intra-cortical microstimulation

Intra-cortical microstimulation (ICMS) is a technique to provide tactile sensations for a somatosensory brain-machine interface (BMI). A viable BMI must function within the rich, multisensory environment of the real world, but how ICMS is integrated with other sensory modalities is poorly understood. To investigate how ICMS percepts are integrated with visual information, ICMS and visual stimuli were delivered at varying times relative to one another. Both visual context and ICMS current amplitude were found to bias the qualitative experience of ICMS. In two tetraplegic participants, ICMS and visual stimuli were more likely to be experienced as occurring simultaneously when visual stimuli were more realistic, demonstrating an effect of visual context on the temporal binding window. The peak of the temporal binding window varied but was consistently offset from zero, suggesting that multisensory integration with ICMS can suffer from temporal misalignment. Recordings from primary somatosensory cortex (S1) during catch trials where visual stimuli were delivered without ICMS demonstrated that S1 represents visual information related to ICMS across visual contexts.

Neuromorphic hardware for somatosensory neuroprostheses

In individuals with sensory-motor impairments, missing limb functions can be restored using neuroprosthetic devices that directly interface with the nervous system. However, restoring the natural tactile experience through electrical neural stimulation requires complex encoding strategies. Indeed, they are presently limited in effectively conveying or restoring tactile sensations by bandwidth constraints. Neuromorphic technology, which mimics the natural behavior of neurons and synapses, holds promise for replicating the encoding of natural touch, potentially informing neurostimulation design. In this perspective, we propose that incorporating neuromorphic technologies into neuroprostheses could be an effective approach for developing more natural human-machine interfaces, potentially leading to advancements in device performance, acceptability, and embeddability. We also highlight ongoing challenges and the required actions to facilitate the future integration of these advanced technologies.

Plantar somatosensory restoration enhances gait, speed perception, and motor adaptation

Lower limb loss is a major insult to the body's nervous and musculoskeletal systems. Despite technological advances in prosthesis design, artificial limbs are not yet integrated into the body's physiological systems. Therefore, lower limb amputees (LLAs) experience lower balance confidence, higher fear of falls, and impaired gait compared with their able-bodied peers (ABs). Previous studies have demonstrated that restored sensations perceived as originating directly from the missing limb via neural interfaces improve balance and performance in certain ambulatory tasks; however, the effects of such evoked sensations on neural circuitries involved in the locomotor activity are not well understood. In this work, we investigated the effects of plantar sensation elicited by peripheral nerve stimulation delivered by multicontact nerve cuff electrodes on gait symmetry and stability, speed perception, and motor adaptation. We found that restored plantar sensation increased stance time and propulsive force on the prosthetic side, improved gait symmetry, and yielded an enhanced perception of prosthetic limb movement. Our results show that the locomotor adaptation among LLAs with plantar sensation became similar to that of ABs. These findings suggest that our peripheral nerve-based approach to elicit plantar sensation directly affects central nervous pathways involved in locomotion and motor adaptation during walking. Our neuroprosthesis provided a unique model to investigate the role of somatosensation in the lower limb during walking and its effects on perceptual recalibration after a locomotor adaptation task. Furthermore, we demonstrated how plantar sensation in LLAs could effectively increase mobility, improve walking dynamics, and possibly reduce fall risks.

A framework for understanding prosthetic embodiment for the plastic surgeon

Plastic surgeons play a critical role in the management of amputations and are uniquely positioned to improve the lives and functional abilities of patients with limb loss. The embodiment of a prosthesis describes how effectively it replaces a missing limb and is an important aspect of patient care. Despite its importance, the current prosthetics literature lacks a formal definition of embodiment, and descriptions are often vague or incomplete. In this narrative review, we assess the current literature on prosthetic embodiment to explore the main mechanisms of embodiment and how each allows a prosthesis to interface with the human body. In doing so, we provide a comprehensive, holistic framework for understanding this concept.

Olfactory-gustatory simultaneity judgments: A preliminary study on the congruency-dependent temporal window of multisensory binding

BACKGROUND

A greater congruency of audio and video expands the temporal binding window (TBW). A similar phenomenon may occur with a combination of odor and taste, which are the main components of flavor.

OBJECTIVE

TBW is defined as the temporal resolution of synchrony perception. The larger the TBW, the lower the resolution. We hypothesized that the more congruent the odor and taste, the lower the temporal resolution of synchrony perception.

METHODS

To examine this hypothesis, 10 female participants performed simultaneity judgment (SJ) tasks under congruent (soy sauce odor with saline) and incongruent (cherry tree leaf odor with saline) conditions and evaluated the congruency with saltiness for the two odors. In the SJ tasks, participants reported whether odor and taste were presented simultaneously or successively. We assumed a Gaussian distribution for the temporal distributions of the simultaneous response rates and calculated the approximations. In addition, we computed the half width at half height (HWHH) as an index of TBW based on the coefficient of approximation for the temporal distribution of the simultaneous response rates.

RESULTS

HWHH was significantly larger under congruent condition than under incongruent condition. In addition, congruency with saltiness had a significant moderate positive correlation with HWHH.

CONCLUSION

The larger the HWHH, the lower the temporal resolution of synchrony perception, supporting the hypothesis. This study suggests that the width of TBW depends on the cross-modal congruency similar to the case for audio-visual SJs. However, methodological improvements, including a larger sample size and gender-independent recruitment of participants, are essential to enhance the reliability of the findings because some of the results did not provide sufficient ESs or power.

Peripheral neurostimulation for encoding artificial somatosensations

The direct neural stimulation of peripheral or central nervous systems has been shown as an effective tool to treat neurological conditions. The electrical activation of the nervous sensory pathway can be adopted to restore the artificial sense of touch and proprioception in people suffering from sensory-motor disorders. The modulation of the neural stimulation parameters has a direct effect on the electrically induced sensations, both when targeting the somatosensory cortex and the peripheral somatic nerves. The properties of the artificial sensations perceived, as their location, quality and intensity are strongly dependent on the direct modulation of pulse width, amplitude and frequency of the neural stimulation. Different sensory encoding schemes have been tested in patients showing distinct effects and outcomes according to their impact on the neural activation. Here, I reported the most adopted neural stimulation strategies to artificially encode somatosensation into the peripheral nervous system. The real-time implementation of these strategies in bionic devices is crucial to exploit the artificial sensory feedback in prosthetics. Thus, neural stimulation becomes a tool to directly communicate with the human nervous system. Given the importance of adding artificial sensory information to neuroprosthetic devices to improve their control and functionality, the choice of an optimal neural stimulation paradigm could increase the impact of prosthetic devices on the quality of life of people with sensorimotor disabilities.

Artificial referred sensation in upper and lower limb prosthesis users: a systematic review

Objective.Electrical stimulation can induce sensation in the phantom limb of individuals with amputation. It is difficult to generalize existing findings as there are many approaches to delivering stimulation and to assessing the characteristics and benefits of sensation. Therefore, the goal of this systematic review was to explore the stimulation parameters that effectively elicited referred sensation, the qualities of elicited sensation, and how the utility of referred sensation was assessed.Approach.We searched PubMed, Web of Science, and Engineering Village through January of 2022 to identify relevant papers. We included papers which electrically induced referred sensation in individuals with limb loss and excluded papers that did not contain stimulation parameters or outcome measures pertaining to stimulation. We extracted information on participant demographics, stimulation approaches, and participant outcomes.Main results.After applying exclusion criteria, 49 papers were included covering nine stimulation methods. Amplitude was the most commonly adjusted parameter (n= 25), followed by frequency (n= 22), and pulse width (n= 15). Of the 63 reports of sensation quality, most reported feelings of pressure (n= 52), paresthesia (n= 48), or vibration (n= 40) while less than half (n= 29) reported a sense of position or movement. Most papers evaluated the functional benefits of sensation (n= 33) using force matching or object identification tasks, while fewer papers quantified subjective measures (n= 16) such as pain or embodiment. Only 15 studies (36%) observed percept intensity, quality, or location over multiple sessions.Significance.Most studies that measured functional performance demonstrated some benefit to providing participants with sensory feedback. However, few studies could experimentally manipulate sensation location or quality. Direct comparisons between studies were limited by variability in methodologies and outcome measures. As such, we offer recommendations to aid in more standardized reporting for future research.

Multisensory stimulation decreases phantom limb distortions and is optimally integrated

The multisensory integration of signals from different senses is crucial to develop an unambiguous percept of the environment and our body. Losing a limb causes drastic changes in the body, sometimes causing pain and distorted phantom limb perception. Despite the debate over why these phenomena arise, some researchers suggested that they might be linked to an impairment of multisensory signals inflow and integration. Therefore, reestablishing optimally integrated sensory feedback could be crucial. The related benefits on sensory performance and body self-representation are still to be demonstrated, particularly in lower-limb amputees. We present a multisensory framework combining Virtual reality and electro-cutaneous stimulation that allows the optimal integration of visuo-tactile stimuli in lower-limb amputees even if nonspatially matching. We also showed that this multisensory stimulation allowed faster sensory processing, higher embodiment, and reductions in phantom limb distortions. Our findings support the development of multisensory rehabilitation approaches, restoring a correct body representation.

•

Multisensory platform combining virtual reality and electro-cutaneous stimulation.

•

Leg amputees optimally integrate nonspatially matching visuo-tactile stimulation.

•

Multisensory stimulation allows faster information processing and higher embodiment.

•

Phantom limb distortions are reduced after multisensory stimulation.

Prosthetic embodiment: systematic review on definitions, measures, and experimental paradigms

The term embodiment has become omnipresent within prosthetics research and is often used as a metric of the progress made in prosthetic technologies, as well as a hallmark for user acceptance. However, despite the frequent use of the term, the concept of prosthetic embodiment is often left undefined or described incongruently, sometimes even within the same article. This terminological ambiguity complicates the comparison of studies using embodiment as a metric of success, which in turn hinders the advancement of prosthetics research. To resolve these terminological ambiguities, we systematically reviewed the used definitions of embodiment in the prosthetics literature. We performed a thematic analysis of the definitions and found that embodiment is often conceptualized in either of two frameworks based on body representations or experimental phenomenology. We concluded that treating prosthetic embodiment within an experimental phenomenological framework as the combination of ownership and agency allows for embodiment to be a quantifiable metric for use in translational research. To provide a common reference and guidance on how to best assess ownership and agency, we conducted a second systematic review, analyzing experiments and measures involving ownership and agency. Together, we highlight a pragmatic definition of prosthetic embodiment as the combination of ownership and agency, and in an accompanying article, we provide a perspective on a multi-dimensional framework for prosthetic embodiment. Here, we concluded by providing recommendations on metrics that allow for outcome comparisons between studies, thereby creating a common reference for further discussions within prosthetics research.

Bidirectional bionic limbs: a perspective bridging technology and physiology

Precise control of bionic limbs relies on robust decoding of motor commands from nerves or muscles signals and sensory feedback from artificial limbs to the nervous system by interfacing the afferent nerve pathways. Implantable devices for bidirectional communication with bionic limbs have been developed in parallel with research on physiological alterations caused by an amputation. In this perspective article, we question whether increasing our effort on bridging these technologies with a deeper understanding of amputation pathophysiology and human motor control may help to overcome pressing stalls in the next generation of bionic limbs.

Embedded Electrotactile Feedback System for Hand Prostheses Using Matrix Electrode and Electronic Skin

As the technology moves towards more human-like bionic limbs, it is necessary to develop a feedback system that provides active touch feedback to a user of a prosthetic hand. Most of the contemporary sensory substitution methods comprise simple position and force sensors combined with few discrete stimulation units, and hence they are characterized with a limited amount of information that can be transmitted by the feedback. The present study describes a novel system for tactile feedback integrating advanced multipoint sensing (electronic skin) and stimulation (matrix electrodes). The system comprises a flexible sensing array (16 sensors) integrated on the index finger of a Michelangelo prosthetic hand mockup, embedded interface electronics and multichannel stimulator connected to a flexible matrix electrode (24 pads). The developed system conveys contact information (binary detections) to the user. To demonstrate the feasibility, the system was tested in six able-bodied subjects who were asked to recognize static patterns (contact position) with two different spatial resolutions and dynamic movement patterns (i.e., sliding along and/or across the finger) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into electrotactile profiles, which the subjects could recognize with good performance. The success rates (mean ± standard deviation) for the static patterns were 91 ± 4% and 58 ± 10% for low and high spatial resolution, respectively, while the success rate for sliding touch was 94 ± 4%. These results demonstrate that the developed system is an important step towards a new generation of tactile feedback interfaces that can provide high-bandwidth connection between the user and his/her bionic limb. Such systems would allow mimicking spatially distributed natural feedback, thereby facilitating the control and embodiment of the artificial device into the user body scheme.

Lower-Limb Amputees Adjust Quiet Stance in Response to Manipulations of Plantar Sensation

Interfering with or temporarily eliminating foot-sole tactile sensations causes postural adjustments. Furthermore, individuals with impaired or missing foot-sole sensation, such as lower-limb amputees, exhibit greater postural instability than those with intact sensation. Our group has developed a method of providing tactile feedback sensations projected to the missing foot of lower-limb amputees via electrical peripheral nerve stimulation (PNS) using implanted nerve cuff electrodes. As a step toward effective implementation of the system in rehabilitation and everyday use, we compared postural adjustments made in response to tactile sensations on the missing foot elicited by our system, vibration on the intact foot-sole, and a control condition in which no additional sensory input was applied. Three transtibial amputees with at least a year of experience with tactile sensations provided by our PNS system participated in the study. Participants stood quietly with their eyes closed on their everyday prosthesis while electrically elicited, vibratory, or no additional sensory input was administered for 20 s. Early and steady-state postural adjustments were quantified by center of pressure location, path length, and average angle over the course of each trial. Electrically elicited tactile sensations and vibration both caused shifts in center of pressure location compared to the control condition. Initial (first 3 s) shifts in center of pressure location with electrically elicited or vibratory sensory inputs often differed from shifts measured over the full 20 s trial. Over the full trial, participants generally shifted toward the foot receiving additional sensory input, regardless of stimulation type. Similarities between responses to electrically elicited tactile sensations projected to the missing foot and responses to vibration in analogous regions on the intact foot suggest that the motor control system treats electrically elicited tactile inputs similarly to native tactile inputs. The ability of electrically elicited tactile inputs to cause postural adjustments suggests that these inputs are incorporated into sensorimotor control, despite arising from artificial nerve stimulation. These results are encouraging for application of neural stimulation in restoring missing sensory feedback after limb loss and suggest PNS could provide an alternate method to perturb foot-sole tactile information for investigating integration of tactile feedback with other sensory modalities.

Perceptual simultaneity and its modulation during EMG-triggered motion induction with electrical muscle stimulation

When human movement is assisted or controlled with a muscle actuator, such as electrical muscle stimulation, a critical issue is the integration of such induced movement with the person's motion intention and how this movement then affects their motor control. Towards achieving optimal integration and reducing feelings of artificiality and enforcement, we explored perceptual simultaneity through electrical muscle stimulation, which involved changing the interval between intentional and induced movements. We report on two experiments in which we evaluated the ranges between detection and stimulus for perceptual simultaneity achievable with an electromyography-triggered electrical muscle stimulation system. We found that the peak range was approximately 80-160 ms, with the timing of perceptual simultaneity shifting according to different adaptation states. Our results indicate that perceptual simultaneity is controllable using this adaptation strategy.

Ambulatory searching task reveals importance of somatosensation for lower-limb amputees

The contribution of somatosensation to locomotor deficits in below-knee amputees (BKAs) has not been fully explored. Unilateral disruption of plantar sensation causes able-bodied individuals to adopt locomotor characteristics that resemble those of unilateral BKAs, suggesting that restoring somatosensation may improve locomotion for amputees. In prior studies, we demonstrated that electrically stimulating the residual nerves of amputees elicited somatosensory percepts that were felt as occurring in the missing foot. Subsequently, we developed a sensory neuroprosthesis that modulated stimulation-evoked sensation in response to interactions between the prosthesis and the environment. To characterize the impact of the sensory neuroprosthesis on locomotion, we created a novel ambulatory searching task. The task involved walking on a horizontal ladder while blindfolded, which engaged plantar sensation while minimizing visual compensation. We first compared the performance of six BKAs to 14 able-bodied controls. Able-bodied individuals demonstrated higher foot placement accuracy than BKAs, indicating that the ladder test was sensitive enough to detect locomotor deficits. When three of the original six BKAs used the sensory neuroprosthesis, the tradeoff between speed and accuracy significantly improved for two of them. This study advanced our understanding of how cutaneous plantar sensation can be used to acquire action-related information during challenging locomotor tasks.

Sensory neuroprosthesis improves postural stability during Sensory Organization Test in lower-limb amputees

To maintain postural stability, unilateral lower-limb amputees (LLAs) heavily rely on visual and vestibular inputs, and somatosensory cues from their intact leg to compensate for missing somatosensory information from the amputated limb. When any of these resources are compromised, LLAs exhibit poor balance control compared to able-bodied individuals. We hypothesized that restoring somatosensation related to the missing limb via direct activation of the sensory nerves in the residuum would improve the standing stability of LLAs. We developed a closed-loop sensory neuroprosthesis utilizing non-penetrating multi-contact cuff electrodes implanted around the residual nerves to elicit perceptions of the location and intensity of plantar pressures under the prosthetic feet of two transtibial amputees. Effects of the sensory neuroprosthesis on balance were quantified with the Sensory Organization Test and other posturographic measures of sway. In both participants, the sensory neuroprosthesis improved equilibrium and sway when somatosensation from the intact leg and visual inputs were perturbed simultaneously. One participant also showed improvement with the sensory neuroprosthesis whenever somatosensation in the intact leg was compromised via perturbations of the platform. These observations suggest the sensory feedback elicited by neural stimulation can significantly improve the standing stability of LLAs, particularly when other sensory inputs are depleted or otherwise compromised.