A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Reinforcement Learning May Demystify the Limited Human Motor Learning Efficacy Due to Visual-Proprioceptive Mismatch

Vision and proprioception have fundamental sensory mismatches in delivering locational information, and such mismatches are critical factors limiting the efficacy of motor learning. However, it is still not clear how and to what extent this mismatch limits motor learning outcomes. To further the understanding of the effect of sensory mismatch on motor learning outcomes, a reinforcement learning algorithm and the simplified biomechanical elbow joint model were employed to mimic the motor learning process in a computational environment. By applying a reinforcement learning algorithm to the motor learning of elbow joint flexion task, simulation results successfully explained how visual-proprioceptive mismatch limits motor learning outcomes in terms of motor control accuracy and task completion speed. The larger the perceived angular offset between the two sensory modalities, the lower the motor control accuracy. Also, the more similar the peak reward amplitude of the two sensory modalities, the lower the motor control accuracy. In addition, simulation results suggest that insufficient exploration rate limits task completion speed, and excessive exploration rate limits motor control accuracy. Such a speed-accuracy trade-off shows that a moderate exploration rate could serve as another important factor in motor learning.

Improving electrotactile communication with a multi-pad electrode under cognitive load

BACKGROUND

Electrotactile systems are compact interfaces that can be used to convey information through the skin by producing a range of haptic sensations. In many applications, however, the user needs to perceive and interpret haptic stimulation while being engaged in parallel activities. Developing methods that ensure reliable recognition of electrotactile messages despite additional cognitive load is, therefore, an important step for the practical application of electrotactile displays.

METHODS

This study investigated if a simple strategy of repeating electrotactile messages can improve message identification during multitasking. Ten participants identified 36 spatiotemporal electrotactile messages delivered through a 3 × 2 pad-matrix electrode placed on the torso while performing a concomitant cognitive task in three conditions: the messages were presented once (No-REP), and each message was repeated three (REP3) and five (REP5) times. The main outcome measure was the success rate (SR) of message identification.

RESULTS

During multitasking, in the No-REP condition, the SR (median (IQR)) dropped to 56.25% (22.62%), demonstrating that the cognitive task decreased performance. However, the SR significantly improved with message repetitions, reaching 72.92% (21.87%) and 81.25% (18.66%) in REP3 and REP5 conditions respectively, without a statistically significant difference between REP3 and REP5.

CONCLUSIONS

Multitasking affected the efficacy of haptic communication, but message repetition was shown to be an effective strategy for improving performance. Additionally, only three repetitions were enough, as an additional increase in the duration of message transmission (5 repetitions) did not lead to further improvement. This study is an important step toward delivering electrotactile communication that can cope with the demands of real-world applications.

Excitation of natural spinal reflex loops in the sensory-motor control of hand prostheses

Sensory feedback for prosthesis control is typically based on encoding sensory information in specific types of sensory stimuli that the users interpret to adjust the control of the prosthesis. However, in physiological conditions, the afferent feedback received from peripheral nerves is not only processed consciously but also modulates spinal reflex loops that contribute to the neural information driving muscles. Spinal pathways are relevant for sensory-motor integration, but they are commonly not leveraged for prosthesis control. We propose an approach to improve sensory-motor integration for prosthesis control based on modulating the excitability of spinal circuits through the vibration of tendons in a closed loop with muscle activity. We measured muscle signals in healthy participants and amputees during different motor tasks, and we closed the loop by applying vibration on tendons connected to the muscles, which modulated the excitability of motor neurons. The control signals to the prosthesis were thus the combination of voluntary control and additional spinal reflex inputs induced by tendon vibration. Results showed that closed-loop tendon vibration was able to modulate the neural drive to the muscles. When closed-loop tendon vibration was used, participants could achieve similar or better control performance in interfaces using muscle activation than without stimulation. Stimulation could even improve prosthetic grasping in amputees. Overall, our results indicate that closed-loop tendon vibration can integrate spinal reflex pathways in the myocontrol system and open the possibility of incorporating natural feedback loops in prosthesis control.

Context-informed incremental learning improves both the performance and resilience of myoelectric control

Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.

Sensory Symptoms Across the Lifespan in People With Cerebral Palsy

BACKGROUND

To estimate the prevalence of sensory symptoms in people with cerebral palsy (CP) across the lifespan.

METHODS

In this cross-sectional study, the self-reported Sensory Processing Scale Inventory (SPS-I) was administered via Research Electronic Data Capture (REDCap) between February 1, 2022, and August 15, 2022, to people with CP or their caregivers enrolled in the online MyCP Community Registry. We determined the association between SPS-I scores and age (Pearson correlation) and functional status as assessed using five validated functional classification systems for CP (analysis of variance [ANOVA]). We hypothesized that sensory symptoms would differ between younger and older individuals with CP.

RESULTS

Of 155 responses (28% response rate, age one to 76 years, 34% male), 97% reported at least one bothersome sensory symptom. Total sensory symptoms decreased with age (R2 = 0.12, P < 0.0001), driven by decreases in hyposensitivity symptoms (R2 = 0.32, P < 0.0001), primarily tactile hyposensitivity (R2 = 0.29, P < 0.0001). Sensory symptoms increased with greater functional impairment across all functional domains (ANOVA, P < 0.0001). However, the age-specific decrease in hyposensitivities was most pronounced in people with the greatest gross motor functional impairment (R2 = 0.70, P = 0.0004).

CONCLUSION

Our findings suggest that hyposensitivity, primarily tactile sensitivity, decreases with age in people with CP. Future work should assess whether decreased hyposensitivity contributes to other age-related changes in CP like increased pain.

Experimental evaluation of the impact of sEMG interfaces in enhancing embodiment of virtual myoelectric prostheses

INTRODUCTION

Despite recent technological advances that have led to sophisticated bionic prostheses, attaining embodied solutions still remains a challenge. Recently, the investigation of prosthetic embodiment has become a topic of interest in the research community, which deals with enhancing the perception of artificial limbs as part of users' own body. Surface electromyography (sEMG) interfaces have emerged as a promising technology for enhancing upper-limb prosthetic control. However, little is known about the impact of these sEMG interfaces on users' experience regarding embodiment and their interaction with different functional levels.

METHODS

To investigate this aspect, a comparison is conducted among sEMG configurations with different number of sensors (4 and 16 channels) and different time delay. We used a regression algorithm to simultaneously control hand closing/opening and forearm pronation/supination in an immersive virtual reality environment. The experimental evaluation includes 24 able-bodied subjects and one prosthesis user. We assess functionality with the Target Achievement Control test, and the sense of embodiment with a metric for the users perception of self-location, together with a standard survey.

RESULTS

Among the four tested conditions, results proved a higher subjective embodiment when participants used sEMG interfaces employing an increased number of sensors. Regarding functionality, significant improvement over time is observed in the same conditions, independently of the time delay implemented.

CONCLUSIONS

Our work indicates that a sufficient number of sEMG sensors improves both, functional and subjective embodiment outcomes. This prompts discussion regarding the potential relationship between these two aspects present in bionic integration. Similar embodiment outcomes are observed in the prosthesis user, showing also differences due to the time delay, and demonstrating the influence of sEMG interfaces on the sense of agency.

Measuring and monitoring skill learning in closed-loop myoelectric hand prostheses using speed-accuracy tradeoffs

Objective.Closed-loop myoelectric prostheses, which combine supplementary sensory feedback and electromyography (EMG) based control, hold the potential to narrow the divide between natural and bionic hands. The use of these devices, however, requires dedicated training. Therefore, it is crucial to develop methods that quantify how users acquire skilled control over their prostheses to effectively monitor skill progression and inform the development of interfaces that optimize this process.Approach.Building on theories of skill learning in human motor control, we measured speed-accuracy tradeoff functions (SAFs) to comprehensively characterize learning-induced changes in skill-as opposed to merely tracking changes in task success across training-facilitated by a closed-loop interface that combined proportional control and EMG feedback. Sixteen healthy participants and one individual with a transradial limb loss participated in a three-day experiment where they were instructed to perform the box-and-blocks task using a timed force-matching paradigm at four specified speeds to reach two target force levels, such that the SAF could be determined.Main results.We found that the participants' accuracy increased in a similar way across all speeds we tested. Consequently, the shape of the SAF remained similar across days, at both force levels. Further, we observed that EMG feedback enabled participants to improve their motor execution in terms of reduced trial-by-trial variability, a hallmark of skilled behavior. We then fit a power law model of the SAF, and demonstrated how the model parameters could be used to identify and monitor changes in skill.Significance.We comprehensively characterized how an EMG feedback interface enabled skill acquisition, both at the level of task performance and movement execution. More generally, we believe that the proposed methods are effective for measuring and monitoring user skill progression in closed-loop prosthesis control.

Sensory Restoration for Improved Motor Control of Prostheses

Somatosensory neuroprostheses are devices with the potential to restore the senses of touch and movement from prosthetic limbs for people with limb amputation or paralysis. By electrically stimulating the peripheral or central nervous system, these devices evoke sensations that appear to emanate from the missing or insensate limb, and when paired with sensors on the prosthesis, they can improve the functionality and embodiment of the prosthesis. There have been major advances in the design of these systems over the past decade, although several important steps remain before they can achieve widespread clinical adoption outside the lab setting. Here, we provide a brief overview of somatosensory neuroprostheses and explores these hurdles and potential next steps towards clinical translation.

Favoring the cognitive-motor process in the closed-loop of BCI mediated post stroke motor function recovery: challenges and approaches

The brain-computer interface (BCI)-mediated rehabilitation is emerging as a solution to restore motor skills in paretic patients after stroke. In the human brain, cortical motor neurons not only fire when actions are carried out but are also activated in a wired manner through many cognitive processes related to movement such as imagining, perceiving, and observing the actions. Moreover, the recruitment of motor cortexes can usually be regulated by environmental conditions, forming a closed-loop through neurofeedback. However, this cognitive-motor control loop is often interrupted by the impairment of stroke. The requirement to bridge the stroke-induced gap in the motor control loop is promoting the evolution of the BCI-based motor rehabilitation system and, notably posing many challenges regarding the disease-specific process of post stroke motor function recovery. This review aimed to map the current literature surrounding the new progress in BCI-mediated post stroke motor function recovery involved with cognitive aspect, particularly in how it refired and rewired the neural circuit of motor control through motor learning along with the BCI-centric closed-loop.

Vibrotactile Feedback for a Person with Transradial Amputation and Visual Loss: A Case Report

Background and Objectives: After major upper-limb amputation, people face challenges due to losing tactile information and gripping function in their hands. While vision can confirm the success of an action, relying on it diverts attention from other sensations and tasks. This case report presents a 30-year-old man with traumatic, complete vision loss and transradial left forearm amputation. It emphasizes the importance of restoring tactile abilities when visual compensation is impossible. Materials and Methods: A prototype tactile feedback add-on system was developed, consisting of a sensor glove and upper arm cuff with related vibration actuators. Results: We found a 66% improvement in the Box and Blocks test and an overall functional score increase from 30% to 43% in the Southampton Hand Assessment Procedure with feedback. Qualitative improvements in bimanual activities, ergonomics, and reduced reliance on the unaffected hand were observed. Incorporating the tactile feedback system improved the precision of grasping and the utility of the myoelectric hand prosthesis, freeing the unaffected hand for other tasks. Conclusions: This case demonstrated improvements in prosthetic hand utility achieved by restoring peripheral sensitivity while excluding the possibility of visual compensation. Restoring tactile information from the hand and fingers could benefit individuals with impaired vision and somatosensation, improving acceptance, embodiment, social integration, and pain management.

Integration of proprioception in upper limb prostheses through non-invasive strategies: a review

Proprioception plays a key role in moving our body dexterously and effortlessly. Nevertheless, the majority of investigations evaluating the benefits of providing supplemental feedback to prosthetics users focus on delivering touch restitution. These studies evaluate the influence of touch sensation in an attempt to improve the controllability of current robotic devices. Contrarily, investigations evaluating the capabilities of proprioceptive supplemental feedback have yet to be comprehensively analyzed to the same extent, marking a major gap in knowledge within the current research climate. The non-invasive strategies employed so far to restitute proprioception are reviewed in this work. In the absence of a clearly superior strategy, approaches employing vibrotactile, electrotactile and skin-stretch stimulation achieved better and more consistent results, considering both kinesthetic and grip force information, compared with other strategies or any incidental feedback. Although emulating the richness of the physiological sensory return through artificial feedback is the primary hurdle, measuring its effects to eventually support the integration of cumbersome and energy intensive hardware into commercial prosthetic devices could represent an even greater challenge. Thus, we analyze the strengths and limitations of previous studies and discuss the possible benefits of coupling objective measures, like neurophysiological parameters, as well as measures of prosthesis embodiment and cognitive load with behavioral measures of performance. Such insights aim to provide additional and collateral outcomes to be considered in the experimental design of future investigations of proprioception restitution that could, in the end, allow researchers to gain a more detailed understanding of possibly similar behavioral results and, thus, support one strategy over another.

The Effect of Feedback Modality When Learning a Novel Wrist Sensorimotor Transformation Through a Body-Machine Interface

Body-Machine Interfaces (BoMIs) are promising assistive and rehabilitative tools for helping individuals with impaired motor abilities regain independence. When operating a BoMI, the user has to learn a novel sensorimotor transformation between the movement of certain body parts and the output of the device. In this study, we investigated how different feedback modalities impacted learning to operate a BoMI. Forty-seven able-bodied participants learned to control the velocity of a 1D cursor using the 3D rotation of their dominant wrist to reach as many targets as possible in a given amount of time. The map was designed to maximize cursor speed for movements around a predefined axis of wrist rotation. We compared the user's performance and control efficiency under three feedback modalities: i) visual feedback of the cursor position, ii) proprioceptive feedback of the cursor position delivered by a wrist manipulandum, iii) both i) and ii). We found that visual feedback led to a greater number of targets reached than proprioceptive feedback alone. Conversely, proprioceptive feedback yielded greater alignment between the axis of rotation of the wrist and the optimal axis represented by the map. These results suggest that proprioceptive feedback may be preferable over visual feedback when information about intrinsic task components, i.e. joint configurations, is of interest as in rehabilitative interventions aiming to promote more effective learning strategies.

Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation

Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r² = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.

A Review of Myoelectric Control for Prosthetic Hand Manipulation

Myoelectric control for prosthetic hands is an important topic in the field of rehabilitation. Intuitive and intelligent myoelectric control can help amputees to regain upper limb function. However, current research efforts are primarily focused on developing rich myoelectric classifiers and biomimetic control methods, limiting prosthetic hand manipulation to simple grasping and releasing tasks, while rarely exploring complex daily tasks. In this article, we conduct a systematic review of recent achievements in two areas, namely, intention recognition research and control strategy research. Specifically, we focus on advanced methods for motion intention types, discrete motion classification, continuous motion estimation, unidirectional control, feedback control, and shared control. In addition, based on the above review, we analyze the challenges and opportunities for research directions of functionality-augmented prosthetic hands and user burden reduction, which can help overcome the limitations of current myoelectric control research and provide development prospects for future research.

Nonlinear Mapping From EMG to Prosthesis Closing Velocity Improves Force Control With EMG Biofeedback

When using EMG biofeedback to control the grasping force of a myoelectric prosthesis, subjects need to activate their muscles and maintain the myoelectric signal within an appropriate interval. However, their performance decreases for higher forces, because the myoelectric signal is more variable for stronger contractions. Therefore, the present study proposes to implement EMG biofeedback using nonlinear mapping, in which EMG intervals of increasing size are mapped to equal-sized intervals of the prosthesis velocity. To validate this approach, 20 non-disabled subjects performed force-matching tasks using Michelangelo prosthesis with and without EMG biofeedback with linear and nonlinear mapping. Additionally, four transradial amputees performed a functional task in the same feedback and mapping conditions. The success rate in producing desired force was significantly higher with feedback (65.4±15.9%) compared to no feedback (46.2±14.9%) as well as when using nonlinear (62.4±16.8%) versus linear mapping (49.2±17.2%). Overall, in non-disabled subjects, the highest success rate was obtained when EMG biofeedback was combined with nonlinear mapping (72%), and the opposite for linear mapping with no feedback (39.6%). The same trend was registered also in four amputee subjects. Therefore, EMG biofeedback improved prosthesis force control, especially when combined with nonlinear mapping, which showed to be an effective approach to counteract increasing variability of myoelectric signal for stronger contractions.

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

A scoping review of eye tracking metrics used to assess visuomotor behaviours of upper limb prosthesis users

Advanced upper limb prostheses aim to restore coordinated hand and arm function. However, this objective can be difficult to quantify as coordinated movements require an intact visuomotor system. Eye tracking has recently been applied to study the visuomotor behaviours of upper limb prosthesis users by enabling the calculation of eye movement metrics. This scoping review aims to characterize the visuomotor behaviours of upper limb prosthesis users as described by eye tracking metrics, to summarize the eye tracking metrics used to describe prosthetic behaviour, and to identify gaps in the literature and potential areas for future research. A review of the literature was performed to identify articles that reported eye tracking metrics to evaluate the visual behaviours of individuals using an upper limb prosthesis. Data on the level of amputation, type of prosthetic device, type of eye tracker, primary eye metrics, secondary outcome metrics, experimental task, aims, and key findings were extracted. Seventeen studies were included in this scoping review. A consistently reported finding is that prosthesis users have a characteristic visuomotor behaviour that differs from that of individuals with intact arm function. Visual attention has been reported to be directed more towards the hand and less towards the target during object manipulation tasks. A gaze switching strategy and delay to disengage gaze from the current target has also been reported. Differences in the type of prosthetic device and experimental task have revealed some distinct gaze behaviours. Control factors have been shown to be related to gaze behaviour, while sensory feedback and training interventions have been demonstrated to reduce the visual attention associated with prosthesis use. Eye tracking metrics have also been used to assess the cognitive load and sense of agency of prosthesis users. Overall, there is evidence that eye tracking is an effective tool to quantitatively assess the visuomotor behaviour of prosthesis users and the recorded eye metrics are sensitive to change in response to various factors. Additional studies are needed to validate the eye metrics used to assess cognitive load and sense of agency in upper limb prosthesis users.

Temporal and spatial characteristics of bone conduction as non-invasive haptic sensory feedback for upper-limb prosthesis

Bone conduction is a promising haptic feedback modality for upper-limb prosthesis users, however, its potential and characteristics as a non-invasive feedback modality have not been thoroughly investigated. This study aimed to establish the temporal and spatial characteristics of non-invasive bone conduction as a sensory feedback interface for upper-limb prostheses. Psychometric human-subject experiments were conducted on three bony landmarks of the elbow, with a vibrotactile transducer affixed to each to provide the stimulus. The study characterized the temporal domain by testing perception threshold and resolution in amplitude and frequency. The spatial domain was evaluated by assessing the ability of subjects to detect the number of simultaneous active stimulation sites. The experiment was conducted with ten able-bodied subjects and compared to two subjects with trans-radial amputation. The psychometric evaluation of the proposed non-invasive bone conduction feedback showed results comparable to invasive methods. The experimental results demonstrated similar amplitude and frequency resolution of the interface for all three stimulation sites for both able-bodied subjects and subjects with trans-radial amputation, highlighting its potential as a non-invasive feedback modality for upper-limb prostheses.

Multichannel Sensorimotor Integration with a Dexterous Artificial Hand

Background

People use their hands to perform sophisticated tasks like playing a musical instrument by integrating manifold and diverse sensations of touch with motor control strategies. In contrast, prosthetic hands lack the capacity for multichannel haptic feedback and multitasking functionality remains rudimentary. There is a dearth of research exploring the potential of upper limb absent (ULA) people to integrate multiple channels of haptic feedback into dexterous prosthetic hand control strategies.

Methods

In this paper, we designed a novel experimental paradigm for three ULA people and nine additional subjects to investigate their ability to integrate two simultaneously activated channels of context-specific haptic feedback into their dexterous artificial hand control strategies. Artificial neural networks (ANN) were designed for pattern recognition of the array of efferent electromyogram signals that controlled the dexterous artificial hand. ANNs were also used to classify the directions that objects were sliding across two tactile sensor arrays on the index (I) and little (L) fingertips of the robotic hand. The direction of sliding contact at each robotic fingertip was encoded by different stimulation frequencies of wearable vibrotactile actuators for haptic feedback. The subjects were tasked with implementing different control strategies with each finger simultaneously depending upon the perceived directions of sliding contact. This required the 12 subjects to concurrently control individual fingers of the artificial hand by successfully interpreting two channels of simultaneously activated context-specific haptic feedback.

Results

Subjects were able to accomplish this complex feat of multichannel sensorimotor integration with an overall accuracy of 95.53% ± 0.23%. While there was no statistically significant difference in the classification accuracy between ULA people and the other subjects, the ULA people required more time to correctly respond to the simultaneous haptic feedback slip signals, suggesting a higher cognitive load required by the ULA people.

Conclusion

ULA people can integrate multiple channels of simultaneously activated and nuanced haptic feedback with their control of individual fingers of an artificial hand. These findings provide a step toward empowering amputees to multitask with dexterous prosthetic hands, which remains an ongoing challenge.

Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis

Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.

Assessment of quality of life after upper extremity transplantation: Framework for patient-reported outcome scale domains

Upper extremity transplantation offers the promise of restored function and regained quality of life (QOL) for individuals who have sustained hand or arm amputation. However, a major challenge for this procedure becoming an accessible treatment option for patients is the lack of standard measures to document benefits to QOL. Patient-reported outcomes (PRO) measures are well-suited for this kind of intervention, where the perspective of the patient is central to defining treatment success. To date, qualitative work with experts, clinicians, and patients has been used to identify the most important domains of QOL for PRO item development. Specifically, our group's qualitative work has identified several domains of QOL that are unique to individuals who have received upper extremity transplants, which are distinct from topics covered by existing PRO measures. These include emotional and social aspects of upper extremity transplant, such as Expectations and Perceived Outcomes, Integration and Assimilation of Transplant, Fitting in, and Post-Surgical Challenges and Complications. The broad topic of Satisfaction with Transplant was subdivided into three subtopics: Function, Sensation, and Aesthetics. Satisfaction with Sensation was also identified as a unique domain not evaluated by existing PRO measures. This report operationalizes these eight QOL domains by presenting scoping definitions. This manuscript describes the work that has been completed for domain characterization as an early step toward developing standardized PRO measures to evaluate these important outcomes specific to upper extremity transplantation.

Biomechanically-Consistent Skin Stretch as an Intuitive Mechanism for Sensory Feedback: A Preliminary Investigation in the Lower Limb

The proprioceptive loss accompanied by lower limb amputation can impair function and mobility. We explore a simple, mechanical skin-stretch array configured to generate superficial tissue behaviour that might occur with movement about an intact joint. Four adhesive pads attached around the circumference of the lower leg were connected via cords to a remote "foot" mounted on a ball joint attached to the underside of a fracture boot, such that "foot" reorientation would result in skin stretch. In two discrimination experiments performed with and without the connection, with no view of the mechanism, and with minimal training, unimpaired adults (i) estimated foot orientation following passive foot rotations (eight directions), either with or without contact between the lower leg and boot, and (ii) actively lowered the "foot" to estimate slope orientation (four directions). In (i), 56-60% of responses (depending on contact condition) were correct and 88-94% were either correct or one of the two adjacent choices. In (ii), 56% of responses were correct. In contrast, without the connection, participants performed near or no different to chance. A biomechanically-consistent skin stretch array may be an intuitive means to convey proprioceptive information from an artificial or poorly innervated joint.

Haptic shared control improves neural efficiency during myoelectric prosthesis use

Clinical myoelectric prostheses lack the sensory feedback and sufficient dexterity required to complete activities of daily living efficiently and accurately. Providing haptic feedback of relevant environmental cues to the user or imbuing the prosthesis with autonomous control authority have been separately shown to improve prosthesis utility. Few studies, however, have investigated the effect of combining these two approaches in a shared control paradigm, and none have evaluated such an approach from the perspective of neural efficiency (the relationship between task performance and mental effort measured directly from the brain). In this work, we analyzed the neural efficiency of 30 non-amputee participants in a grasp-and-lift task of a brittle object. Here, a myoelectric prosthesis featuring vibrotactile feedback of grip force and autonomous control of grasping was compared with a standard myoelectric prosthesis with and without vibrotactile feedback. As a measure of mental effort, we captured the prefrontal cortex activity changes using functional near infrared spectroscopy during the experiment. It was expected that the prosthesis with haptic shared control would improve both task performance and mental effort compared to the standard prosthesis. Results showed that only the haptic shared control system enabled users to achieve high neural efficiency, and that vibrotactile feedback was important for grasping with the appropriate grip force. These results indicate that the haptic shared control system synergistically combines the benefits of haptic feedback and autonomous controllers, and is well-poised to inform such hybrid advancements in myoelectric prosthesis technology.

Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach

Introduction

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction.

Methods

Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees.

Results

The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).

Evaluating the effect of non-invasive force feedback on prosthetic grasp force modulation in participants with and without limb loss

Grasping an object is one of the most common and complex actions performed by humans. The human brain can adapt and update the grasp dynamics through information received from sensory feedback. Prosthetic hands can assist with the mechanical performance of grasping, however currently commercially available prostheses do not address the disruption of the sensory feedback loop. Providing feedback about a prosthetic hand's grasp force magnitude is a top priority for those with limb loss. This study tested a wearable haptic system, i.e., the Clenching Upper-Limb Force Feedback device (CUFF), which was integrated with a novel robotic hand (The SoftHand Pro). The SoftHand Pro was controlled with myoelectrics of the forearm muscles. Five participants with limb loss and nineteen able-bodied participants completed a constrained grasping task (with and without feedback) which required modulation of the grasp to reach a target force. This task was performed while depriving participants of incidental sensory sources (vision and hearing were significantly limited with glasses and headphones). The data were analyzed with Functional Principal Component Analysis (fPCA). CUFF feedback improved grasp precision for participants with limb loss who typically use body-powered prostheses as well as a sub-set of able-bodied participants. Further testing, that is more functional and allows participants to use all sensory sources, is needed to determine if CUFF feedback can accelerate mastery of myoelectric control or would benefit specific patient sub-groups.

Conceptualization of an Anthropomorphic Replacement Hand with a Sensory Feedback System

In this paper, a concept for an anthropomorphic replacement hand cast with silicone with an integrated sensory feedback system is presented. In order to construct the personalized replacement hand, a 3D scan of a healthy hand was used to create a 3D-printed mold using computer-aided design (CAD). To allow for movement of the index and middle fingers, a motorized orthosis was used. Information about the applied force for grasping and the degree of flexion of the fingers is registered using two pressure sensors and one bending sensor in each movable finger. To integrate the sensors and additional cavities for increased flexibility, the fingers were cast in three parts, separately from the rest of the hand. A silicone adhesive (Silpuran 4200) was examined to combine the individual parts afterwards. For this, tests with different geometries were carried out. Furthermore, different test series for the secure integration of the sensors were performed, including measurements of the registered information of the sensors. Based on these findings, skin-toned individual fingers and a replacement hand with integrated sensors were created. Using Silpuran 4200, it was possible to integrate the needed cavities and to place the sensors securely into the hand while retaining full flexion using a motorized orthosis. The measurements during different loadings and while grasping various objects proved that it is possible to realize such a sensory feedback system in a replacement hand. As a result, it can be stated that the cost-effective realization of a personalized, anthropomorphic replacement hand with an integrated sensory feedback system is possible using 3D scanning and 3D printing. By integrating smaller sensors, the risk of damaging the sensors through movement could be decreased.

Touch Hand 4.5: low-cost additive manufacturing prosthetic hand participated in Cybathlon 2020 ARM discipline

BACKGROUND

The Touch Hand 4.5 is a highly customisable prosthetic hand, which features an optimised modular design of the Touch Hand 4. The Touch Hand team has developed a low-cost prosthetic hand, which has been built using an additive manufacturing process. The functionality and features are discussed that are crucial for amputees.

METHODS

This paper documents the design and integration of the Touch Hand 4.5 to be used in the Cybathlon 2020 event as well as the development of the mechanical structure of the hand, socket, electronics and control system utilized. The Touch Hand 4.5 was designed and continuously optimized, with the goal to achieve the tasks in the Cybathlon 2020 event.

RESULTS

The performance and functionality of the Touch Hand 4.5 was tested on a global scale at the Cybathlon 2020. The device and technology were evaluated against the leading prosthetics and prototypes from around the world. A series of everyday tasks, as set by the Cybathlon event, were performed to determine the capabilities of the device, with the pinch grip, full grip, half grip, and a thumb grip. The Touch Hand team was the only team to complete the Haptic Box task in all three races, which comprised of the identification of objects without the aid of visual input or perception, with a duration between 100 and 120 s. The Breakfast task entailed completing a series of everyday breakfast tasks, such as cutting a loaf of bread, lighting a candle, opening a sugar packet, opening a plastic bottle and a jar, as well as opening a tin can with a can opener. This task was only completed in Race 3, with a duration of 132 s, due to a faulty equipment that was supplied.

CONCLUSION

The first contribution that was achieved was the design and development of an additive manufactured hand and socket, considering the socket to have comfort, breathability and decreased irritability. The second contribution was the design optimisation with the linear actuator integration, for a multi-grip hand, which allowed for the pinch grip, full grip, half grip, and a thumb grip. Slippage prevention with grip force control system integration was also implemented.

TRIAL REGISTRATION NUMBER

Ethical clearance certificate HCC/0161/011.

Asynchronous Intermittent Regulation of Human Arm Movement with Markovian Jumping Parameters

In this paper, the regulation stability problem of the human arm continuous movement is investigated based on Markovian jumping parameters. In particular, the intermittent control mechanism is adopted in the arm movement regulation procedure to model the human intermittent motor control strategy. Furthermore, by taking into account the Markovian jumping parameters with different modes, the asynchronous regulation issue is proposed to model mode mismatch between the motor control and arm movement. On the basis of model transformation, sufficient stability conditions are established during the arm movements, and the desired regulation gain can be obtained by the convex optimization method. In the end, an illustrative example is presented to show the applicability and effectiveness of our developed model and optimized regulation approach.

Vibrotactile mapping of the upper extremity: Absolute perceived intensity is location-dependent; perception of relative changes is not

Vibrotactile sensation is an essential part of the sense of touch. In this study, the localized vibrotactile sensation of the arm-shoulder region was quantified in 10 able-bodied subjects. For this analysis, the six relevant dermatomes (C3-T2) and three segments—the lower arm, the upper arm, and the shoulder region were studied. For psychometric evaluation, tasks resulting in the quantification of sensation threshold, just noticeable difference, Weber fraction, and perception of dynamically changing vibrotactile stimuli were performed. We found that healthy subjects could reliably detect vibration in all tested regions at low amplitude (2–6% of the maximal amplitude of commonly used vibrotactors). The detection threshold was significantly lower in the lower arm than that in the shoulder, as well as ventral in comparison with the dorsal. There were no significant differences in Weber fraction (20%) detectable between the studied locations. A compensatory tracking task resulted in a significantly higher average rectified error in the shoulder than that in the upper arm, while delay and correlation coefficient showed no difference between the regions. Here, we presented a conclusive map of the vibrotactile sense of the healthy upper limb. These data give an overview of the sensory bandwidth that can be achieved with vibrotactile stimulation at the arm and may help in the design of vibrotactile feedback interfaces (displays) for the hand/arm/shoulder-region.

FeetBack–Redirecting touch sensation from a prosthetic hand to the human foot

Introduction

Adding sensory feedback to myoelectric prosthetic hands was shown to enhance the user experience in terms of controllability and device embodiment. Often this is realized non-invasively by adding devices, such as actuators or electrodes, within the prosthetic shaft to deliver the desired feedback. However, adding a feedback system in the socket adds more weight, steals valuable space, and may interfere with myoelectric signals. To circumvent said drawbacks we tested for the first time if force feedback from a prosthetic hand could be redirected to another similarly sensitive part of the body: the foot.

Methods

We developed a vibrotactile insole that vibrates depending on the sensed force on the prosthetic fingers. This self-controlled clinical pilot trial included four experienced users of myoelectric prostheses. The participants solved two types of tasks with the artificial hands: 1) sorting objects depending on their plasticity with the feedback insole but without audio-visual feedback, and 2) manipulating fragile, heavy, and delicate objects with and without the feedback insole. The sorting task was evaluated with Goodman-Kruskal's gamma for ranked correlation. The manipulation tasks were assessed by the success rate.

Results

The results from the sorting task with vibrotactile feedback showed a substantial positive effect. The success rates for manipulation tasks with fragile and heavy objects were high under both conditions (feedback on or off, respectively). The manipulation task with delicate objects revealed inferior success with feedback in three of four participants.

Conclusion

We introduced a novel approach to touch sensation in myoelectric prostheses. The results for the sorting task and the manipulation tasks diverged. This is likely linked to the availability of various feedback sources. Our results for redirected feedback to the feet fall in line with previous similar studies that applied feedback to the residual arm.

Clinical trial registration

Name: Sensor Glove and Non-Invasive Vibrotactile Feedback Insole to Improve Hand Prostheses Functions and Embodiment (FeetBack). Date of registration: 23 April 2019. Date the first participant was enrolled: 3 September 2021. ClinicalTrials.gov Identifier: NCT03924310.

Encoding of spatial patterns using electrotactile stimulation via a multi-pad electrode placed on the torso

BACKGROUND

Tactile stimulation can be used to convey information to a user in different scenarios while avoiding overloading other senses. Tactile messages can be transmitted as spatial patterns, potentially allowing for a high information throughput. The aim of the present study was to design and test different encoding schemes to determine the best approach for conveying spatial patterns.

METHODS

Encoding schemes with simultaneous (SIM) and sequential pad activation (SEQ) were evaluated, including four SEQ variants designed to potentially facilitate the recognition. In SEQ-col and SEQ-row, the column and row of the activated pad were signified using different frequencies, while SEQ-all and SEQ-all-fast included the activation of all pads where those belonging to the pattern were indicated by changes in frequency (ON pads). The success rate (SR) of the pattern identification and the response time were quantified in 15 participants who recognized 20 patterns delivered through a 3 × 2 pad matrix placed on the lateral torso.

RESULTS

SIM was not a feasible method to present the patterns (median, 15%; IQR, 5%). The SR improved with SEQ (median, 60%; IQR, 20%) and further increased with additional cues, particularly with SEQ-row (median, 78.3%; IQR, 23.3%) and SEQ-all (median, 96.7%; IQR, 5%). Importantly, the stimulation time of SEQ-all could be decreased without a substantial drop in accuracy (SEQ-all-fast: median, 89.2%; IQR, 19.2%).

CONCLUSIONS

The spatiotemporal stimulation with sequential activation of all pads (SEQ-all) seems to be the method of choice when conveying tactile messages as spatial patterns. This is an important outcome for increasing the information bandwidth of communication through the tactile channel.

Context matters during pick-and-place in VR: Impact on search and transport phases

When considering external assistive systems for people with motor impairments, gaze has been shown to be a powerful tool as it is anticipatory to motor actions and is promising for understanding intentions of an individual even before the action. Up until now, the vast majority of studies investigating the coordinated eye and hand movement in a grasping task focused on single objects manipulation without placing them in a meaningful scene. Very little is known about the impact of the scene context on how we manipulate objects in an interactive task. In the present study, it was investigated how the scene context affects human object manipulation in a pick-and-place task in a realistic scenario implemented in VR. During the experiment, participants were instructed to find the target object in a room, pick it up, and transport it to a predefined final location. Thereafter, the impact of the scene context on different stages of the task was examined using head and hand movement, as well as eye tracking. As the main result, the scene context had a significant effect on the search and transport phases, but not on the reach phase of the task. The present work provides insights into the development of potential supporting intention predicting systems, revealing the dynamics of the pick-and-place task behavior once it is realized in a realistic context-rich scenario.

Artificial referred sensation in upper and lower limb prosthesis users: a systematic review

Objective.Electrical stimulation can induce sensation in the phantom limb of individuals with amputation. It is difficult to generalize existing findings as there are many approaches to delivering stimulation and to assessing the characteristics and benefits of sensation. Therefore, the goal of this systematic review was to explore the stimulation parameters that effectively elicited referred sensation, the qualities of elicited sensation, and how the utility of referred sensation was assessed.Approach.We searched PubMed, Web of Science, and Engineering Village through January of 2022 to identify relevant papers. We included papers which electrically induced referred sensation in individuals with limb loss and excluded papers that did not contain stimulation parameters or outcome measures pertaining to stimulation. We extracted information on participant demographics, stimulation approaches, and participant outcomes.Main results.After applying exclusion criteria, 49 papers were included covering nine stimulation methods. Amplitude was the most commonly adjusted parameter (n= 25), followed by frequency (n= 22), and pulse width (n= 15). Of the 63 reports of sensation quality, most reported feelings of pressure (n= 52), paresthesia (n= 48), or vibration (n= 40) while less than half (n= 29) reported a sense of position or movement. Most papers evaluated the functional benefits of sensation (n= 33) using force matching or object identification tasks, while fewer papers quantified subjective measures (n= 16) such as pain or embodiment. Only 15 studies (36%) observed percept intensity, quality, or location over multiple sessions.Significance.Most studies that measured functional performance demonstrated some benefit to providing participants with sensory feedback. However, few studies could experimentally manipulate sensation location or quality. Direct comparisons between studies were limited by variability in methodologies and outcome measures. As such, we offer recommendations to aid in more standardized reporting for future research.

Estimating speed-accuracy trade-offs to evaluate and understand closed-loop prosthesis interfaces

OBJECTIVE

Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces.

APPROACH

Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a one-week washout period. Importantly, both interfaces used (identical) direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). Therefore, we estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions.

MAIN RESULTS

We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands.

SIGNIFICANCE

Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces.

Reimagining Prosthetic Control: A Novel Body-Powered Prosthetic System for Simultaneous Control and Actuation

Globally, the most popular upper-limb prostheses are powered by the human body. For body-powered (BP) upper-limb prostheses, control is provided by changing the tension of (Bowden) cables to open or close the terminal device. This technology has been around for centuries, and very few BP alternatives have been presented since. This paper introduces a new BP paradigm that can overcome certain limitations of the current cabled systems, such as a restricted operation space and user discomfort caused by the harness to which the cables are attached. A new breathing-powered system is introduced to give the user full control of the hand motion anywhere in space. Users can regulate their breathing, and this controllable airflow is then used to power a small Tesla turbine that can accurately control the prosthetic finger movements. The breathing-powered device provides a novel prosthetic option that can be used without limiting any of the user’s body movements. Here we prove that it is feasible to produce a functional breathing-powered prosthetic hand and show the models behind it along with a preliminary demonstration. This work creates a step-change in the potential BP options available to patients in the future.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future Directions

Haptic feedback is critical in a broad range of human-machine/computer-interaction applications. However, the high cost and low portability/wearability of haptic devices remain unresolved issues, severely limiting the adoption of this otherwise promising technology. Electrotactile interfaces have the advantage of being more portable and wearable due to their reduced actuators' size, as well as their lower power consumption and manufacturing cost. The applications of electrotactile feedback have been explored in human-computer interaction and human-machine-interaction for facilitating hand-based interactions in applications, such as prosthetics, virtual reality, robotic teleoperation, surface haptics, portable devices, and rehabilitation. This article presents a technological overview of electrotactile feedback, as well a systematic review and meta-analysis of its applications for hand-based interactions. We discuss the different electrotactile systems according to the type of application. We also discuss over a quantitative congregation of the findings, to offer a high-level overview into the state-of-art and suggest future directions. Electrotactile feedback systems showed increased portability/wearability, and they were successful in rendering and/or augmenting most tactile sensations, eliciting perceptual processes, and improving performance in many scenarios. However, knowledge gaps (e.g., embodiment), technical (e.g., recurrent calibration, electrodes' durability) and methodological (e.g., sample size) drawbacks were detected, which should be addressed in future studies.

Sensory substitution of elbow proprioception to improve myoelectric control of upper limb prosthesis: experiment on healthy subjects and amputees

BACKGROUND

Current myoelectric prostheses lack proprioceptive information and rely on vision for their control. Sensory substitution is increasingly developed with non-invasive vibrotactile or electrotactile feedback, but most systems are designed for grasping or object discriminations, and few were tested for online control in amputees. The objective of this work was evaluate the effect of a novel vibrotactile feedback on the accuracy of myoelectric control of a virtual elbow by healthy subjects and participants with an upper-limb amputation at humeral level.

METHODS

Sixteen, healthy participants and 7 transhumeral amputees performed myoelectric control of a virtual arm under different feedback conditions: vision alone (VIS), vibration alone (VIB), vision plus vibration (VIS + VIB), or no feedback at all (NO). Reach accuracy was evaluated by angular errors during discrete as well as back and forth movements. Healthy participants' workloads were assessed with the NASA-TLX questionnaire, and feedback conditions were ranked according to preference at the end of the experiment.

RESULTS

Reach errors were higher in NO than in VIB, indicating that our vibrotactile feedback improved performance as compared to no feedback. Conditions VIS and VIS+VIB display similar levels of performance and produced lower errors than in VIB. Vision remains therefore critical to maintain good performance, which is not ameliorated nor deteriorated by the addition of vibrotactile feedback. The workload associated with VIB was higher than for VIS and VIS+VIB, which did not differ from each other. 62.5% of healthy subjects preferred the VIS+VIB condition, and ranked VIS and VIB second and third, respectively.

CONCLUSION

Our novel vibrotactile feedback improved myoelectric control of a virtual elbow as compared to no feedback. Although vision remained critical, the addition of vibrotactile feedback did not improve nor deteriorate the control and was preferred by participants. Longer training should improve performances with VIB alone and reduce the need of vision for close-loop prosthesis control.

Floating EMG sensors and stimulators wirelessly powered and operated by volume conduction for networked neuroprosthetics

Background

Implantable neuroprostheses consisting of a central electronic unit wired to electrodes benefit thousands of patients worldwide. However, they present limitations that restrict their use. Those limitations, which are more adverse in motor neuroprostheses, mostly arise from their bulkiness and the need to perform complex surgical implantation procedures. Alternatively, it has been proposed the development of distributed networks of intramuscular wireless microsensors and microstimulators that communicate with external systems for analyzing neuromuscular activity and performing stimulation or controlling external devices. This paradigm requires the development of miniaturized implants that can be wirelessly powered and operated by an external system. To accomplish this, we propose a wireless power transfer (WPT) and communications approach based on volume conduction of innocuous high frequency (HF) current bursts. The currents are applied through external textile electrodes and are collected by the wireless devices through two electrodes for powering and bidirectional digital communications. As these devices do not require bulky components for obtaining power, they may have a flexible threadlike conformation, facilitating deep implantation by injection.

Methods

We report the design and evaluation of advanced prototypes based on the above approach. The system consists of an external unit, floating semi-implantable devices for sensing and stimulation, and a bidirectional communications protocol. The devices are intended for their future use in acute human trials to demonstrate the distributed paradigm. The technology is assayed in vitro using an agar phantom, and in vivo in hindlimbs of anesthetized rabbits.

Results

The semi-implantable devices were able to power and bidirectionally communicate with the external unit. Using 13 commands modulated in innocuous 3 MHz HF current bursts, the external unit configured the sensing and stimulation parameters, and controlled their execution. Raw EMG was successfully acquired by the wireless devices at 1 ksps.

Conclusions

The demonstrated approach overcomes key limitations of existing neuroprostheses, paving the way to the development of distributed flexible threadlike sensors and stimulators. To the best of our knowledge, these devices are the first based on WPT by volume conduction that can work as EMG sensors and as electrical stimulators in a network of wireless devices.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01033-3.

Preliminary Evaluation of the Effect of Mechanotactile Feedback Location on Myoelectric Prosthesis Performance Using a Sensorized Prosthetic Hand

A commonly cited reason for the high abandonment rate of myoelectric prostheses is a lack of grip force sensory feedback. Researchers have attempted to restore grip force sensory feedback by stimulating the residual limb’s skin surface in response to the prosthetic hand’s measured grip force. Recent work has focused on restoring natural feedback to the missing digits directly through invasive surgical procedures. However, the functional benefit of utilizing somatotopically matching feedback has not been evaluated. In this paper, we propose an experimental protocol centered on a fragile object grasp and lift task using a sensorized myoelectric prosthesis to evaluate sensory feedback techniques. We formalized a suite of outcome measures related to task success, timing, and strategy. A pilot study (n = 3) evaluating the effect of utilizing a somatotopically accurate feedback stimulation location in able-bodied participants was conducted to evaluate the feasibility of the standardized platform, and to inform future studies on the role of feedback stimulation location in prosthesis use. Large between-participant effect sizes were observed in all outcome measures, indicating that the feedback location likely plays a role in myoelectric prosthesis performance. The success rate decreased, and task timing and task focus metrics increased, when using somatotopically-matched feedback compared to non-somatotopically-matched feedback. These results were used to conduct a power analysis, revealing that a sample size of n = 8 would be sufficient to achieve significance in all outcome measures.

Electrotactile feedback improves grip force control and enables object stiffness recognition while using a myoelectric hand

Current myoelectric hands are limited in their ability to provide effective sensory feedback to the users, which highly affects their functionality and utility. Although the sensory information of a myoelectric hand can be acquired with equipped sensors, transforming the sensory signals into effective tactile sensations on users for functional tasks is a largely unsolved challenge. The purpose of this study aims to demonstrate that electrotactile feedback of the grip force improves the sensorimotor control of a myoelectric hand and enables object stiffness recognition. The grip force of a sensorized myoelectric hand was delivered to its users via electrotactile stimulation based on four kinds of typical encoding strategies, including graded (G), linear amplitude (LA), linear frequency (LF), and biomimetic (B) modulation. Object stiffness was encoded with the change of electrotactile sensations triggered by final grip force, as the prosthesis grasped the objects. Ten able-bodied subjects and two transradial amputees were recruited to participate in a dual-task virtual eggs test (VET) and an object stiffness discrimination test (OSDT) to quantify the prosthesis users' ability to handle fragile objects and recognize object stiffnesses, respectively. The quantified results showed that with electrotactile feedback enabled, the four kinds of encoding strategies allowed subjects to better able to handle fragile objects with similar performance, and the subjects were able to differentiate four levels of object stiffness at favorable accuracies (>86%) and high manual efficiency. Strategy LA presented the best stiffness discrimination performance, while strategy B was able to reduce the discrimination time but the discrimination accuracy was not better than the other three strategies. Electrotactile feedback also enhanced prosthesis embodiment and improved the users' confidence in prosthetic control. Outcomes indicate electrotactile feedback can be effectively exploited by the prosthesis users for grip force control and object stiffness recognition, proving the feasibility of functional sensory restoration of myoelectric prostheses equipped with electrotactile feedback.

A Haptic Sleeve as a Method of Mechanotactile Feedback Restoration for Myoelectric Hand Prosthesis Users

Current myoelectric upper limb prostheses do not restore sensory feedback, impairing fine motor control. Mechanotactile feedback restoration with a haptic sleeve may rectify this problem. This randomised crossover within-participant controlled study aimed to assess a prototype haptic sleeve's effect on routine grasping tasks performed by eight able-bodied participants. Each participant completed 15 repetitions of the three tasks: Task 1—normal grasp, Task 2—strong grasp and Task 3—weak grasp, using visual, haptic, or combined feedback All data were collected in April 2021 in the Scottish Microelectronics Centre, Edinburgh, UK. Combined feedback correlated with significantly higher grasp success rates compared to the vision alone in Task 1 (p < 0.0001), Task 2 (p = 0.0057), and Task 3 (p = 0.0170). Similarly, haptic feedback was associated with significantly higher grasp success rates compared to vision in Task 1 (p < 0.0001) and Task 2 (p = 0.0015). Combined feedback correlated with significantly lower energy expenditure compared to visual feedback in Task 1 (p < 0.0001) and Task 3 (p = 0.0003). Likewise, haptic feedback was associated with significantly lower energy expenditure compared to the visual feedback in Task 1 (p < 0.0001), Task 2 (p < 0.0001), and Task 3 (p < 0.0001). These results suggest that mechanotactile feedback provided by the haptic sleeve effectively augments grasping and reduces its energy expenditure.

Prosthetic embodiment: systematic review on definitions, measures, and experimental paradigms

The term embodiment has become omnipresent within prosthetics research and is often used as a metric of the progress made in prosthetic technologies, as well as a hallmark for user acceptance. However, despite the frequent use of the term, the concept of prosthetic embodiment is often left undefined or described incongruently, sometimes even within the same article. This terminological ambiguity complicates the comparison of studies using embodiment as a metric of success, which in turn hinders the advancement of prosthetics research. To resolve these terminological ambiguities, we systematically reviewed the used definitions of embodiment in the prosthetics literature. We performed a thematic analysis of the definitions and found that embodiment is often conceptualized in either of two frameworks based on body representations or experimental phenomenology. We concluded that treating prosthetic embodiment within an experimental phenomenological framework as the combination of ownership and agency allows for embodiment to be a quantifiable metric for use in translational research. To provide a common reference and guidance on how to best assess ownership and agency, we conducted a second systematic review, analyzing experiments and measures involving ownership and agency. Together, we highlight a pragmatic definition of prosthetic embodiment as the combination of ownership and agency, and in an accompanying article, we provide a perspective on a multi-dimensional framework for prosthetic embodiment. Here, we concluded by providing recommendations on metrics that allow for outcome comparisons between studies, thereby creating a common reference for further discussions within prosthetics research.

Continuous Semi-autonomous Prosthesis Control Using a Depth Sensor on the Hand

Modern myoelectric prostheses can perform multiple functions (e.g., several grasp types and wrist rotation) but their intuitive control by the user is still an open challenge. It has been recently demonstrated that semi-autonomous control can allow the subjects to operate complex prostheses effectively; however, this approach often requires placing sensors on the user. The present study proposes a system for semi-autonomous control of a myoelectric prosthesis that requires a single depth sensor placed on the dorsal side of the hand. The system automatically pre-shapes the hand (grasp type, size, and wrist rotation) and allows the user to grasp objects of different shapes, sizes and orientations, placed individually or within cluttered scenes. The system “reacts” to the side from which the object is approached, and enables the user to target not only the whole object but also an object part. Another unique aspect of the system is that it relies on online interaction between the user and the prosthesis; the system reacts continuously on the targets that are in its focus, while the user interprets the movement of the prosthesis to adjust aiming. Experimental assessment was conducted in ten able-bodied participants to evaluate the feasibility and the impact of training on prosthesis-user interaction. The subjects used the system to grasp a set of objects individually (Phase I) and in cluttered scenarios (Phase II), while the time to accomplish the task (TAT) was used as the performance metric. In both phases, the TAT improved significantly across blocks. Some targets (objects and/or their parts) were more challenging, requiring thus significantly more time to handle, but all objects and scenes were successfully accomplished by all subjects. The assessment therefore demonstrated that the system is indeed robust and effective, and that the subjects could successfully learn how to aim with the system after a brief training. This is an important step toward the development of a self-contained semi-autonomous system convenient for clinical applications.

Sensory Feedback for Upper-Limb Prostheses: Opportunities and Barriers

The addition of sensory feedback to upper-limb prostheses has been shown to improve control, increase embodiment, and reduce phantom limb pain. However, most commercial prostheses do not incorporate sensory feedback due to several factors. This paper focuses on the major challenges of a lack of deep understanding of user needs, the unavailability of tailored, realistic outcome measures and the segregation between research on control and sensory feedback. The use of methods such as the Person-Based Approach and co-creation can improve the design and testing process. Stronger collaboration between researchers can integrate different prostheses research areas to accelerate the translation process.

Bidirectional bionic limbs: a perspective bridging technology and physiology

Precise control of bionic limbs relies on robust decoding of motor commands from nerves or muscles signals and sensory feedback from artificial limbs to the nervous system by interfacing the afferent nerve pathways. Implantable devices for bidirectional communication with bionic limbs have been developed in parallel with research on physiological alterations caused by an amputation. In this perspective article, we question whether increasing our effort on bridging these technologies with a deeper understanding of amputation pathophysiology and human motor control may help to overcome pressing stalls in the next generation of bionic limbs.