Tactile feedback is an effective instrument for the training of grasping with a prosthesis at low- and medium-force levels

Excitation of natural spinal reflex loops in the sensory-motor control of hand prostheses

Sensory feedback for prosthesis control is typically based on encoding sensory information in specific types of sensory stimuli that the users interpret to adjust the control of the prosthesis. However, in physiological conditions, the afferent feedback received from peripheral nerves is not only processed consciously but also modulates spinal reflex loops that contribute to the neural information driving muscles. Spinal pathways are relevant for sensory-motor integration, but they are commonly not leveraged for prosthesis control. We propose an approach to improve sensory-motor integration for prosthesis control based on modulating the excitability of spinal circuits through the vibration of tendons in a closed loop with muscle activity. We measured muscle signals in healthy participants and amputees during different motor tasks, and we closed the loop by applying vibration on tendons connected to the muscles, which modulated the excitability of motor neurons. The control signals to the prosthesis were thus the combination of voluntary control and additional spinal reflex inputs induced by tendon vibration. Results showed that closed-loop tendon vibration was able to modulate the neural drive to the muscles. When closed-loop tendon vibration was used, participants could achieve similar or better control performance in interfaces using muscle activation than without stimulation. Stimulation could even improve prosthetic grasping in amputees. Overall, our results indicate that closed-loop tendon vibration can integrate spinal reflex pathways in the myocontrol system and open the possibility of incorporating natural feedback loops in prosthesis control.

An early force prediction control scheme using multimodal sensing of electromyography and digit force signals

Different grasping gestures result in the change of muscular activity of the forearm muscles. Similarly, the muscular activity changes with a change in grip force while grasping the object. This change in muscular activity, measured by a technique called Electromyography (EMG) is used in the upper limb bionic devices to select the grasping gesture. Previous research studies have shown gesture classification using pattern recognition control schemes. However, the use of EMG signals for force manipulation is less focused, especially during precision grasping. In this study, an early predictive control scheme is designed for the efficient determination of grip force using EMG signals from forearm muscles and digit force signals. The optimal pattern recognition (PR) control schemes are investigated using three different inputs of two signals: EMG signals, digit force signals and a combination of EMG and digit force signals. The features extracted from EMG signals included Slope Sign Change, Willison Amplitude, Auto Regressive Coefficient and Waveform Length. The classifiers used to predict force levels are Random Forest, Gradient Boosting, Linear Discriminant Analysis, Support Vector Machines, k-nearest Neighbors and Decision Tree. The two-fold objectives of early prediction and high classification accuracy of grip force level were obtained using EMG signals and digit force signals as inputs and Random Forest as a classifier. The earliest prediction was possible at 1000 ms from the onset of the gripping of the object with a mean classification accuracy of 90 % for different grasping gestures. Using this approach to study, an early prediction will result in the determination of force level before the object is lifted from the surface. This approach will also result in better biomimetic regulation of the grip force during precision grasp, especially for a population facing vision deficiency.

EMG feedback improves grasping of compliant objects using a myoelectric prosthesis

BACKGROUND

Closing the control loop in myoelectric prostheses by providing artificial somatosensory feedback is recognized as an important goal. However, designing a feedback interface that is effective in realistic conditions is still a challenge. Namely, in some situations, feedback can be redundant, as the information it provides can be readily obtained through hearing or vision (e.g., grasping force estimated from the deformation of a compliant object). EMG feedback is a non-invasive method wherein the tactile stimulation conveys to the user the level of their own myoelectric signal, hence a measurement intrinsic to the interface, which cannot be accessed incidentally.

METHODS

The present study investigated the efficacy of EMG feedback in prosthesis force control when 10 able-bodied participants and a person with transradial amputation used a myoelectric prosthesis to grasp compliant objects of different stiffness values. The performance with feedback was compared to that achieved when the participants relied solely on incidental cues.

RESULTS

The main outcome measures were the task success rate and completion time. EMG feedback resulted in significantly higher success rates regardless of pin stiffness, indicating that the feedback enhanced the accuracy of force application despite the abundance of incidental cues. Contrary to expectations, there was no difference in the completion time between the two feedback conditions. Additionally, the data revealed that the participants could produce smoother control signals when they received EMG feedback as well as more consistent commands across trials, signifying better control of the system by the participants.

CONCLUSIONS

The results presented in this study further support the efficacy of EMG feedback when closing the prosthesis control loop by demonstrating its benefits in particularly challenging conditions which maximized the utility of intrinsic feedback sources.

Nonlinear Mapping From EMG to Prosthesis Closing Velocity Improves Force Control With EMG Biofeedback

When using EMG biofeedback to control the grasping force of a myoelectric prosthesis, subjects need to activate their muscles and maintain the myoelectric signal within an appropriate interval. However, their performance decreases for higher forces, because the myoelectric signal is more variable for stronger contractions. Therefore, the present study proposes to implement EMG biofeedback using nonlinear mapping, in which EMG intervals of increasing size are mapped to equal-sized intervals of the prosthesis velocity. To validate this approach, 20 non-disabled subjects performed force-matching tasks using Michelangelo prosthesis with and without EMG biofeedback with linear and nonlinear mapping. Additionally, four transradial amputees performed a functional task in the same feedback and mapping conditions. The success rate in producing desired force was significantly higher with feedback (65.4±15.9%) compared to no feedback (46.2±14.9%) as well as when using nonlinear (62.4±16.8%) versus linear mapping (49.2±17.2%). Overall, in non-disabled subjects, the highest success rate was obtained when EMG biofeedback was combined with nonlinear mapping (72%), and the opposite for linear mapping with no feedback (39.6%). The same trend was registered also in four amputee subjects. Therefore, EMG biofeedback improved prosthesis force control, especially when combined with nonlinear mapping, which showed to be an effective approach to counteract increasing variability of myoelectric signal for stronger contractions.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Proprioception: A New Era Set in Motion by Emerging Genetic and Bionic Strategies?

The generation of an internal body model and its continuous update is essential in sensorimotor control. Although known to rely on proprioceptive sensory feedback, the underlying mechanism that transforms this sensory feedback into a dynamic body percept remains poorly understood. However, advances in the development of genetic tools for proprioceptive circuit elements, including the sensory receptors, are beginning to offer new and unprecedented leverage to dissect the central pathways responsible for proprioceptive encoding. Simultaneously, new data derived through emerging bionic neural machine-interface technologies reveal clues regarding the relative importance of kinesthetic sensory feedback and insights into the functional proprioceptive substrates that underlie natural motor behaviors.

Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis

Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.

Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach

Introduction

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction.

Methods

Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees.

Results

The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).

Evaluating the effect of non-invasive force feedback on prosthetic grasp force modulation in participants with and without limb loss

Grasping an object is one of the most common and complex actions performed by humans. The human brain can adapt and update the grasp dynamics through information received from sensory feedback. Prosthetic hands can assist with the mechanical performance of grasping, however currently commercially available prostheses do not address the disruption of the sensory feedback loop. Providing feedback about a prosthetic hand's grasp force magnitude is a top priority for those with limb loss. This study tested a wearable haptic system, i.e., the Clenching Upper-Limb Force Feedback device (CUFF), which was integrated with a novel robotic hand (The SoftHand Pro). The SoftHand Pro was controlled with myoelectrics of the forearm muscles. Five participants with limb loss and nineteen able-bodied participants completed a constrained grasping task (with and without feedback) which required modulation of the grasp to reach a target force. This task was performed while depriving participants of incidental sensory sources (vision and hearing were significantly limited with glasses and headphones). The data were analyzed with Functional Principal Component Analysis (fPCA). CUFF feedback improved grasp precision for participants with limb loss who typically use body-powered prostheses as well as a sub-set of able-bodied participants. Further testing, that is more functional and allows participants to use all sensory sources, is needed to determine if CUFF feedback can accelerate mastery of myoelectric control or would benefit specific patient sub-groups.

EMG feedback outperforms force feedback in the presence of prosthesis control disturbance

Closing the prosthesis control loop by providing artificial somatosensory feedback can improve utility and user experience. Additionally, closed-loop control should be more robust with respect to disturbance, but this might depend on the type of feedback provided. Thus, the present study investigates and compares the performance of EMG and force feedback in the presence of control disturbances. Twenty able-bodied subjects and one transradial amputee performed delicate and power grasps with a prosthesis in a functional task, while the control signal gain was temporarily increased (high-gain disturbance) or decreased (low-gain disturbance) without their knowledge. Three outcome measures were considered: the percentage of trials successful in the first attempt (reaction to disturbance), the average number of attempts in trials where the wrong force was initially applied (adaptation to disturbance), and the average completion time of the last attempt in every trial. EMG feedback was shown to offer significantly better performance compared to force feedback during power grasping in terms of reaction to disturbance and completion time. During power grasping with high-gain disturbance, the median first-attempt success rate was significantly higher with EMG feedback (73.3%) compared to that achieved with force feedback (60%). Moreover, the median completion time for power grasps with low-gain disturbance was significantly longer with force feedback than with EMG feedback (3.64 against 2.48 s, an increase of 32%). Contrary to our expectations, there was no significant difference between feedback types with regards to adaptation to disturbances and the two feedback types performed similarly in delicate grasps. The results indicated that EMG feedback displayed better performance than force feedback in the presence of control disturbances, further demonstrating the potential of this approach to provide a reliable prosthesis-user interaction.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Sensory substitution of elbow proprioception to improve myoelectric control of upper limb prosthesis: experiment on healthy subjects and amputees

BACKGROUND

Current myoelectric prostheses lack proprioceptive information and rely on vision for their control. Sensory substitution is increasingly developed with non-invasive vibrotactile or electrotactile feedback, but most systems are designed for grasping or object discriminations, and few were tested for online control in amputees. The objective of this work was evaluate the effect of a novel vibrotactile feedback on the accuracy of myoelectric control of a virtual elbow by healthy subjects and participants with an upper-limb amputation at humeral level.

METHODS

Sixteen, healthy participants and 7 transhumeral amputees performed myoelectric control of a virtual arm under different feedback conditions: vision alone (VIS), vibration alone (VIB), vision plus vibration (VIS + VIB), or no feedback at all (NO). Reach accuracy was evaluated by angular errors during discrete as well as back and forth movements. Healthy participants' workloads were assessed with the NASA-TLX questionnaire, and feedback conditions were ranked according to preference at the end of the experiment.

RESULTS

Reach errors were higher in NO than in VIB, indicating that our vibrotactile feedback improved performance as compared to no feedback. Conditions VIS and VIS+VIB display similar levels of performance and produced lower errors than in VIB. Vision remains therefore critical to maintain good performance, which is not ameliorated nor deteriorated by the addition of vibrotactile feedback. The workload associated with VIB was higher than for VIS and VIS+VIB, which did not differ from each other. 62.5% of healthy subjects preferred the VIS+VIB condition, and ranked VIS and VIB second and third, respectively.

CONCLUSION

Our novel vibrotactile feedback improved myoelectric control of a virtual elbow as compared to no feedback. Although vision remained critical, the addition of vibrotactile feedback did not improve nor deteriorate the control and was preferred by participants. Longer training should improve performances with VIB alone and reduce the need of vision for close-loop prosthesis control.

Preliminary Evaluation of the Effect of Mechanotactile Feedback Location on Myoelectric Prosthesis Performance Using a Sensorized Prosthetic Hand

A commonly cited reason for the high abandonment rate of myoelectric prostheses is a lack of grip force sensory feedback. Researchers have attempted to restore grip force sensory feedback by stimulating the residual limb’s skin surface in response to the prosthetic hand’s measured grip force. Recent work has focused on restoring natural feedback to the missing digits directly through invasive surgical procedures. However, the functional benefit of utilizing somatotopically matching feedback has not been evaluated. In this paper, we propose an experimental protocol centered on a fragile object grasp and lift task using a sensorized myoelectric prosthesis to evaluate sensory feedback techniques. We formalized a suite of outcome measures related to task success, timing, and strategy. A pilot study (n = 3) evaluating the effect of utilizing a somatotopically accurate feedback stimulation location in able-bodied participants was conducted to evaluate the feasibility of the standardized platform, and to inform future studies on the role of feedback stimulation location in prosthesis use. Large between-participant effect sizes were observed in all outcome measures, indicating that the feedback location likely plays a role in myoelectric prosthesis performance. The success rate decreased, and task timing and task focus metrics increased, when using somatotopically-matched feedback compared to non-somatotopically-matched feedback. These results were used to conduct a power analysis, revealing that a sample size of n = 8 would be sufficient to achieve significance in all outcome measures.

A Haptic Sleeve as a Method of Mechanotactile Feedback Restoration for Myoelectric Hand Prosthesis Users

Current myoelectric upper limb prostheses do not restore sensory feedback, impairing fine motor control. Mechanotactile feedback restoration with a haptic sleeve may rectify this problem. This randomised crossover within-participant controlled study aimed to assess a prototype haptic sleeve's effect on routine grasping tasks performed by eight able-bodied participants. Each participant completed 15 repetitions of the three tasks: Task 1—normal grasp, Task 2—strong grasp and Task 3—weak grasp, using visual, haptic, or combined feedback All data were collected in April 2021 in the Scottish Microelectronics Centre, Edinburgh, UK. Combined feedback correlated with significantly higher grasp success rates compared to the vision alone in Task 1 (p < 0.0001), Task 2 (p = 0.0057), and Task 3 (p = 0.0170). Similarly, haptic feedback was associated with significantly higher grasp success rates compared to vision in Task 1 (p < 0.0001) and Task 2 (p = 0.0015). Combined feedback correlated with significantly lower energy expenditure compared to visual feedback in Task 1 (p < 0.0001) and Task 3 (p = 0.0003). Likewise, haptic feedback was associated with significantly lower energy expenditure compared to the visual feedback in Task 1 (p < 0.0001), Task 2 (p < 0.0001), and Task 3 (p < 0.0001). These results suggest that mechanotactile feedback provided by the haptic sleeve effectively augments grasping and reduces its energy expenditure.

Conceptualization of a Sensory Feedback System in an Anthropomorphic Replacement Hand

(1) Background: This paper presents a conceptual design for an anthropomorphic replacement hand made of silicone that integrates a sensory feedback system. In combination with a motorized orthosis, it allows performing movements and registering information on the flexion and the pressure of the fingers. (2) Methods: To create the replacement hand, a three-dimensional (3D) scanner was used to scan the hand of the test person. With computer-aided design (CAD), a mold was created from the hand, then 3D-printed. Bending and force sensors were attached to the mold before silicone casting to implement the sensory feedback system. To achieve a functional and anthropomorphic appearance of the replacement hand, a material analysis was carried out. In two different test series, the properties of the used silicones were analyzed regarding their mechanical properties and the manufacturing process. (3) Results: Individual fingers and an entire hand with integrated sensors were realized, which demonstrated in several tests that sensory feedback in such an anthropomorphic replacement hand can be realized. Nevertheless, the choice of silicone material remains an open challenge, as there is a trade-off between the hardness of the material and the maximum mechanical force of the orthosis. (4) Conclusion: Apart from manufacturing-related issues, it is possible to cost-effectively create a personalized, anthropomorphic replacement hand, including sensory feedback, by using 3D scanning and 3D printing techniques.

Neurorobotic fusion of prosthetic touch, kinesthesia, and movement in bionic upper limbs promotes intrinsic brain behaviors

[Figure: see text].

The effect of calibration parameters on the control of a myoelectric hand prosthesis using EMG feedback

Objective.The implementation of somatosensory feedback in upper limb myoelectric prostheses is an important step towards the restoration of lost sensory-motor functions. EMG feedback is a recently proposed method for closing the control loop wherein the myoelectric signal that drives the prosthesis is also used to generate the feedback provided to the user. Therefore, the characteristics of the myoelectric signal (variability and sensitivity) are likely to significantly affect the ability of the subject to utilize this feedback for online control of the prosthesis.Approach.In the present study, we investigated how the cutoff frequency of the low-pass filter (0.5, 1 and 1.5 Hz) and normalization value (20%, 40% and 60% of the maximum voluntary contraction (MVC)), that are used for the generation of the myoelectric signal, affect the quality of closed-loop control with EMG feedback. Lower cutoff and normalization decrease the intrinsic variability of the EMG but also increase the time lag between the contraction and the feedback (cutoff) as well as the sensitivity of the myoelectric signal (normalization). Ten participants were asked to generate three grasp force levels with a myoelectric prosthetic hand, while receiving five-level vibrotactile EMG feedback, over nine experimental runs (all parameter combinations).Main results.The outcome measure was the success rate (SR) in achieving the appropriate level of myoelectric signal (primary outcome) and grasping force (secondary outcome). Overall, the experiments demonstrated that EMG feedback provided robust control across conditions. Nevertheless, the performance was significantly better for the lowest cutoff (0.5 Hz) and higher normalization (40% and 60%). The highest SR for the EMG was 71.9%, achieved in the condition (40% MVC and 0.5 Hz), and this was 24.1% higher than that in the condition (20% MVC and 1.5 Hz), which resulted in the lowest performance. The SR for the force followed a similar trend.Significance.This is the first study that systematically explored the parameter space for the calibration of EMG feedback, which is a critical step for the future clinical application of this approach.

Joint speed feedback improves myoelectric prosthesis adaptation after perturbed reaches in non amputees

Accurate control of human limbs involves both feedforward and feedback signals. For prosthetic arms, feedforward control is commonly accomplished by recording myoelectric signals from the residual limb to predict the user's intent, but augmented feedback signals are not explicitly provided in commercial devices. Previous studies have demonstrated inconsistent results when artificial feedback was provided in the presence of vision; some studies showed benefits, while others did not. We hypothesized that negligible benefits in past studies may have been due to artificial feedback with low precision compared to vision, which results in heavy reliance on vision during reaching tasks. Furthermore, we anticipated more reliable benefits from artificial feedback when providing information that vision estimates with high uncertainty (e.g. joint speed). In this study, we test an artificial sensory feedback system providing joint speed information and how it impacts performance and adaptation during a hybrid positional-and-myoelectric ballistic reaching task. We found that overall reaching errors were reduced after perturbed control, but did not significantly improve steady-state reaches. Furthermore, we found that feedback about the joint speed of the myoelectric prosthesis control improved the adaptation rate of biological limb movements, which may have resulted from high prosthesis control noise and strategic overreaching with the positional control and underreaching with the myoelectric control. These results provide insights into the relevant factors influencing the improvements conferred by artificial sensory feedback.

Online Closed-Loop Control Using Tactile Feedback Delivered Through Surface and Subdermal Electrotactile Stimulation

Aim

Limb loss is a dramatic event with a devastating impact on a person’s quality of life. Prostheses have been used to restore lost motor abilities and cosmetic appearance. Closing the loop between the prosthesis and the amputee by providing somatosensory feedback to the user might improve the performance, confidence of the amputee, and embodiment of the prosthesis. Recently, a minimally invasive method, in which the electrodes are placed subdermally, was presented and psychometrically evaluated. The present study aimed to assess the quality of online control with subdermal stimulation and compare it to that achieved using surface stimulation (common benchmark) as well as to investigate the impact of training on the two modalities.

Methods

Ten able-bodied subjects performed a PC-based compensatory tracking task. The subjects employed a joystick to track a predefined pseudorandom trajectory using feedback on the momentary tracking error, which was conveyed via surface and subdermal electrotactile stimulation. The tracking performance was evaluated using the correlation coefficient (CORR), root mean square error (RMSE), and time delay between reference and generated trajectories.

Results

Both stimulation modalities resulted in good closed-loop control, and surface stimulation outperformed the subdermal approach. There was significant difference in CORR (86 vs 77%) and RMSE (0.23 vs 0.31) between surface and subdermal stimulation (all p < 0.05). The RMSE of the subdermal stimulation decreased significantly in the first few trials.

Conclusion

Subdermal stimulation is a viable method to provide tactile feedback. The quality of online control is, however, somewhat worse compared to that achieved using surface stimulation. Nevertheless, due to minimal invasiveness, compactness, and power efficiency, the subdermal interface could be an attractive solution for the functional application in sensate prostheses.

Prosthesis embodiment and attenuation of prosthetic touch in upper limb amputees - A proof-of-concept study

Sensory attenuation of self-touch, that is, the perceptual reduction of self-generated tactile stimuli, is considered a neurocognitive basis for self-other distinction. However, whether this effect can also be found in upper limb amputees using a prosthesis is unknown. Thirteen participants were asked to touch their foot sole with a) their intact hand (self-touch), b) their prosthesis (prosthesis-touch), or c) let it be touched by another person (other-touch). Intensity of touch was assessed with a questionnaire. In addition, prosthesis embodiment was assessed in nine participants. Self-touch as well as prosthesis-touch was characterized by significant perceptual attenuation compared to other-touch, while self- and prosthesis-touch did not differ. The more embodied the prosthesis was, the more similar was its elicited touch perception to actual self-touch. These findings - although preliminary - suggest that perceptually embodied prostheses can be represented as an actual limb by the users' sensorimotor system.

Information about contact force and surface texture is mixed in the firing rates of cutaneous afferent neurons

Cutaneous mechanoreceptors in our hands gather information about the objects we handle. Tactile fibers encode mixed information about contact events and object properties. Neural coding in tactile afferents is typically studied by varying a single aspect of tactile stimuli, avoiding the confounds of real-world haptic interactions. We instead record responses of small populations of dorsal root ganglia (DRG) neurons to variable tactile stimuli and find that neurons primarily respond to force, though some texture information can be detected. Tactile nerve fibers convey information about many features of haptic interactions, including the force and speed of contact, as well as the texture and shape of the objects being handled. How we perceive these object features is relatively unaffected by the forces and movements we use when interacting with the object. Because signals related to contact events and object properties are mixed in the responses of tactile fibers, our ability to disentangle these different components of our tactile experience implies that they are demultiplexed as they propagate along the neuraxis. To understand how texture and contact mechanics are encoded together by tactile fibers, we studied the activity of multiple neurons recorded simultaneously in the cervical DRG of two anesthetized rhesus monkeys while textured surfaces were applied to the glabrous skin of the fingers and palm using a handheld probe. A transducer at the tip of the textured probe measured contact forces as tactile stimuli were applied at different locations on the finger-pads and palm. We examined how a sample population of DRG neurons encode force and texture and found that firing rates of individual neurons are modulated by both force and texture. In particular, slowly adapting (SA) neurons were more responsive to force than texture, and rapidly adapting (RA) neurons were more responsive to texture than force. Although force could be decoded accurately throughout the entire contact interval, texture signals were most salient during onset and offset phases of the contact interval.NEW & NOTEWORTHY Cutaneous mechanoreceptors in our hands gather information about the objects we handle. Tactile fibers encode mixed information about contact events and object properties. Neural coding in tactile afferents is typically studied by varying a single aspect of tactile stimuli, avoiding the confounds of real-world haptic interactions. We instead record responses of small populations of DRG neurons to variable tactile stimuli and find that neurons primarily respond to force, though some texture information can be detected.

Wearable multichannel haptic device for encoding proprioception in the upper limb

OBJECTIVE

We present the design, implementation, and evaluation of a wearable multichannel haptic system. The device is a wireless closed-loop armband driven by surface electromyography (EMG) and provides sensory feedback encoding proprioception. The study is motivated by restoring proprioception information in upper limb prostheses.

APPROACH

The armband comprises eight vibrotactile actuators that generate distributed patterns of mechanical waves around the limb to stimulate perception and to transfer proportional information on the arm motion. An experimental study was conducted to assess: the sensory threshold in eight locations around the forearm, the user adaptation to the sensation provided by the device, the user performance in discriminating multiple stimulation levels, and the device performance in coding proprioception using four spatial patterns of stimulation. Eight able-bodied individuals performed reaching tasks by controlling a cursor with an EMG interface in a virtual environment. Vibrotactile patterns were tested with and without visual information on the cursor position with the addition of a random rotation of the reference control system to disturb the natural control and proprioception.

MAIN RESULTS

The sensation threshold depended on the actuator position and increased over time. The maximum resolution for stimuli discrimination was four. Using this resolution, four patterns of vibrotactile activation with different spatial and magnitude properties were generated to evaluate their performance in enhancing proprioception. The optimal vibration pattern varied among the participants. When the feedback was used in closed-loop control with the EMG interface, the task success rate, completion time, execution efficiency, and average target-cursor distance improved for the optimal stimulation pattern compared to the condition without visual or haptic information on the cursor position.

SIGNIFICANCE

The results indicate that the vibrotactile device enhanced the participants' perceptual ability, suggesting that the proposed closed-loop system has the potential to code proprioception and enhance user performance in the presence of perceptual perturbation.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

Long-Term Home-Use of Sensory-Motor-Integrated Bidirectional Bionic Prosthetic Arms Promotes Functional, Perceptual, and Cognitive Changes

Cutaneous sensation is vital to controlling our hands and upper limbs. It helps close the motor control loop by informing adjustments of grasping forces during object manipulations and provides much of the information the brain requires to perceive our limbs as a part of our bodies. This sensory information is absent to upper-limb prosthesis users. Although robotic prostheses are becoming increasingly sophisticated, the absence of feedback imposes a reliance on open-loop control and limits the functional potential as an integrated part of the body. Experimental systems to restore physiologically relevant sensory information to prosthesis users are beginning to emerge. However, the impact of their long-term use on functional abilities, body image, and neural adaptation processes remains unclear. Understanding these effects is essential to transition sensate prostheses from sophisticated assistive tools to integrated replacement limbs. We recruited three participants with high-level upper-limb amputation who previously received targeted reinnervation surgery. Each participant was fit with a neural-machine-interface prosthesis that allowed participants to operate their device by thinking about moving their missing limb. Additionally, we fit a sensory feedback system that allowed participants to experience touch to the prosthesis as touch on their missing limb. All three participants performed a long-term take-home trial. Two participants used their neural-machine-interface systems with touch feedback and one control participant used his prescribed, insensate prosthesis. A series of functional outcome metrics and psychophysical evaluations were performed using sensate neural-machine-interface prostheses before and after the take-home period to capture changes in functional abilities, limb embodiment, and neural adaptation. Our results demonstrated that the relationship between users and sensate neural-machine-interface prostheses is dynamic and changes with long-term use. The presence of touch sensation had a near-immediate impact on how the users operated their prostheses. In the multiple independent measures of users’ functional abilities employed, we observed a spectrum of performance changes following long-term use. Furthermore, after the take-home period, participants more appropriately integrated their prostheses into their body images and psychophysical tests provided strong evidence that neural and cortical adaptation occurred.

Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback

Brain computer interface (BCI) adopts human brain signals to control external devices directly without using normal neural pathway. Recent study has explored many applications, such as controlling a teleoperation robot by electroencephalography (EEG) signals. However, utilizing the motor imagery EEG-based BCI to perform teleoperation for reach and grasp task still has many difficulties, especially in continuous multidimensional control of robot and tactile feedback. In this research, a motor imagery EEG-based continuous teleoperation robot control system with tactile feedback was proposed. Firstly, mental imagination of different hand movements was translated into continuous command to control the remote robotic arm to reach the hover area of the target through a wireless local area network (LAN). Then, the robotic arm automatically completed the task of grasping the target. Meanwhile, the tactile information of remote robotic gripper was detected and converted to the feedback command. Finally, the vibrotactile stimulus was supplied to users to improve their telepresence. Experimental results demonstrate the feasibility of using the motor imagery EEG acquired by wireless portable equipment to realize the continuous teleoperation robot control system to finish the reach and grasp task. The average two-dimensional continuous control success rates for online Task 1 and Task 2 of the six subjects were 78.0% ± 6.1% and 66.2% ± 6.0%, respectively. Furthermore, compared with the traditional EEG triggered robot control using the predefined trajectory, the continuous fully two-dimensional control can not only improve the teleoperation robot system’s efficiency but also give the subject a more natural control which is critical to human–machine interaction (HMI). In addition, vibrotactile stimulus can improve the operator’s telepresence and task performance.

The importance of visuo-motor coordination in upper limb rehabilitation after ischemic stroke by robotic therapy

Abstract Stroke is an acute hypoperfusion of cerebral parenchyma that most often leads to outstanding motor deficits that can last for the rest of the patient’s life. The purpose of the neurorehabilitation process is to limit, as far is possible for the motor deficits and to bring the patient to an independent life. A modern method consists in robotic neurorehabilitation which is more and more used, associated with functional electrical stimulation (FES). At the lower limb, the use of robotic rehabilitation associated with FES is already considered a success due to relatively stereotypical movements of the lower limb. In opposition, the upper limb is more difficult to rehabilitate due to its more complex movements. Therefore, eye-hand coordination (EHC) constitutes an important factor that is conditioning the rehabilitation progress. The eye-hand coordination can be brutally disturbed by stroke with critical consequences on motor-executive component. The EHC development depends on the interaction between a feedback complex and the prediction of the upper limb motility in the space, and requires the association between visual system, oculomotor system and hand motor system. We analyzed the stroke impact on this sensorial-motor functional integration and looked for a possible solution for the interruption of coordination between eyes and the movements of the superior limb. We consider that our study can contribute to a better understanding and to a faster rehabilitation of the motor deficit in the upper limb after stroke. Key words: stroke, rehabilitation, eye-hand coordination, robotic neurorehabilitation,

Inter-Limb Transfer of Grasp Force Perception With Closed-Loop Hand Prosthesis

Sensory feedback of grasp forces provides important information about physical interactions between the hand and objects, enabling both reactive and anticipatory neural control mechanisms. The numerous studies have shown artificial sensory feedback of various forms improves force control during grasping tasks by prosthetic hand users through a closed-feedback loop. However, little is known about how perceptual information is transferred between an intact limb and a closed-loop prosthetic limb, and the extent to which training inter-limb transfer may improve myoelectric prosthetic control. We addressed these gaps by using a contralateral force-matching task in which able-bodied participants were asked to generate grasp forces with their native hand, and then match it using the contralateral hand or a soft-synergy prosthetic hand worn on the contralateral arm that was coupled with a mechanotactile feedback device. We found that absolute matching error and matching time were greater when using the prosthetic system than the native hand. However, with contralateral specific training, subjects were able to produce similar relative matching error with the prosthetic system and the native hand, especially at the untrained force level. These findings suggest that an association can be established between the perception produced by the prosthetic limb and the contralateral intact limb, and provide novel insights about potential applications to training and design of the closed-loop prosthesis.

Applications of Future Technologies to Detect Skill Decay and Improve Procedural Performance

Medical simulation training has progressed in its use of incorporating various technologies to provide quality training interfaces from novices to experts. The purpose of this paper is to explore modeling, simulation and visualization training technology interfaces to improve precision learning, rigorous, objective assessment, and performance improvement feedback for clinical procedural skill training and sustainment. Technologies to include augmented reality (AR), haptic technology and computer vision will be defined and clarified. It is believed that by exploring the combination of using AR, haptics and computer vision technologies it is possible to develop a fully immersive learning system that can automate mentoring while detecting and measuring gross and fine motor skills. Such a system can be used to predict or delay the onset of skills decay (SD) by capturing rigorous, objective measures, and human performance metrics that can provide feedback to individual performers for skills improvement in real time.

Restoring Tactile sensations via neural interfaces for real-time force-and-slippage closed-loop control of bionic hands

Despite previous studies on the restoration of tactile sensation on the fingers and the hand, there are no examples of use of the routed sensory information to finely control the prosthesis hand in complex grasp and manipulation tasks. Here it is shown that force and slippage sensations can be elicited in an amputee subject by means of biologically-inspired slippage detection and encoding algorithms, supported by a stick-slip model of the performed grasp. A combination of cuff and intraneural electrodes was implanted for eleven weeks in a young woman with hand amputation, and was shown to provide close-to-natural force and slippage sensations, paramount for significantly improving the subject's manipulative skills with the prosthesis. Evidence is provided about the improvement of the subject's grasping and manipulation capabilities over time, thanks to neural feedback. The elicited tactile sensations enabled the successful fulfillment of fine grasp and manipulation tasks with increasing complexity. Grasp performance was quantitatively assessed by means of instrumented objects and a purposely developed metrics. Closed-loop control capabilities enabled by the neural feedback were compared to those achieved without feedback. Further, the work investigates whether the described amelioration of motor performance in dexterous tasks had as central neurophysiological correlates changes in motor cortex plasticity and whether such changes were of purely motor origin, or else the effect of a strong and persistent drive of the sensory feedback.

Joint Speed Discrimination and Augmentation For Prosthesis Feedback

Sensory feedback is critical in fine motor control, learning, and adaptation. However, robotic prosthetic limbs currently lack the feedback segment of the communication loop between user and device. Sensory substitution feedback can close this gap, but sometimes this improvement only persists when users cannot see their prosthesis, suggesting the provided feedback is redundant with vision. Thus, given the choice, users rely on vision over artificial feedback. To effectively augment vision, sensory feedback must provide information that vision cannot provide or provides poorly. Although vision is known to be less precise at estimating speed than position, no work has compared speed precision of biomimetic arm movements. In this study, we investigated the uncertainty of visual speed estimates as defined by different virtual arm movements. We found that uncertainty was greatest for visual estimates of joint speeds, compared to absolute rotational or linear endpoint speeds. Furthermore, this uncertainty increased when the joint reference frame speed varied over time, potentially caused by an overestimation of joint speed. Finally, we demonstrate a joint-based sensory substitution feedback paradigm capable of significantly reducing joint speed uncertainty when paired with vision. Ultimately, this work may lead to improved prosthesis control and capacity for motor learning.

Robust simultaneous myoelectric control of multiple degrees of freedom in wrist-hand prostheses by real-time neuromusculoskeletal modeling

OBJECTIVE

Robotic prosthetic limbs promise to replace mechanical function of lost biological extremities and restore amputees' capacity of moving and interacting with the environment. Despite recent advances in biocompatible electrodes, surgical procedures, and mechatronics, the impact of current solutions is hampered by the lack of intuitive and robust man-machine interfaces.

APPROACH

This work presents a biomimetic interface that synthetizes the musculoskeletal function of an individual's phantom limb as controlled by neural surrogates, i.e. electromyography-derived neural activations. With respect to current approaches based on machine learning, our method employs explicit representations of the musculoskeletal system to reduce the space of feasible solutions in the translation of electromyograms into prosthesis control commands. Electromyograms are mapped onto mechanical forces that belong to a subspace contained within the broader operational space of an individual's musculoskeletal system.

MAIN RESULTS

Our results show that this constraint makes the approach applicable to real-world scenarios and robust to movement artefacts. This stems from the fact that any control command must always exist within the musculoskeletal model operational space and be therefore physiologically plausible. The approach was effective both on intact-limbed individuals and a transradial amputee displaying robust online control of multi-functional prostheses across a large repertoire of challenging tasks.

SIGNIFICANCE

The development and translation of man-machine interfaces that account for an individual's neuromusculoskeletal system creates unprecedented opportunities to understand how disrupted neuro-mechanical processes can be restored or replaced via biomimetic wearable assistive technologies.

Design and Integration of an Inexpensive Wearable Mechanotactile Feedback System for Myoelectric Prostheses

The aim of this paper was to demonstrate the functionality of an inexpensive mechanotactile sensory feedback system for transhumeral myoelectric prostheses. We summarize the development of a tactile-integrated prosthesis, including 1) evaluation of sensors that were retrofit onto existing commercial terminal devices; 2) design of two custom mechanotactile tactors that were integrated into a socket without compromising suction suspension; 3) design of a modular controller which translated sensor input to tactor output, was wirelessly adjusted, and fit within a prosthetic forearm; and 4) evaluation of the system with a single transhumeral participant. Prosthesis functionality was demonstrated over three test sessions; the participant was able to identify tactor stimulation location and demonstrated a reduction in grasp force with the mechanotactile stimulation. This system offers an inexpensive and modular solution for integration of a mechanotactile sensory feedback system into a prosthetic socket without compromising the suction seal. These principles can be applied in future studies to investigate the direct impact of sensory feedback on tangible outcomes for prosthetic users, thereby reducing barriers to clinical translation.