Vibrotactile grasping force and hand aperture feedback for myoelectric forearm prosthesis users

A distributed, high-channel-count, implanted bidirectional system for restoration of somatosensation and myoelectric control

Objective. We intend to chronically restore somatosensation and provide high-fidelity myoelectric control for those with limb loss via a novel, distributed, high-channel-count, implanted system.Approach.We have developed the implanted Somatosensory Electrical Neurostimulation and Sensing (iSens®) system to support peripheral nerve stimulation through up to 64, 96, or 128 electrode contacts with myoelectric recording from 16, 8, or 0 bipolar sites, respectively. The rechargeable central device has Bluetooth® wireless telemetry to communicate to external devices and wired connections for up to four implanted satellite stimulation or recording devices. We characterized the stimulation, recording, battery runtime, and wireless performance and completed safety testing to support its use in human trials.Results.The stimulator operates as expected across a range of parameters and can schedule multiple asynchronous, interleaved pulse trains subject to total charge delivery limits. Recorded signals in saline show negligible stimulus artifact when 10 cm from a 1 mA stimulating source. The wireless telemetry range exceeds 1 m (direction and orientation dependent) in a saline torso phantom. The bandwidth supports 100 Hz bidirectional update rates of stimulation commands and data features or streaming select full bandwidth myoelectric signals. Preliminary first-in-human data validates the bench testing result.Significance.We developed, tested, and clinically implemented an advanced, modular, fully implanted peripheral stimulation and sensing system for somatosensory restoration and myoelectric control. The modularity in electrode type and number, including distributed sensing and stimulation, supports a wide variety of applications; iSens® is a flexible platform to bring peripheral neuromodulation applications to clinical reality. ClinicalTrials.gov ID NCT04430218.

Three Sliding Probes Placed on Forelimb Skin for Proprioceptive Feedback Differentially yet Complementarily Contribute to Hand Gesture Detection and Object-Size Discrimination

The purpose was to assess the effectiveness of three sliding tactile probes placed on the forelimb skin to provide proprioceptive feedback for the detection of hand gestures and discrimination of object size. Tactile contactors representing the first three fingers were driven along the proximodistal axis by linear servo motors. Twenty healthy subjects were involved in the gesture detection test, with 10 of them also participating in the object-size discrimination task. Motors were controlled by computer in the first four sessions of the gesture detection experiment, while the fifth session utilized a sensorized glove. Both the volar and dorsal sides of the forearm were examined. In the object-size discrimination experiment, the method was exclusively assessed on the volar surface under four distinct feedback conditions, including all fingers and each finger separately. The psychophysical data were further analyzed using a structural equation model (SEM) to evaluate the specific contributions of each individual contactor. Subjects consistently outperformed the chance level in detecting gestures. Performance improved up to the third session, with better results obtained on the volar side. The performances were similar in the fourth and fifth sessions. The just noticeable difference for achieving a 75% discrimination accuracy was found to be 2.90 mm of movement on the skin. SEM analysis indicated that the contactor for the index finger had the lowest importance in gesture detection, while it played a more significant role in object-size discrimination. However, all fingers were found to be significant predictors of subjects' responses in both experiments, except for the thumb, which was deemed insignificant in object-size discrimination. The study highlights the importance of considering the partial contribution of each degree of freedom in a sensory feedback system, especially concerning the task, when designing such systems.

Integration of proprioception in upper limb prostheses through non-invasive strategies: a review

Proprioception plays a key role in moving our body dexterously and effortlessly. Nevertheless, the majority of investigations evaluating the benefits of providing supplemental feedback to prosthetics users focus on delivering touch restitution. These studies evaluate the influence of touch sensation in an attempt to improve the controllability of current robotic devices. Contrarily, investigations evaluating the capabilities of proprioceptive supplemental feedback have yet to be comprehensively analyzed to the same extent, marking a major gap in knowledge within the current research climate. The non-invasive strategies employed so far to restitute proprioception are reviewed in this work. In the absence of a clearly superior strategy, approaches employing vibrotactile, electrotactile and skin-stretch stimulation achieved better and more consistent results, considering both kinesthetic and grip force information, compared with other strategies or any incidental feedback. Although emulating the richness of the physiological sensory return through artificial feedback is the primary hurdle, measuring its effects to eventually support the integration of cumbersome and energy intensive hardware into commercial prosthetic devices could represent an even greater challenge. Thus, we analyze the strengths and limitations of previous studies and discuss the possible benefits of coupling objective measures, like neurophysiological parameters, as well as measures of prosthesis embodiment and cognitive load with behavioral measures of performance. Such insights aim to provide additional and collateral outcomes to be considered in the experimental design of future investigations of proprioception restitution that could, in the end, allow researchers to gain a more detailed understanding of possibly similar behavioral results and, thus, support one strategy over another.

Encoding force modulation in two electrotactile feedback parameters strengthens sensory integration according to maximum likelihood estimation

Bidirectional human-machine interfaces involve commands from the central nervous system to an external device and feedback characterizing device state. Such feedback may be elicited by electrical stimulation of somatosensory nerves, where a task-relevant variable is encoded in stimulation amplitude or frequency. Recently, concurrent modulation in amplitude and frequency (multimodal encoding) was proposed. We hypothesized that feedback with multimodal encoding may effectively be processed by the central nervous system as two independent inputs encoded in amplitude and frequency, respectively, thereby increasing state estimate quality in accordance with maximum-likelihood estimation. Using an adaptation paradigm, we tested this hypothesis during a grasp force matching task where subjects received electrotactile feedback encoding instantaneous force in amplitude, frequency, or both, in addition to their natural force feedback. The results showed that adaptations in grasp force with multimodal encoding could be accurately predicted as the integration of three independent inputs according to maximum-likelihood estimation: amplitude modulated electrotactile feedback, frequency modulated electrotactile feedback, and natural force feedback (r² = 0.73). These findings show that multimodal electrotactile feedback carries an intrinsic advantage for state estimation accuracy with respect to single-variable modulation and suggest that this scheme should be the preferred strategy for bidirectional human-machine interfaces with electrotactile feedback.

Utility and Usability of Two Forms of Supplemental Vibrotactile Kinesthetic Feedback for Enhancing Movement Accuracy and Efficiency in Goal-Directed Reaching

Recent advances in wearable sensors and computing have made possible the development of novel sensory augmentation technologies that promise to enhance human motor performance and quality of life in a wide range of applications. We compared the objective utility and subjective user experience for two biologically inspired ways to encode movement-related information into supplemental feedback for the real-time control of goal-directed reaching in healthy, neurologically intact adults. One encoding scheme mimicked visual feedback encoding by converting real-time hand position in a Cartesian frame of reference into supplemental kinesthetic feedback provided by a vibrotactile display attached to the non-moving arm and hand. The other approach mimicked proprioceptive encoding by providing real-time arm joint angle information via the vibrotactile display. We found that both encoding schemes had objective utility in that after a brief training period, both forms of supplemental feedback promoted improved reach accuracy in the absence of concurrent visual feedback over performance levels achieved using proprioception alone. Cartesian encoding promoted greater reductions in target capture errors in the absence of visual feedback (Cartesian: 59% improvement; Joint Angle: 21% improvement). Accuracy gains promoted by both encoding schemes came at a cost in terms of temporal efficiency; target capture times were considerably longer (1.5 s longer) when reaching with supplemental kinesthetic feedback than without. Furthermore, neither encoding scheme yielded movements that were particularly smooth, although movements made with joint angle encoding were smoother than movements with Cartesian encoding. Participant responses on user experience surveys indicate that both encoding schemes were motivating and that both yielded passable user satisfaction scores. However, only Cartesian endpoint encoding was found to have passable usability; participants felt more competent using Cartesian encoding than joint angle encoding. These results are expected to inform future efforts to develop wearable technology to enhance the accuracy and efficiency of goal-directed actions using continuous supplemental kinesthetic feedback.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

Bio-robotics research for non-invasive myoelectric neural interfaces for upper-limb prosthetic control: a 10-year perspective review

A decade ago, a group of researchers from academia and industry identified a dichotomy between the industrial and academic state-of-the-art in upper-limb prosthesis control, a widely used bio-robotics application. They proposed that four key technical challenges, if addressed, could bridge this gap and translate academic research into clinically and commercially viable products. These challenges are unintuitive control schemes, lack of sensory feedback, poor robustness and single sensor modality. Here, we provide a perspective review on the research effort that occurred in the last decade, aiming at addressing these challenges. In addition, we discuss three research areas essential to the recent development in upper-limb prosthetic control research but were not envisioned in the review 10 years ago: deep learning methods, surface electromyogram decomposition and open-source databases. To conclude the review, we provide an outlook into the near future of the research and development in upper-limb prosthetic control and beyond.

Extended training improves the accuracy and efficiency of goal-directed reaching guided by supplemental kinesthetic vibrotactile feedback

Prior studies have shown that the accuracy and efficiency of reaching can be improved using novel sensory interfaces to apply task-specific vibrotactile feedback (VTF) during movement. However, those studies have typically evaluated performance after less than 1 h of training using VTF. Here, we tested the effects of extended training using a specific form of vibrotactile cues-supplemental kinesthetic VTF-on the accuracy and temporal efficiency of goal-directed reaching. Healthy young adults performed planar reaching with VTF encoding of the moving hand's instantaneous position, applied to the non-moving arm. We compared target capture errors and movement times before, during, and after approximately 10 h (20 sessions) of training on the VTF-guided reaching task. Initial performance of VTF-guided reaching showed that people were able to use supplemental VTF to improve reaching accuracy. Performance improvements were retained from one training session to the next. After 20 sessions of training, the accuracy and temporal efficiency of VTF-guided reaching were equivalent to or better than reaches performed with only proprioception. However, hand paths during VTF-guided reaching exhibited a persistent strategy where movements were decomposed into discrete sub-movements along the cardinal axes of the VTF display. We also used a dual-task condition to assess the extent to which performance gains in VTF-guided reaching resist dual-task interference. Dual-tasking capability improved over the 20 sessions, such that the primary VTF-guided reaching and a secondary choice reaction time task were performed with increasing concurrency. Thus, VTF-guided reaching is a learnable skill in young adults, who can achieve levels of accuracy and temporal efficiency equaling or exceeding those observed during movements guided only by proprioception. Future studies are warranted to explore learnability in older adults and patients with proprioceptive deficits, who might benefit from using wearable sensory augmentation technologies to enhance control of arm movements.

Neural evidence for functional roles of tactile and visual feedback in the application of myoelectric prosthesis

Objective. The primary purpose of this study was to investigate the electrophysiological mechanism underlying different modalities of sensory feedback and multi-sensory integration in typical prosthesis control tasks.Approach. We recruited 15 subjects and developed a closed-loop setup for three prosthesis control tasks which covered typical activities in the practical prosthesis application, i.e. prosthesis finger position control (PFPC), equivalent grasping force control (GFC) and box and block control (BABC). All the three tasks were conducted under tactile feedback (TF), visual feedback (VF) and tactile-visual feedback (TVF), respectively, with a simultaneous electroencephalography (EEG) recording to assess the electroencephalogram (EEG) response underlying different types of feedback. Behavioral and psychophysical assessments were also administered in each feedback condition.Results. EEG results showed that VF played a predominant role in GFC and BABC tasks. It was reflected by a significantly lower somatosensory alpha event-related desynchronization (ERD) in TVF than in TF and no significant difference in visual alpha ERD between TVF and VF. In PFPC task, there was no significant difference in somatosensory alpha ERD between TF and TVF, while a significantly lower visual alpha ERD was found in TVF than in VF, indicating that TF was essential in situations related to proprioceptive position perception. Tactile-visual integration was found when TF and VF were congruently implemented, showing an obvious activation over the premotor cortex in the three tasks. Behavioral and psychophysical results were consistent with EEG evaluations.Significance. Our findings could provide neural evidence for multi-sensory integration and functional roles of tactile and VF in a practical setting of prosthesis control, shedding a multi-dimensional insight into the functional mechanisms of sensory feedback.

Wrist speed feedback improves elbow compensation and reaching accuracy for myoelectric transradial prosthesis users in hybrid virtual reaching task

BACKGROUND

Myoelectric prostheses are a popular choice for restoring motor capability following the loss of a limb, but they do not provide direct feedback to the user about the movements of the device-in other words, kinesthesia. The outcomes of studies providing artificial sensory feedback are often influenced by the availability of incidental feedback. When subjects are blindfolded and disconnected from the prosthesis, artificial sensory feedback consistently improves control; however, when subjects wear a prosthesis and can see the task, benefits often deteriorate or become inconsistent. We theorize that providing artificial sensory feedback about prosthesis speed, which cannot be precisely estimated via vision, will improve the learning and control of a myoelectric prosthesis.

METHODS

In this study, we test a joint-speed feedback system with six transradial amputee subjects to evaluate how it affects myoelectric control and adaptation behavior during a virtual reaching task.

RESULTS

Our results showed that joint-speed feedback lowered reaching errors and compensatory movements during steady-state reaches. However, the same feedback provided no improvement when control was perturbed.

CONCLUSIONS

These outcomes suggest that the benefit of joint speed feedback may be dependent on the complexity of the myoelectric control and the context of the task.

Electromyography biofeedback system with visual and vibratory feedbacks designed for lower limb rehabilitation

Purpose

One of the main causes of long-term prosthetic abandonment is the lack of ownership over the prosthesis, which was caused mainly by the absence of sensory information regarding the lost limb. The period where the patient learns how to interact with a prosthetic device is critical in rehabilitation. This ideally happens within the first months after amputation, which is also a period associated with the consolidation of brain changes. Different studies have shown that the introduction of feedback mechanisms can be crucial to bypass the lack of sensorial information. To develop a biofeedback system for the rehabilitation of transfemoral amputees – controlled via electromyographic (EMG) activity from the leg muscles – that can provide real-time visual and/or vibratory feedback for the user.

Design/methodology/approach

The system uses surface EMG to control two feedback mechanisms, which are the knee joint of a prosthetic leg of a humanoid avatar in a virtual reality (VR) environment (visual feedback) and a matrix of 16 vibrotactile actuators placed in the back of the user (vibratory feedback). Data acquisition was inside a Faraday Cage using an OpenEphys® acquisition board for the surface EMG recordings. The tasks were performed on able-bodied participants, with no amputation, and for this, the dominant leg of the user was immobilized using an orthopedic boot fixed on the chair, allowing only isometric contractions of target muscles, according to the Surface EMG for Non-Invasive Assessment of Muscles (SENIAM) standard. The authors test the effectiveness of combining vibratory and visual feedback and how task difficulty affects overall performance.

Findings

The authors' results show no negative interference combining both feedback modalities and that performance peaked at the intermediate difficulty. These results provide powerful insights of what can be accomplished with the population of amputee people. By using this biofeedback system, the authors expect to engage another sensory modality in the process of spatial representation of a virtual leg, bypassing the lack of information associated with the disruption of afferent pathways following amputation.

Research limitations/implications

The authors developed a showcase with a new protocol and feedback mechanisms showing the protocol's safety, efficiency and reliability. However, since this system is designed for patients with leg amputation, the full extent of the effects of the biofeedback training can only be assessed after the evaluation with the amputees, and the results obtained so far establish a safe and operational protocol to accomplish this.

Practical implications

In this study, the authors proposed a new biofeedback device intended to be used in the preprosthetic rehabilitation phase for people with transfemoral amputation. With this new system, the authors propose a mechanism to bypass the lack of sensory information from a virtual prosthesis and help to assimilate visual and vibrotactile stimuli as a cue for movement representation.

Social implications

With this new system, the authors propose a mechanism to bypass the lack of sensory information from a virtual prosthesis and help to assimilate visual and vibrotactile stimuli as a cue for movement representation.

Originality/value

The authors' results show that all users were capable of recognizing both feedback modalities, both separate and combined, being able to respond accordingly throughout the tasks. The authors also show that for a one-session protocol, the last difficulty level imposed a greater challenge for most users, explained by the significant drop in performance disregarding the feedback modality. Lastly, the authors believe this paradigm can provide a better process for the embodiment of prosthetic devices, fulfilling the lack of sensory information for the users.

Substitutive proprioception feedback of a prosthetic wrist by electrotactile stimulation

Objective

Sensory feedback of upper-limb prostheses is widely desired and studied. As important components of proprioception, position, and movement feedback help users to control prostheses better. Among various feedback methods, electrotactile stimulation is a potential method for coding proprioceptive information of a prosthesis. This study was motivated by the need for proprioception information for a prosthetic wrist. The flexion-extension (FE) position and movement information of the prosthetic wrist are transmitted back to the human body through multichannel electrotactile stimulation.

Approach

We developed an electrotactile scheme to encode the FE position and movement of the prosthetic wrist and designed an integrated experimental platform. A preliminary experiment on the sensory threshold and discomfort threshold was performed. Then, two proprioceptive feedback experiments were performed: a position sense experiment (Exp 1) and a movement sense experiment (Exp 2). Each experiment included a learning session and a test session. The success rate (SR) and discrimination reaction time (DRT) were analyzed to evaluate the recognition effect. The acceptance of the electrotactile scheme was evaluated by a questionnaire.

Main results

Our results showed that the average position SRs of five able-bodied subjects, amputee 1, and amputee 2 were 83.78, 97.78, and 84.44%, respectively. The average movement SR, and the direction and range SR of wrist movement in five able-bodied subjects were 76.25, 96.67%, respectively. Amputee 1 and amputee 2 had movement SRs of 87.78 and 90.00% and direction and range SRs of 64.58 and 77.08%, respectively. The average DRT of five able-bodied subjects was less than 1.5 s and that of amputees was less than 3.5 s.

Conclusion

The results indicate that after a short period of learning, the subjects can sense the position and movement of wrist FE. The proposed substitutive scheme has the potential for amputees to sense a prosthetic wrist, thus enhancing the human-machine interaction.

Human–Machine Interaction through Advanced Haptic Sensors: A Piezoelectric Sensory Glove with Edge Machine Learning for Gesture and Object Recognition

Human–machine interaction (HMI) refers to systems enabling communication between machines and humans. Systems for human–machine interfaces have advanced significantly in terms of materials, device design, and production methods. Energy supply units, logic circuits, sensors, and data storage units must be flexible, stretchable, undetectable, biocompatible, and self-healing to act as human–machine interfaces. This paper discusses the technologies for providing different haptic feedback of different natures. Notably, the physiological mechanisms behind touch perception are reported, along with a classification of the main haptic interfaces. Afterward, a comprehensive overview of wearable haptic interfaces is presented, comparing them in terms of cost, the number of integrated actuators and sensors, their main haptic feedback typology, and their future application. Additionally, a review of sensing systems that use haptic feedback technologies—specifically, smart gloves—is given by going through their fundamental technological specifications and key design requirements. Furthermore, useful insights related to the design of the next-generation HMI devices are reported. Lastly, a novel smart glove based on thin and conformable AlN (aluminum nitride) piezoelectric sensors is demonstrated. Specifically, the device acquires and processes the signal from the piezo sensors to classify performed gestures through an onboard machine learning (ML) algorithm. Then, the design and testing of the electronic conditioning section of AlN-based sensors integrated into the smart glove are shown. Finally, the architecture of a wearable visual-tactile recognition system is presented, combining visual data acquired by a micro-camera mounted on the user’s glass with the haptic ones provided by the piezoelectric sensors.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Psychophysical Evaluation of Proprioceptive Feedback Through a Probe Sliding on the Forearm Skin of Healthy Humans

We have tested the forearm skin of humans as a target organ to deliver proprioceptive feedback via a tactile sensory substitution method. In the proposed method, a contactor probe was actuated by a linear servo motor and moved on the skin in proximo-distal axis depending on the angle of a virtual joint moving on a 180° arc. Twenty healthy subjects were tested to stop the joint at a given target under no-feedback, visual feedback, and tactile (dorsal and volar) feedback conditions. The absolute difference between the target and the response angle was recorded. Tests were repeated 4 times with ~ 1-week intervals. Two joint movement speeds were tested. The subjects performed best with visual feedback, and worst if no feedback was provided. Their performances with tactile feedback were not as good as in the visual feedback condition, but better than in the no-feedback condition. Subjects equally performed with volar and dorsal tactile feedback. The movement speed had no significant effects on tactile feedback. The performance improved with training only in tactile feedback conditions. The proprioceptive information from a motorized prosthesis can be provided through probes moving on the forearm skin, while the efficacy of the feedback may improve with extensive training.

Alpha-band activity in parietofrontal cortex predicts future availability of vibrotactile feedback in prosthesis use

Prosthesis disuse and abandonment is an ongoing issue in upper-limb amputation. In addition to lost structural and motor function, amputation also results in decreased task-specific sensory information. One proposed remedy is augmenting somatosensory information using vibrotactile feedback to provide tactile feedback of grasping objects. While the role of frontal and parietal areas in motor tasks is well established, the neural and kinematic effects of this augmented vibrotactile feedback remain in question. In this study, we sought to understand the neurobehavioral effects of providing augmented feedback during a reach-grasp-transport task. Ten persons with sound limbs performed a motor task while wearing a prosthesis simulator with and without vibrotactile feedback. We hypothesized that providing vibrotactile feedback during prosthesis use would increase activity in frontal and parietal areas and improve grasp-related behavior. Results show that anticipation of upcoming vibrotactile feedback may be encoded in motor and parietal areas during the reach-to-grasp phase of the task. While grasp aperture is unaffected by vibrotactile feedback, the availability of vibrotactile feedback does lead to a reduction in velocity during object transport. These results help shed light on how engineered feedback is utilized by prostheses users and provide methodologies for further assessment in advanced prosthetics research.

Towards optimizing the non-invasive sensory feedback interfaces in a neural prosthetic control

Objective. The somatotopic interface (SI) and non-somatotopic interface (NI) are commonly used to provide non-invasive sensory feedback. Nevertheless, differences between SI and NI are rarely reported, and objective evaluations of the corresponding brain response are missing as well. Few studies have reported how to design the stimulation encoding based on the two interfaces. The objective of this study was to investigate the difference in sensory characteristics between SI and NI, and propose an optimal encoding method for non-invasive feedback interfaces. Approach. We recruited seven amputees and compared the tactile sensitivity to stimulated positions and intensities between SI (phantom finger area) and NI (upper arm) in a tactile discrimination task. Electroencephalography (EEG) evaluation task was subsequently conducted to objectively evaluate the stimulus-evoked brain response. Finally, the two kinds of tactile information (stimulated position and intensity) was applied to an object recognition task. Specifically, the object size was reflected by the prosthetic finger position through stimulated position encoding, and the object stiffness was reflected by the contact force of prosthetic fingers through stimulated intensity encoding. We compared the performance under four feedback conditions (combinations between two kinds of tactile information and two interfaces). Results. Behavioral results showed that NI was more sensitive to position information while SI was more sensitive to intensity information. EEG results were consistent with behavioral results, showing a higher sensitivity of sensory alpha ERD for NI in the position discrimination, while the trend was opposite in the intensity discrimination. The feedback encoding allowed amputees to discriminate the size and stiffness of nine objects with the best performance of 62% overall accuracy (84% for size discrimination, 71% for stiffness discrimination) when position and intensity information was delivered on the NI and SI, respectively. Signicance. Our results provided an instructive strategy for sensory feedback via non-invasive solutions.

Object Recognition via Evoked Sensory Feedback during Control of a Prosthetic Hand

Haptic and proprioceptive feedback is critical for sensorimotor integration when we use our hand to perform daily tasks. Here, we evaluated how externally evoked haptic and proprioceptive feedback and myoelectric control strategies affected the recognition of object properties when participants controlled a prosthetic hand. Fingertip haptic sensation was elicited using a transcutaneous nerve stimulation grid to encode the prosthetic's fingertip forces. An array of tactors elicited patterned vibratory stimuli to encode tactile-proprioceptive kinematic information of the prosthetic finger joint. Myoelectric signals of the finger flexor and extensor were used to control the position or velocity of joint angles of the prosthesis. Participants were asked to perform object property (stiffness and size) recognition, by controlling the prosthetic hand with concurrent haptic and tactile-proprioceptive feedback. With the evoked feedback, intact and amputee participants recognized the object stiffness and size at success rates ranging from 50% to 100% in both position and velocity control with no significant difference across control schemes. Our findings show that evoked somatosensory feedback in a non-invasive manner can facilitate closed-loop control of the prosthetic hand and allowed for simultaneous recognition of different object properties. The outcomes can facilitate our understanding on the role of sensory feedback during bidirectional human-machine interactions, which can potentially promote user experience in object interactions using prosthetic hands.

Closed-loop control of a prosthetic finger via evoked proprioceptive information

Objective.Proprioceptive information plays an important role for recognizing and coordinating our limb's static and dynamic states relative to our body or the environment. In this study, we determined how artificially evoked proprioceptive feedback affected the continuous control of a prosthetic finger.Approach.We elicited proprioceptive information regarding the joint static position and dynamic movement of a prosthetic finger via a vibrotactor array placed around the subject's upper arm. Myoelectric signals of the finger flexor and extensor muscles were used to control the prosthesis, with or without the evoked proprioceptive feedback. Two control modes were evaluated: the myoelectric signal amplitudes were continuously mapped to either the position or the velocity of the prosthetic joint.Main results.Our results showed that the evoked proprioceptive information improved the control accuracy of the joint angle, with comparable performance in the position- and velocity-control conditions. However, greater angle variability was prominent during position-control than velocity-control. Without the proprioceptive feedback, the position-control tended to show a smaller angle error than the velocity-control condition.Significance.Our findings suggest that closed-loop control of a prosthetic device can potentially be achieved using non-invasive evoked proprioceptive feedback delivered to intact participants. Moreover, the evoked sensory information was integrated during myoelectric control effectively for both control strategies. The outcomes can facilitate our understanding of the sensorimotor integration process during human-machine interactions, which can potentially promote fine control of prosthetic hands.

Static and dynamic proprioceptive recognition through vibrotactile stimulation

Objective.Proprioceptive information provides individuals with a sense of our limb's static position and dynamic movement. Impaired or a lack of such feedback can diminish our ability to perform dexterous motions with our biological limbs or assistive devices. Here we seek to determine whether both static and dynamic components of proprioception can be recognized using variation of the spatial and temporal components of vibrotactile feedback.Approach.An array of five vibrotactors was placed on the forearm of each subject. Each tactor was encoded to represent one of the five forearm postures. Vibratory stimulus was elicited to convey the static position and movement of the forearm. Four experimental blocks were performed to test each subject's recognition of a forearm's simulated static position, rotational amplitude, rotational amplitude and direction, and rotational speed.Main results.Our results showed that the subjects were able to perform proprioceptive recognition based on the delivered vibrotactile information. Specifically, rotational amplitude recognition resulted in the highest level of accuracy (99.0%), while the recognition accuracy of the static position and the rotational amplitude-direction was the lowest (91.7% and 90.8%, respectively). Nevertheless, all proprioceptive properties were perceived with >90% accuracy, indicating that the implemented vibrotactile encoding scheme could effectively provide proprioceptive information to the users.Significance.The outcomes suggest that information pertaining to static and dynamic aspects of proprioception can be accurately delivered using an array of vibrotactors. This feedback approach could be used to potentially evaluate the sensorimotor integration processes during human-machine interactions, and to improve sensory feedback in clinical populations with somatosensory impairments.

Joint speed feedback improves myoelectric prosthesis adaptation after perturbed reaches in non amputees

Accurate control of human limbs involves both feedforward and feedback signals. For prosthetic arms, feedforward control is commonly accomplished by recording myoelectric signals from the residual limb to predict the user's intent, but augmented feedback signals are not explicitly provided in commercial devices. Previous studies have demonstrated inconsistent results when artificial feedback was provided in the presence of vision; some studies showed benefits, while others did not. We hypothesized that negligible benefits in past studies may have been due to artificial feedback with low precision compared to vision, which results in heavy reliance on vision during reaching tasks. Furthermore, we anticipated more reliable benefits from artificial feedback when providing information that vision estimates with high uncertainty (e.g. joint speed). In this study, we test an artificial sensory feedback system providing joint speed information and how it impacts performance and adaptation during a hybrid positional-and-myoelectric ballistic reaching task. We found that overall reaching errors were reduced after perturbed control, but did not significantly improve steady-state reaches. Furthermore, we found that feedback about the joint speed of the myoelectric prosthesis control improved the adaptation rate of biological limb movements, which may have resulted from high prosthesis control noise and strategic overreaching with the positional control and underreaching with the myoelectric control. These results provide insights into the relevant factors influencing the improvements conferred by artificial sensory feedback.

Online Closed-Loop Control Using Tactile Feedback Delivered Through Surface and Subdermal Electrotactile Stimulation

Aim

Limb loss is a dramatic event with a devastating impact on a person’s quality of life. Prostheses have been used to restore lost motor abilities and cosmetic appearance. Closing the loop between the prosthesis and the amputee by providing somatosensory feedback to the user might improve the performance, confidence of the amputee, and embodiment of the prosthesis. Recently, a minimally invasive method, in which the electrodes are placed subdermally, was presented and psychometrically evaluated. The present study aimed to assess the quality of online control with subdermal stimulation and compare it to that achieved using surface stimulation (common benchmark) as well as to investigate the impact of training on the two modalities.

Methods

Ten able-bodied subjects performed a PC-based compensatory tracking task. The subjects employed a joystick to track a predefined pseudorandom trajectory using feedback on the momentary tracking error, which was conveyed via surface and subdermal electrotactile stimulation. The tracking performance was evaluated using the correlation coefficient (CORR), root mean square error (RMSE), and time delay between reference and generated trajectories.

Results

Both stimulation modalities resulted in good closed-loop control, and surface stimulation outperformed the subdermal approach. There was significant difference in CORR (86 vs 77%) and RMSE (0.23 vs 0.31) between surface and subdermal stimulation (all p < 0.05). The RMSE of the subdermal stimulation decreased significantly in the first few trials.

Conclusion

Subdermal stimulation is a viable method to provide tactile feedback. The quality of online control is, however, somewhat worse compared to that achieved using surface stimulation. Nevertheless, due to minimal invasiveness, compactness, and power efficiency, the subdermal interface could be an attractive solution for the functional application in sensate prostheses.

Amplitude versus spatially modulated electrotactile feedback for myoelectric control of two degrees of freedom

OBJECTIVE

Artificial proprioceptive feedback from a myoelectric prosthesis is an important aspect in enhancing embodiment and user satisfaction, possibly lowering the demand for visual attention while controlling a prosthesis in everyday tasks. Contemporary myoelectric prostheses are advanced mechatronic systems with multiple degrees of freedom, and therefore, to communicate the prosthesis state, the feedback interface needs to transmit several variables simultaneously. In the present study, two different configurations for conveying proprioceptive information of wrist rotation and hand aperture through multichannel electrotactile stimulation were developed and evaluated during online myoelectric control.

APPROACH

Myoelectric recordings were acquired from the dominant forearm and electrotactile stimulation was delivered on the non-dominant forearm using a compact interface. The first feedback configuration, which was based on spatial coding, transmitted the information using a moving tactile stimulus, whereas the second, amplitude-based configuration conveyed the position via sensation intensity. Thirteen able-bodied subjects used pattern classification-based myoelectric control with both feedback configurations to accomplish a target-reaching task.

MAIN RESULTS

High task performance (completion rate > 90%) was observed for both configurations, with no significant difference in completion rate, time to reach the target, distance error and path efficiency, respectively.

SIGNIFICANCE

Overall, the results demonstrated that both feedback configurations allowed subjects to perceive and interpret two feedback variables delivered simultaneously, despite using a compact stimulation interface. This is an encouraging result for the prospect of communicating the full state of a multifunctional hand prosthesis.

Sensory Feedback in Hand Prostheses: A Prospective Study of Everyday Use

Introduction

Sensory feedback in hand prostheses is lacking but wished for. Many amputees experience a phantom hand map on their residual forearm. When the phantom hand map is touched, it is experienced as touch on the amputated hand. A non-invasive sensory feedback system, applicable to existing hand prostheses, can transfer somatotopical sensory information via phantom hand map. The aim was to evaluate how forearm amputees experienced a non-invasive sensory feedback system used in daily life over a 4-week period.

Methods

This longitudinal cohort study included seven forearm amputees. A non-invasive sensory feedback system was used over 4 weeks. For analysis, a mixed method was used, including quantitative tests (ACMC, proprioceptive pointing task, questionnaire) and interviews. A directed content analysis with predefined categories sensory feedback from the prosthesis, agency, body ownership, performance in activity, and suggestions for improvements was applied.

Results

The results from interviews showed that sensory feedback was experienced as a feeling of touch which contributed to an experience of completeness. However, the results from the questionnaire showed that the sense of agency and performance remained unchanged or deteriorated. The ability to feel and manipulate small objects was difficult and a stronger feedback was wished for. Phantom pain was alleviated in four out of five patients.

Conclusion

This is the first time a non-invasive sensory feedback system for hand prostheses was implemented in the home environment. The qualitative and quantitative results diverged. The sensory feedback was experienced as a feeling of touch which contributed to a feeling of completeness, linked to body ownership. The qualitative result was not verified in the quantitative measurements.

Clinical Trial Registration

Name: Evaluation of a Non-invasive Sensory Feedback System in Hand Prostheses. Date of registration: March 15, 2019. Date the first participant was enrolled: April 1, 2015. ClinicalTrials.gov Identifier: NCT03876405 ORCID ID: https://orcid.org/0000-0002-4140-7478.

The Interaction Between Feedback Type and Learning in Routine Grasping With Myoelectric Prostheses

While prosthetic fitting after upper-limb loss allows for restoration of motor functions, it deprives the amputee of tactile sensations that are essential for grasp control in able-bodied subjects. Therefore, it is commonly assumed that restoring the force feedback would improve the control of prosthesis grasping force. However, the literature regarding the benefit of feedback is controversial. Here, we investigated how the type of feedback affects learning and steady-state performance of routine grasping with a prosthesis. The experimental task was to grasp an object using a prosthesis and generate a low or high target-force range (TFR), both initially unknown, in three feedback conditions: basic auditory feedback on task outcome, and additional visual or vibratory feedback on the force magnitude. The results demonstrated that the performance was rather good and stable for the low TFR, whereas it was substantially worse for the high TFR with a pronounced training effect. Surprisingly, learning curve and steady-state performance did not depend on the feedback condition. Hence, in the specific context of routing grasping with a prosthesis controlled via surface EMG, the basic feedback on task outcome was not outperformed by force-related end-of-trial feedback and hence seemed to be sufficient for accomplishing the task.This conclusion applies to the context of routine grasping using a myoelectric prosthesis with surface EMG electrodes, which means that the control signals are variable and the feedback is perceived and processed at the end of the trial (motor adaption).

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

Wearable Dual-Frequency Vibrotactile System for Restoring Force and Stiffness Perception

Recently, there has been substantial progress in the mechatronic design and myoelectric control of active prostheses. However, a significant unmet need is the lack of sensory feedback in commercial prostheses for upper-limb amputees. The lack of sensory perception impacts on the control performance and embodiment, determining relatively high rejection rates. Previous research has been conducted to evaluate various non-invasive substitutional sensory channels, mainly to regenerate haptic perception. However, providing sensory channels for stiffness perception has been much less explored. In this short paper, we propose a non-invasive wearable sensory armband, named vibrotactile frequency modulation (V-FM) system. Using the V-FM, we implement a closed-loop myocontrol system with force and stiffness perception. A user study was conducted based on a repetitive two-forced alternative choice discrimination test in six able-bodied participants. The study was designed according to the method of constant stimuli. Results showed that using the V-FM armband, the participants recovered sensation comparable (in terms of difference threshold) to the natural stiffness perception. This demonstrated the potential of the proposed V-FM armband in restoring haptic and stiffness perception non-invasively.

Psychometric characterization of incidental feedback sources during grasping with a hand prosthesis

Background

A prosthetic system should ideally reinstate the bidirectional communication between the user’s brain and its end effector by restoring both motor and sensory functions lost after an amputation. However, current commercial prostheses generally do not incorporate somatosensory feedback. Even without explicit feedback, grasping using a prosthesis partly relies on sensory information. Indeed, the prosthesis operation is characterized by visual and sound cues that could be exploited by the user to estimate the prosthesis state. However, the quality of this incidental feedback has not been objectively evaluated.

Methods

In this study, the psychometric properties of the auditory and visual feedback of prosthesis motion were assessed and compared to that of a vibro-tactile interface. Twelve able-bodied subjects passively observed prosthesis closing and grasping an object, and they were asked to discriminate (experiment I) or estimate (experiment II) the closing velocity of the prosthesis using visual (VIS), acoustic (SND), or combined (VIS + SND) feedback. In experiment II, the subjects performed the task also with a vibrotactile stimulus (VIB) delivered using a single tactor. The outcome measures for the discrimination and estimation experiments were just noticeable difference (JND) and median absolute estimation error (MAE), respectively.

Results

The results demonstrated that the incidental sources provided a remarkably good discrimination and estimation of the closing velocity, significantly outperforming the vibrotactile feedback. Using incidental sources, the subjects could discriminate almost the minimum possible increment/decrement in velocity that could be commanded to the prosthesis (median JND < 2% for SND and VIS + SND). Similarly, the median MAE in estimating the prosthesis velocity randomly commanded from the full working range was also low, i.e., approximately 5% in SND and VIS + SND.

Conclusions

Since the closing velocity is proportional to grasping force in state-of-the-art myoelectric prostheses, the results of the present study imply that the incidental feedback, when available, could be usefully exploited for grasping force control. Therefore, the impact of incidental feedback needs to be considered when designing a feedback interface in prosthetics, especially since the quality of estimation using supplemental sources (e.g., vibration) can be worse compared to that of the intrinsic cues.

Bidirectional Control of Myoelectric Prostheses in Upper Limb Amputees: Current Results and Expectations

The most common causes of upper-limb amputations include traumatic etiology and malignity, followed by peripheral vascular diseases. Prosthetic fitting along with conducting a rehabilitation program provide the greatest possible degree of independence to the upper-limb amputees in performing their daily tasks, occupational, recreational and work activities. Despite recent advance in strategies of design and control, the lack of sensory feedback is, according to patients, one of the most important characteristics lacked by commercial myoelectric prostheses. This reason has led to the need for the development of comprehensive prosthetic part which would provide intuitive control and realistic sensory feedback to the amputees enabling them thus to more easily accomplish the tasks which are essential for easier performance of activities of daily life. Electromyography, and recently, electroneurography signals have been used for the development of more efficacious upper-limb prosthetic control. Several recent studies have demonstrated the efficacy of homologous and somatotopic approach in upper-limb amputees, by applying implanted and surface electrodes. This work presents novel methods for effective bidirectional control of myoelectric prostheses in patients with upper-limb amputations using motor control and sensory feedback. The above-mentioned approaches are applicable and have good prospects in further clinical use. The intraneural, extraneural and surface approach can be more or less applicable depending on the etiology and the level of amputation. From a clinical point of view, various approaches should be combined for obtaining more efficient control of bidirectional prostheses and corresponding sensory feedback.

The Variability of Psychophysical Parameters Following Surface and Subdermal Stimulation: A Multiday Study in Amputees

Electrotactile stimulation has been suggested as a modality for providing sensory feedback in upper limb prostheses. This study investigates the multiday variability of subdermal and surface stimulation. Electrical stimulation was delivered using either surface or fine wire electrodes placed right under the skin in eight amputees for seven consecutive days. The variability of psychophysical measurements, including detection threshold (DT), pain threshold (PT), dynamic range (DR), just noticeable difference (JND), Weber fraction (WF) and quality of evoked sensations, was evaluated using the coefficient of variation (CoV). In addition, the systematic change in the mean of the parameters across days was assessed in both stimulation modalities. In the case of DT, PT, DR, and perceived intensity at 100 Hz, the CoV of surface stimulation was significantly smaller than that of subdermal stimulation. Only PT showed a significant systematic change in the mean value across days for both modalities. The outcome of this study has implications for the choice of modality in delivering sensory feedback, though the significance of the quantified variability needs to be evaluated using usability tests with user feedback.

Evaluation of Optimal Vibrotactile Feedback for Force-Controlled Upper Limb Myoelectric Prostheses

The main goal of this study is to evaluate how to optimally select the best vibrotactile pattern to be used in a closed loop control of upper limb myoelectric prostheses as a feedback of the exerted force. To that end, we assessed both the selection of actuation patterns and the effects of the selection of frequency and amplitude parameters to discriminate between different feedback levels. A single vibrotactile actuator has been used to deliver the vibrations to subjects participating in the experiments. The results show no difference between pattern shapes in terms of feedback perception. Similarly, changes in amplitude level do not reflect significant improvement compared to changes in frequency. However, decreasing the number of feedback levels increases the accuracy of feedback perception and subject-specific variations are high for particular participants, showing that a fine-tuning of the parameters is necessary in a real-time application to upper limb prosthetics. In future works, the effects of training, location, and number of actuators will be assessed. This optimized selection will be tested in a real-time proportional myocontrol of a prosthetic hand.

Electrical stimulation induces synaptic changes in the peripheral auditory system

Since a rapidly increasing number of neurostimulation devices are used clinically to modulate specific neural functions, the impact of electrical stimulation on targeted neural structure and function has become a key issue. In particular, the specific effect of electrical stimulation via a cochlear implant (CI) on inner hair cell (IHC) synapses remains unclear. Importantly, CI candidacy has recently expanded to include patients with partial hearing loss. Unfortunately, some CI recipients experience progressive hearing loss after activation of electrical stimulation. The mechanism(s) accounting for loss of residual hearing following electrical stimulation is unknown. Here normal-hearing guinea pigs were implanted with customized CIs. Intracochlear electrical stimulation with an intensity equal to or above electrically evoked compound action potential (ECAP) threshold decreased the excitability of auditory nerve. Furthermore, the number of synapses between IHCs and the afferent spiral ganglion neurons (SGNs) also decreased after electrical stimulation with higher intensities. However, no significant change was observed in the packing density and perikaryal area of SGNs as well as the quantity of hair cells. These results carry important implications for use of CIs in patients with residual hearing and for an increasing number of patients treated with other neurostimulation devices. Notably, the results were based on acute electrical stimulation. Considering the complex interaction between CIs and targeted tissues, it is urgent to conduct further research to clarify whether the similar changes could be induced by chronic electrical stimulation.

Spatial and temporal influences on discrimination of vibrotactile stimuli on the arm

Body-machine interfaces (BMIs) provide a non-invasive way to control devices. Vibrotactile stimulation has been used by BMIs to provide performance feedback to the user, thereby reducing visual demands. To advance the goal of developing a compact, multivariate vibrotactile display for BMIs, we performed two psychophysical experiments to determine the acuity of vibrotactile perception across the arm. The first experiment assessed vibration intensity discrimination of sequentially presented stimuli within four dermatomes of the arm (C5, C7, C8, and T1) and on the ulnar head. The second experiment compared vibration intensity discrimination when pairs of vibrotactile stimuli were presented simultaneously vs. sequentially within and across dermatomes. The first experiment found a small but statistically significant difference between dermatomes C7 and T1, but discrimination thresholds at the other three locations did not differ. Thus, while all tested dermatomes of the arm and hand could serve as viable sites of vibrotactile stimulation for a practical BMI, ideal implementations should account for small differences in perceptual acuity across dermatomes. The second experiment found that sequential delivery of vibrotactile stimuli resulted in better intensity discrimination than simultaneous delivery, independent of whether the pairs were located within the same dermatome or across dermatomes. Taken together, our results suggest that the arm may be a viable site to transfer multivariate information via vibrotactile feedback for body-machine interfaces. However, user training may be needed to overcome the perceptual disadvantage of simultaneous vs. sequentially presented stimuli.

Comparison of vibrotactile and joint-torque feedback in a myoelectric upper-limb prosthesis

BACKGROUND

Despite the technological advancements in myoelectric prostheses, body-powered prostheses remain a popular choice for amputees, in part due to the natural sensory advantage they provide. Research on haptic feedback in myoelectric prostheses has delivered mixed results. Furthermore, there is limited research comparing various haptic feedback modalities in myoelectric prostheses. In this paper, we present a comparison of the feedback intrinsically present in body-powered prostheses (joint-torque feedback) to a commonly proposed feedback modality for myoelectric prostheses (vibrotactile feedback). In so doing, we seek to understand whether the advantages of kinesthetic feedback present in body-powered prostheses translate to myoelectric prostheses, and whether there are differences between kinesthetic and cutaneous feedback in prosthetic applications.

METHODS

We developed an experimental testbed that features a cable-driven, voluntary-closing 1-DoF prosthesis, a capstan-driven elbow exoskeleton, and a vibrotactile actuation unit. The system can present grip force to users as either a flexion moment about the elbow or vibration on the wrist. To provide an equal comparison of joint-torque and vibrotactile feedback, a stimulus intensity matching scheme was utilized. Non-amputee participants (n=12) were asked to discriminate objects of varying stiffness with the prosthesis in three conditions: no haptic feedback, vibrotactile feedback, and joint-torque feedback.

RESULTS

Results indicate that haptic feedback increased discrimination accuracy over no haptic feedback, but the difference between joint-torque feedback and vibrotactile feedback was not significant. In addition, our results highlight nuanced differences in performance depending on the objects' stiffness, and suggest that participants likely pay less attention to incidental cues with the addition of haptic feedback.

CONCLUSION

Even when haptic feedback is not modality matched to the task, such as in the case of vibrotactile feedback, performance with a myoelectric prosthesis can improve significantly. This implies it is possible to achieve the same benefits with vibrotactile feedback, which is cheaper and easier to implement than other forms of feedback.

Analysis of Man-Machine Interfaces in Upper-Limb Prosthesis: A Review

This paper compiles and analyzes some of the most current works related to upper limb prosthesis with emphasis on man-machine interfaces. A brief introduction of the basic subjects is given to explain what a prosthesis is, what types of prostheses exist, what they serve for, how they communicate with the user (control and feedback), and what technologies are involved. The method used in this review is also discussed, as well as the cataloging process and analysis of articles for the composition of this review. Each article is analyzed individually and its results are presented in a succinct way, in order to facilitate future research and serve as a source for professionals related to the area of prosthesis, such as doctors, engineers, researchers, and anyone interested in this subject. Finally, the needs and difficulties of the current prostheses, as well as the negative and positive points in the results are analyzed, and the progress achieved so far is discussed.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices – such as prostheses, orthoses, exoskeletons, and devices for teleoperation–often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Myocontrol is closed-loop control: incidental feedback is sufficient for scaling the prosthesis force in routine grasping

BACKGROUND

Sensory feedback is critical for grasping in able-bodied subjects. Consequently, closing the loop in upper-limb prosthetics by providing artificial sensory feedback to the amputee is expected to improve the prosthesis utility. Nevertheless, even though amputees rate the prospect of sensory feedback high, its benefits in daily life are still very much debated. We argue that in order to measure the potential functional benefit of artificial sensory feedback, the baseline open-loop performance needs to be established.

METHODS

The myoelectric control of naïve able-bodied subjects was evaluated during modulation of electromyographic signals (EMG task), and grasping with a prosthesis (Prosthesis task). The subjects needed to activate the wrist flexor muscles and close the prosthesis to reach a randomly selected target level (routine grasping). To assess the baseline performance, the tasks were performed with a different extent of implicit feedback (proprioception, prosthesis motion and sound). Finally, the prosthesis task was repeated with explicit visual force feedback. The subjects' ability to scale the prosthesis command/force was assessed by testing for a statistically significant increase in the median of the generated commands/forces between neighboring levels. The quality of control was evaluated by computing the median absolute error (MAE) with respect to the target.

RESULTS

The subjects could successfully scale their motor commands and generated prosthesis forces across target levels in all tasks, even with the least amount of implicit feedback (only muscle proprioception, EMG task). In addition, the deviation of the generated commands/forces from the target levels decreased with additional feedback. However, the increase in implicit feedback, from proprioception to prosthesis motion and sound, seemed to have a more substantial effect than the final introduction of explicit feedback. Explicit feedback improved the performance mainly at the higher target-force levels.

CONCLUSIONS

The study establishes the baseline performance of myoelectric control and prosthesis grasping force. The results demonstrate that even without additional feedback, naïve subjects can effectively modulate force with good accuracy with respect to that achieved when increasing the amount of feedback information.

Effect of Dual Tasking on Vibrotactile Feedback Guided Reaching - a Pilot Study

Vibrotactile feedback (VTF) has been proposed as a non-invasive way to augment impaired or lost kinesthetic feedback in certain patient populations, thereby enhancing the real-time control of purposeful limb movements and quality of life. We used a dual tasking scenario to investigate the effects of cognitive load and short-term VTF training on VTF-guided reaching. Participants grasped the handle of a planar manipulandum with one hand and received VTF of its motion via a vibrotactile display attached to the non-moving arm. We asked participants to simultaneously perform VTF-guided reaching and a choice reaction time task both before and after training with VTF-guided reaching. Participants readily used VTF to guide goal-directed hand movements in the absence of visual feedback in the dual-task setting, even prior to training. This capability came at the cost of increased movement completion time. Short-term training on VTF-guided reaching induced significant improvements in target capture errors. Pre- and post-training comparisons of dual-task performance found training-related improvements in VTF-guided reach accuracy were resistant to dual-task interference. We found no training-related improvements in movement completion time or button press performance. These results indicate that VTF can be used to complete goal-directed reaches in a dual task situation, and that a single short bout of training sufficed for participants to begin the transition between the cognitive and associative phases of learning for the integration of VTF into the planning and ongoing control of reaching movements.

The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis

Background

To effectively replace the human hand, a prosthesis should seamlessly respond to user intentions but also convey sensory information back to the user. Restoration of sensory feedback is rated highly by the prosthesis users, and feedback is critical for grasping in able-bodied subjects. Nonetheless, the benefits of feedback in prosthetics are still debated. The lack of consensus is likely due to the complex nature of sensory feedback during prosthesis control, so that its effectiveness depends on multiple factors (e.g., task complexity, user learning).

Methods

We evaluated the impact of these factors with a longitudinal assessment in six amputee subjects, using a clinical setup (socket, embedded control) and a range of tasks (box and blocks, block turn, clothespin and cups relocation). To provide feedback, we have proposed a novel vibrotactile stimulation scheme capable of transmitting multiple variables from a multifunction prosthesis. The subjects wore a bracelet with four by two uniformly placed vibro-tactors providing information on contact, prosthesis state (active function), and grasping force. The subjects also completed a questionnaire for the subjective evaluation of the feedback.

Results

The tests demonstrated that feedback was beneficial only in the complex tasks (block turn, clothespin and cups relocation), and that the training had an important, task-dependent impact. In the clothespin relocation and block turn tasks, training allowed the subjects to establish successful feedforward control, and therefore, the feedback became redundant. In the cups relocation task, however, the subjects needed some training to learn how to properly exploit the feedback. The subjective evaluation of the feedback was consistently positive, regardless of the objective benefits. These results underline the multifaceted nature of closed-loop prosthesis control as, depending on the context, the same feedback interface can have different impact on performance. Finally, even if the closed-loop control does not improve the performance, it could be beneficial as it seems to improve the subjective experience.

Conclusions

Therefore, in this study we demonstrate, for the first time, the relevance of an advanced, multi-variable feedback interface for dexterous, multi-functional prosthesis control in a clinically relevant setting.

Electronic supplementary material

The online version of this article (10.1186/s12984-018-0371-1) contains supplementary material, which is available to authorized users.

Adding vibrotactile feedback to a myoelectric-controlled hand improves performance when online visual feedback is disturbed

We investigated whether adding vibrotactile feedback to a myoelectric-controlled hand, when visual feedback is disturbed, can improve performance during a functional test. For this purpose, able-bodied subjects, activating a myoelectric-controlled hand attached to their right hand performed the modified Box & Blocks test, grasping and manipulating wooden blocks over a partition. This was performed in 3 conditions, using a repeated-measures design: in full light, in a dark room where visual feedback was disturbed and no auditory feedback - one time with the addition of tactile feedback provided during object grasping and manipulation, and one time without any tactile feedback. The average time needed to transfer one block was measured, and an infrared camera was used to give information on the number of grasping errors during performance of the test. Our results show that when vibrotactile feedback was provided, performance time was reduced significantly, compared with when no vibrotactile feedback was available. Furthermore, the accuracy of grasping and manipulation was improved, reflected by significantly fewer errors during test performance. In conclusion, adding vibrotactile feedback to a myoelectric-controlled hand has positive effects on functional performance when visual feedback is disturbed. This may have applications to current myoelectric-controlled hands, as adding tactile feedback may help prosthesis users to improve their functional ability during daily life activities in different environments, particularly when limited visual feedback is available or desirable.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices - such as prostheses, orthoses, exoskeletons, and devices for teleoperation-often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review

For those individuals with upper-extremity amputation, a daily normal living activity is no longer possible or it requires additional effort and time. With the aim of restoring their sensory and motor functions, theoretical and technological investigations have been carried out in the field of neuroprosthetic systems. For transmission of sensory feedback, several interfacing modalities including indirect (non-invasive), direct-to-peripheral-nerve (invasive), and cortical stimulation have been applied. Peripheral nerve interfaces demonstrate an edge over the cortical interfaces due to the sensitivity in attaining cortical brain signals. The peripheral nerve interfaces are highly dependent on interface designs and are required to be biocompatible with the nerves to achieve prolonged stability and longevity. Another criterion is the selection of nerves that allows minimal invasiveness and damages as well as high selectivity for a large number of nerve fascicles. In this paper, we review the nerve-machine interface modalities noted above with more focus on peripheral nerve interfaces, which are responsible for provision of sensory feedback. The invasive interfaces for recording and stimulation of electro-neurographic signals include intra-fascicular, regenerative-type interfaces that provide multiple contact channels to a group of axons inside the nerve and the extra-neural-cuff-type interfaces that enable interaction with many axons around the periphery of the nerve. Section Current Prosthetic Technology summarizes the advancements made to date in the field of neuroprosthetics toward the achievement of a bidirectional nerve-machine interface with more focus on sensory feedback. In the Discussion section, the authors propose a hybrid interface technique for achieving better selectivity and long-term stability using the available nerve interfacing techniques.

Short- and Long-Term Learning of Feedforward Control of a Myoelectric Prosthesis with Sensory Feedback by Amputees

Human motor control relies on a combination of feedback and feedforward strategies. The aim of this study was to longitudinally investigate artificial somatosensory feedback and feedforward control in the context of grasping with myoelectric prosthesis. Nine amputee subjects performed routine grasping trials, with the aim to produce four levels of force during four blocks of 60 trials across five days. The electrotactile force feedback was provided in the second and third block using multipad electrode and spatial coding. The first baseline and last validation block (open-loop control) evaluated the effects of long- (across sessions) and short-term (within session) learning, respectively. The outcome measures were the absolute error between the generated and target force, and the number of force saturations. The results demonstrated that the electrotactile feedback improved the performance both within and across sessions. In the validation block, the performance did not significantly decrease and the quality of open-loop control (baseline) improved across days, converging to the performance characterizing closed-loop control. This paper provides important insights into the feedback and feedforward processes in prosthesis control, contributing to the better understanding of the role and design of feedback in prosthetic systems.

Supplemental vibrotactile feedback control of stabilization and reaching actions of the arm using limb state and position error encodings

Background

Deficits of kinesthesia (limb position and movement sensation) commonly limit sensorimotor function and its recovery after neuromotor injury. Sensory substitution technologies providing synthetic kinesthetic feedback might re-establish or enhance closed-loop control of goal-directed behaviors in people with impaired kinesthesia.

Methods

As a first step toward this goal, we evaluated the ability of unimpaired people to use vibrotactile sensory substitution to enhance stabilization and reaching tasks. Through two experiments, we compared the objective and subjective utility of two forms of supplemental feedback – limb state information or hand position error – to eliminate hand position drift, which develops naturally during stabilization tasks after removing visual feedback.

Results

Experiment 1 optimized the encoding of limb state feedback; the best form included hand position and velocity information, but was weighted much more heavily toward position feedback. Upon comparing optimal limb state feedback vs. hand position error feedback in Experiment 2, we found both encoding schemes capable of enhancing stabilization and reach performance in the absence of vision. However, error encoding yielded superior outcomes - objective and subjective - due to the additional task-relevant information it contains.

Conclusions

The results of this study have established the immediate utility and relative merits of two forms of vibrotactile kinesthetic feedback in enhancing stabilization and reaching actions performed with the arm and hand in neurotypical people. These findings can guide future development of vibrotactile sensory substitution technologies for improving sensorimotor function after neuromotor injury in survivors who retain motor capacity, but lack proprioceptive integrity in their more affected arm.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-017-0248-8) contains supplementary material, which is available to authorized users.

GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses

OBJECTIVE

Providing sensory feedback to the user of the prosthesis is an important challenge. The common approach is to use tactile stimulation, which is easy to implement but requires training and has limited information bandwidth. In this study, we propose an alternative approach based on augmented reality.

APPROACH

We have developed the GLIMPSE, a Google Glass application which connects to the prosthesis via a Bluetooth interface and renders the prosthesis states (EMG signals, aperture, force and contact) using augmented reality (see-through display) and sound (bone conduction transducer). The interface was tested in healthy subjects that used the prosthesis with (FB group) and without (NFB group) feedback during a modified clothespins test that allowed us to vary the difficulty of the task. The outcome measures were the number of unsuccessful trials, the time to accomplish the task, and the subjective ratings of the relevance of the feedback.

MAIN RESULTS

There was no difference in performance between FB and NFB groups in the case of a simple task (basic, same-color clothespins test), but the feedback significantly improved the performance in a more complex task (pins of different resistances). Importantly, the GLIMPSE feedback did not increase the time to accomplish the task. Therefore, the supplemental feedback might be useful in the tasks which are more demanding, and thereby less likely to benefit from learning and feedforward control. The subjects integrated the supplemental feedback with the intrinsic sources (vision and muscle proprioception), developing their own idiosyncratic strategies to accomplish the task.

SIGNIFICANCE

The present study demonstrates a novel self-contained, ready-to-deploy, wearable feedback interface. The interface was successfully tested and was proven to be feasible and functionally beneficial. The GLIMPSE can be used as a practical solution but also as a general and flexible instrument to investigate closed-loop prosthesis control.

High Cable Forces Deteriorate Pinch Force Control in Voluntary-Closing Body-Powered Prostheses

Background

It is generally asserted that reliable and intuitive control of upper-limb prostheses requires adequate feedback of prosthetic finger positions and pinch forces applied to objects. Body-powered prostheses (BPPs) provide the user with direct proprioceptive feedback. Currently available BPPs often require high cable operation forces, which complicates control of the forces at the terminal device. The aim of this study is to quantify the influence of high cable forces on object manipulation with voluntary-closing prostheses.

Method

Able-bodied male subjects were fitted with a bypass-prosthesis with low and high cable force settings for the prehensor. Subjects were requested to grasp and transfer a collapsible object as fast as they could without dropping or breaking it. The object had a low and a high breaking force setting.

Results

Subjects conducted significantly more successful manipulations with the low cable force setting, both for the low (33% more) and high (50%) object’s breaking force. The time to complete the task was not different between settings during successful manipulation trials.

Conclusion

High cable forces lead to reduced pinch force control during object manipulation. This implies that low cable operation forces should be a key design requirement for voluntary-closing BPPs.

Learning to use a body-powered prosthesis: changes in functionality and kinematics

BACKGROUND

Little is known about action-perception learning processes underlying prosthetic skills in body-powered prosthesis users. Body-powered prostheses are controlled through a harness connected by a cable that might provide for limited proprioceptive feedback. This study aims to test transfer of training basic tasks to functional tasks and to describe the changes over time in kinematics of basic tasks of novice body-powered prosthesis users.

METHODS

Thirty able-bodied participants and 17 controls participated in the study, using a body-powered prosthetic simulator. Participants in the training group were divided over four groups and practiced during a 2-week-period either direct grasping, indirect grasping, fixation, or a combination of these tasks. Deformable objects with different compliances had to be manipulated while kinematic variables and grip force control were assessed. Functional performance was measured with the Southampton Hand Assessment Procedure (SHAP) prior to and after the training sessions, and after 2 weeks and 3 months retention. The control group only performed the SHAP tests.

RESULTS

All four training groups and the control group improved on the SHAP, also after a period of non-use. Type of training had a small but significant influence on the improvements of the SHAP score. On a kinematic level movement times decreased and hook closing velocities increased over time. The indirect grasping group showed significantly shorter plateau times than the other training groups. Grip force control only improved a little over training.

CONCLUSIONS

Training action-perception couplings of body-powered prosthesis in basic tasks transferred to functional tasks and this lasted after a period of non-use. During training movement times decreased and the indirect grasping group showed advantages. It is advisable to start body-powered training with indirect grasping tasks but also to practice hook-object orientations.