Integrated and flexible multichannel interface for electrotactile stimulation

Encoding contact size using static and dynamic electrotactile finger stimulation: natural decoding vs. trained cues

Electrotactile stimulation through matrix electrodes is a promising technology to restore high-resolution tactile feedback in extended reality applications. One of the fundamental tactile effects that should be simulated is the change in the size of the contact between the finger and a virtual object. The present study investigated how participants perceive the increase of stimulation area when stimulating the index finger using static or dynamic (moving) stimuli produced by activating 1 to 6 electrode pads. To assess the ability to interpret the stimulation from the natural cues (natural decoding), without any prior training, the participants were instructed to draw the size of the stimulated area and identify the size difference when comparing two consecutive stimulations. To investigate if other "non-natural" cues can improve the size estimation, the participants were asked to enumerate the number of active pads following a training protocol. The results demonstrated that participants could perceive the change in size without prior training (e.g., the estimated area correlated with the stimulated area, p < 0.001; ≥ two-pad difference recognized with > 80% success rate). However, natural decoding was also challenging, as the response area changed gradually and sometimes in complex patterns when increasing the number of active pads (e.g., four extra pads needed for the statistically significant difference). Nevertheless, by training the participants to utilize additional cues the limitations of natural perception could be compensated. After the training, the mismatch in the activated and estimated number of pads was less than one pad regardless of the stimulus size. Finally, introducing the movement of the stimulus substantially improved discrimination (e.g., 100% median success rate to recognize ≥ one-pad difference). The present study, therefore, provides insights into stimulation size perception, and practical guidelines on how to modulate pad activation to change the perceived size in static and dynamic scenarios.

Exploration of sensations evoked during electrical stimulation of the median nerve at the wrist level

Objective.Nerve rehabilitation following nerve injury or surgery at the wrist level is a lengthy process during which not only peripheral nerves regrow towards receptors and muscles, but also the brain undergoes plastic changes. As a result, at the time when nerves reach their targets, the brain might have already allocated some of the areas within the somatosensory cortex that originally processed hand signals to some other regions of the body. The aim of this study is to show that it is possible to evoke a variety of somatotopic sensations related to the hand while stimulating proximally to the injury, therefore, providing the brain with the relevant inputs from the hand regions affected by the nerve damage.Approach.This study included electrical stimulation of 28 able-bodied participants where an electrode that acted as a cathode was placed above the Median nerve at the wrist level. The parameters of electrical stimulation, amplitude, frequency, and pulse shape, were modulated within predefined ranges to evaluate their influence on the evoked sensations.Main results.Using this methodology, the participants reported a wide variety of somatotopic sensations from the hand regions distal to the stimulation electrode.Significance.Furthermore, to propose an accelerated stimulation tuning procedure that could be implemented in a clinical protocol and/or standalone device for providing meaningful sensations to the somatosensory cortex during nerve regeneration, we trained machine-learning techniques using the gathered data to predict the location/area, naturalness, and sensation type of the evoked sensations following different stimulation patterns.

Integration of proprioception in upper limb prostheses through non-invasive strategies: a review

Proprioception plays a key role in moving our body dexterously and effortlessly. Nevertheless, the majority of investigations evaluating the benefits of providing supplemental feedback to prosthetics users focus on delivering touch restitution. These studies evaluate the influence of touch sensation in an attempt to improve the controllability of current robotic devices. Contrarily, investigations evaluating the capabilities of proprioceptive supplemental feedback have yet to be comprehensively analyzed to the same extent, marking a major gap in knowledge within the current research climate. The non-invasive strategies employed so far to restitute proprioception are reviewed in this work. In the absence of a clearly superior strategy, approaches employing vibrotactile, electrotactile and skin-stretch stimulation achieved better and more consistent results, considering both kinesthetic and grip force information, compared with other strategies or any incidental feedback. Although emulating the richness of the physiological sensory return through artificial feedback is the primary hurdle, measuring its effects to eventually support the integration of cumbersome and energy intensive hardware into commercial prosthetic devices could represent an even greater challenge. Thus, we analyze the strengths and limitations of previous studies and discuss the possible benefits of coupling objective measures, like neurophysiological parameters, as well as measures of prosthesis embodiment and cognitive load with behavioral measures of performance. Such insights aim to provide additional and collateral outcomes to be considered in the experimental design of future investigations of proprioception restitution that could, in the end, allow researchers to gain a more detailed understanding of possibly similar behavioral results and, thus, support one strategy over another.

The impact of walking on the perception of multichannel electrotactile stimulation in individuals with lower-limb amputation and able-bodied participants

BACKGROUND

One of the drawbacks of lower-limb prostheses is that they do not provide explicit somatosensory feedback to their users. Electrotactile stimulation is an attractive technology to restore such feedback because it enables compact solutions with multiple stimulation points. This allows stimulating a larger skin area to provide more information concurrently and modulate parameters spatially as well as in amplitude. However, for effective use, electrotactile stimulation needs to be calibrated and it would be convenient to perform this procedure while the subject is seated. However, amplitude and spatial perception can be affected by motion and/or physical coupling between the residual limb and the socket. In the present study, we therefore evaluated and compared the psychometric properties of multichannel electrotactile stimulation applied to the thigh/residual limb during sitting versus walking.

METHODS

The comprehensive assessment included the measurement of the sensation and discomfort thresholds (ST & DT), just noticeable difference (JND), number of distinct intervals (NDI), two-point discrimination threshold (2PD), and spatial discrimination performance (SD). The experiment involved 11 able-bodied participants (4 females and 7 males; 29.2 ± 3.8 years), 3 participants with transtibial amputation, and 3 participants with transfemoral amputation.

RESULTS

In able-bodied participants, the results were consistent for all the measured parameters, and they indicated that both amplitude and spatial perception became worse during walking. More specifically, ST and DT increased significantly during walking vs. sitting (2.90 ± 0.82 mA vs. 2.00 ± 0.52 mA; p < 0.001 for ST and 7.74 ± 0.84 mA vs. 7.21 ± 1.30 mA; p < 0.05 for DT) and likewise for the JND (22.47 ± 12.21% vs. 11.82 ± 5.07%; p < 0.01), while the NDI became lower (6.46 ± 3.47 vs. 11.27 ± 5.18 intervals; p < 0.01). Regarding spatial perception, 2PD was higher during walking (69.78 ± 17.66 mm vs. 57.85 ± 14.87 mm; p < 0.001), while the performance of SD was significantly lower (56.70 ± 10.02% vs. 64.55 ± 9.44%; p < 0.01). For participants with lower-limb amputation, the ST, DT, and performance in the SD assessment followed the trends observed in the able-bodied population. The results for 2PD and JND were however different and subject-specific.

CONCLUSION

The conducted evaluation demonstrates that electrotactile feedback should be calibrated in the conditions in which it will be used (e.g., during walking). The calibration during sitting, while more convenient, might lead to an overly optimistic (or in some cases pessimistic) estimate of sensitivity. In addition, the results underline that calibration is particularly important in people affected by lower-limb loss to capture the substantial variability in the conditions of the residual limb and prosthesis setup. These insights are important for the implementation of artificial sensory feedback in lower-limb prosthetics applications.

Toward higher-performance bionic limbs for wider clinical use

Most prosthetic limbs can autonomously move with dexterity, yet they are not perceived by the user as belonging to their own body. Robotic limbs can convey information about the environment with higher precision than biological limbs, but their actual performance is substantially limited by current technologies for the interfacing of the robotic devices with the body and for transferring motor and sensory information bidirectionally between the prosthesis and the user. In this Perspective, we argue that direct skeletal attachment of bionic devices via osseointegration, the amplification of neural signals by targeted muscle innervation, improved prosthesis control via implanted muscle sensors and advanced algorithms, and the provision of sensory feedback by means of electrodes implanted in peripheral nerves, should all be leveraged towards the creation of a new generation of high-performance bionic limbs. These technologies have been clinically tested in humans, and alongside mechanical redesigns and adequate rehabilitation training should facilitate the wider clinical use of bionic limbs.

A portable, programmable, multichannel stimulator with high compliance voltage for noninvasive neural stimulation of motor and sensory nerves in humans

Most neural stimulators do not have a high enough compliance voltage to pass current through the skin. The few stimulators that meet the high compliance voltage necessary for transcutaneous stimulation are typically large benchtop units that are not portable, and the stimulation waveforms cannot be readily customized. To address this, we present the design and validation of a portable, programmable, multichannel, noninvasive neural stimulator that can generate three custom bipolar waveforms at ± 150 V with microsecond temporal resolution. The design is low-cost, open-source, and validated on the benchtop and with a healthy population to demonstrate its functionality for sensory and motor stimulation. Sensory stimulation included electrocutaneous stimulation targeting cutaneous mechanoreceptors at the surface of the skin and transcutaneous nerve stimulation targeting the median nerve at the wrist. Both electrocutaneous stimulation on the hand and transcutaneous stimulation at the wrist can elicit isolated tactile percepts on the hand but changes in pulse frequency are more discriminable for electrocutaneous stimulation. Also, neuromuscular electrical stimulation of the flexor digiti profundus is evoked by applying electrical stimulation directly above the muscle in the forearm and to the median and ulnar nerves in the upper arm. Muscle and nerve stimulation evoked similar grip forces and force rise times, but nerve stimulation had a significantly slower fatigue rate. The development and validation of this noninvasive stimulator and direct comparison of common sensory and motor stimulation targets in a human population constitute an important step towards more widespread use and accessibility of neural stimulation for education and research.

Object stiffness recognition and vibratory feedback without ad-hoc sensing on the Hannes prosthesis: A machine learning approach

Introduction

In recent years, hand prostheses achieved relevant improvements in term of both motor and functional recovery. However, the rate of devices abandonment, also due to their poor embodiment, is still high. The embodiment defines the integration of an external object - in this case a prosthetic device - into the body scheme of an individual. One of the limiting factors causing lack of embodiment is the absence of a direct interaction between user and environment. Many studies focused on the extraction of tactile information via custom electronic skin technologies coupled with dedicated haptic feedback, though increasing the complexity of the prosthetic system. Contrary wise, this paper stems from the authors' preliminary works on multi-body prosthetic hand modeling and the identification of possible intrinsic information to assess object stiffness during interaction.

Methods

Based on these initial findings, this work presents the design, implementation and clinical validation of a novel real-time stiffness detection strategy, without ad-hoc sensing, based on a Non-linear Logistic Regression (NLR) classifier. This exploits the minimum grasp information available from an under-sensorized and under-actuated myoelectric prosthetic hand, Hannes. The NLR algorithm takes as input motor-side current, encoder position, and reference position of the hand and provides as output a classification of the grasped object (no-object, rigid object, and soft object). This information is then transmitted to the user via vibratory feedback to close the loop between user control and prosthesis interaction. This implementation was validated through a user study conducted both on able bodied subjects and amputees.

Results

The classifier achieved excellent performance in terms of F1Score (94.93%). Further, the able-bodied subjects and amputees were able to successfully detect the objects' stiffness with a F1Score of 94.08% and 86.41%, respectively, by using our proposed feedback strategy. This strategy allowed amputees to quickly recognize the objects' stiffness (response time of 2.82 s), indicating high intuitiveness, and it was overall appreciated as demonstrated by the questionnaire. Furthermore, an embodiment improvement was also obtained as highlighted by the proprioceptive drift toward the prosthesis (0.7 cm).

Substitutive proprioception feedback of a prosthetic wrist by electrotactile stimulation

Objective

Sensory feedback of upper-limb prostheses is widely desired and studied. As important components of proprioception, position, and movement feedback help users to control prostheses better. Among various feedback methods, electrotactile stimulation is a potential method for coding proprioceptive information of a prosthesis. This study was motivated by the need for proprioception information for a prosthetic wrist. The flexion-extension (FE) position and movement information of the prosthetic wrist are transmitted back to the human body through multichannel electrotactile stimulation.

Approach

We developed an electrotactile scheme to encode the FE position and movement of the prosthetic wrist and designed an integrated experimental platform. A preliminary experiment on the sensory threshold and discomfort threshold was performed. Then, two proprioceptive feedback experiments were performed: a position sense experiment (Exp 1) and a movement sense experiment (Exp 2). Each experiment included a learning session and a test session. The success rate (SR) and discrimination reaction time (DRT) were analyzed to evaluate the recognition effect. The acceptance of the electrotactile scheme was evaluated by a questionnaire.

Main results

Our results showed that the average position SRs of five able-bodied subjects, amputee 1, and amputee 2 were 83.78, 97.78, and 84.44%, respectively. The average movement SR, and the direction and range SR of wrist movement in five able-bodied subjects were 76.25, 96.67%, respectively. Amputee 1 and amputee 2 had movement SRs of 87.78 and 90.00% and direction and range SRs of 64.58 and 77.08%, respectively. The average DRT of five able-bodied subjects was less than 1.5 s and that of amputees was less than 3.5 s.

Conclusion

The results indicate that after a short period of learning, the subjects can sense the position and movement of wrist FE. The proposed substitutive scheme has the potential for amputees to sense a prosthetic wrist, thus enhancing the human-machine interaction.

FeetBack–Redirecting touch sensation from a prosthetic hand to the human foot

Introduction

Adding sensory feedback to myoelectric prosthetic hands was shown to enhance the user experience in terms of controllability and device embodiment. Often this is realized non-invasively by adding devices, such as actuators or electrodes, within the prosthetic shaft to deliver the desired feedback. However, adding a feedback system in the socket adds more weight, steals valuable space, and may interfere with myoelectric signals. To circumvent said drawbacks we tested for the first time if force feedback from a prosthetic hand could be redirected to another similarly sensitive part of the body: the foot.

Methods

We developed a vibrotactile insole that vibrates depending on the sensed force on the prosthetic fingers. This self-controlled clinical pilot trial included four experienced users of myoelectric prostheses. The participants solved two types of tasks with the artificial hands: 1) sorting objects depending on their plasticity with the feedback insole but without audio-visual feedback, and 2) manipulating fragile, heavy, and delicate objects with and without the feedback insole. The sorting task was evaluated with Goodman-Kruskal's gamma for ranked correlation. The manipulation tasks were assessed by the success rate.

Results

The results from the sorting task with vibrotactile feedback showed a substantial positive effect. The success rates for manipulation tasks with fragile and heavy objects were high under both conditions (feedback on or off, respectively). The manipulation task with delicate objects revealed inferior success with feedback in three of four participants.

Conclusion

We introduced a novel approach to touch sensation in myoelectric prostheses. The results for the sorting task and the manipulation tasks diverged. This is likely linked to the availability of various feedback sources. Our results for redirected feedback to the feet fall in line with previous similar studies that applied feedback to the residual arm.

Clinical trial registration

Name: Sensor Glove and Non-Invasive Vibrotactile Feedback Insole to Improve Hand Prostheses Functions and Embodiment (FeetBack). Date of registration: 23 April 2019. Date the first participant was enrolled: 3 September 2021. ClinicalTrials.gov Identifier: NCT03924310.

Electrotactile Communication via Matrix Electrode Placed on the Torso Using Fast Calibration, and Static vs. Dynamic Encoding

Electrotactile stimulation is a technology that reproducibly elicits tactile sensations and can be used as an alternative channel to communicate information to the user. The presented work is a part of an effort to develop this technology into an unobtrusive communication tool for first responders. In this study, the aim was to compare the success rate (SR) between discriminating stimulation at six spatial locations (static encoding) and recognizing six spatio-temporal patterns where pads are activated sequentially in a predetermined order (dynamic encoding). Additionally, a procedure for a fast amplitude calibration, that includes a semi-automated initialization and an optional manual adjustment, was employed and evaluated. Twenty subjects, including twelve first responders, participated in the study. The electrode comprising the 3 × 2 matrix of pads was placed on the lateral torso. The results showed that high SRs could be achieved for both types of message encoding after a short learning phase; however, the dynamic approach led to a statistically significant improvement in messages recognition (SR of 93.3%), compared to static stimulation (SR of 83.3%). The proposed calibration procedure was also effective since in 83.8% of the cases the subjects did not need to adjust the stimulation amplitude manually.

Continuous Transition Impairs Discrimination of Electrotactile Frequencies

Just-noticeable difference (JND), indicating the ability to accurately identify small differences in stimulation parameters, can be used to choose more sensitive stimulation methods as well as to calibrate tactile feedback in closed-loop human-machine interfacing. The JND is typically estimated using a forced-choice-discrimination task, in which two stimuli with different intensities are delivered separated by a brief pause. In the applications of tactile feedback, however, the stimulation parameters are typically modulated continuously. It is unclear if the discriminability of stimuli separated in time characterizes the ability to distinguish continuous changes in stimulation intensity. The present study compared the JND when pairs of frequency-modulated electrotactile stimuli were separated in time and presented continuously at two different baseline frequencies (20 and 60 Hz). The results showed that the JND was significantly smaller with time-separation between stimuli, but that the JND obtained with different types of transitions were in most cases linearly associated. In conclusion, the discriminability of time-separated stimuli is systematically better compared to that of the stimuli presented continuously. This can have an impact when calibrating the tactile feedback where the conventional method of the JND assessment might lead to an overly optimistic estimate of detectable changes.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Electrotactile Perception Properties and Its Applications: A Review

With the increased demands of human-machine interaction, haptic feedback is becoming increasingly critical. However, the high cost, large size and low efficiency of current haptic systems severely hinder further development. As a portable and efficient technology, cutaneous electrotactile stimulation has shown promising potential for these issues. This paper presents a review on and insight into cutaneous electrotactile perception and its applications. Research results on perceptual properties and evaluation methods have been summarized and discussed to understand the effects of electrotactile stimulation on humans. Electrotactile applications are presented in categories to understand the methods and progress in various fields such as prostheses control, sensory substitution, sensory restoration and sensorimotor restoration. State of the art has demonstrated the superiority of electrotactile feedback, its efficiency and its flexibility. However, the complex factors and the limitations of evaluation methods made it challenging for precise electrotactile control. Groundbreaking innovation in electrotactile theory is expected to overcome challenges such as precise perception control, information capacity increasing, comprehension burden reducing and implementation costs.

Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future Directions

Haptic feedback is critical in a broad range of human-machine/computer-interaction applications. However, the high cost and low portability/wearability of haptic devices remain unresolved issues, severely limiting the adoption of this otherwise promising technology. Electrotactile interfaces have the advantage of being more portable and wearable due to their reduced actuators' size, as well as their lower power consumption and manufacturing cost. The applications of electrotactile feedback have been explored in human-computer interaction and human-machine-interaction for facilitating hand-based interactions in applications, such as prosthetics, virtual reality, robotic teleoperation, surface haptics, portable devices, and rehabilitation. This article presents a technological overview of electrotactile feedback, as well a systematic review and meta-analysis of its applications for hand-based interactions. We discuss the different electrotactile systems according to the type of application. We also discuss over a quantitative congregation of the findings, to offer a high-level overview into the state-of-art and suggest future directions. Electrotactile feedback systems showed increased portability/wearability, and they were successful in rendering and/or augmenting most tactile sensations, eliciting perceptual processes, and improving performance in many scenarios. However, knowledge gaps (e.g., embodiment), technical (e.g., recurrent calibration, electrodes' durability) and methodological (e.g., sample size) drawbacks were detected, which should be addressed in future studies.

Design of multi-pad electrotactile system envisioned as a feedback channel for supernumerary robotic limbs

BACKGROUND

Providing real-time haptic feedback is an important, but still not sufficiently explored aspect of use of supernumerary robotic limbs (SRLs). We present a multi-pad electrode for conveying multi-modal proprioceptive and sensory information from SRL to the user's thigh and propose a method for stimuli calibration.

METHODS

Within two pilot tests we investigated return electrode configuration and active electrode discrimination in three healthy subjects to select the appropriate electrode pad topology. Based on the obtained results and anthropometric data from literature, the electrode was designed to have three branches of 10 pads and two additional pads that can be displaced over/under the electrode branches. The electrode was designed to be connected to the stimulator that allows full multiplexing so that specific branches can serve as common return electrode. To define the procedure for application of this system, the sensation, localization and discomfort thresholds applicable for the novel electrode were determined and analysed in ten subjects.

RESULTS

The results showed no overlaps between the three thresholds for individual pads, with significantly different average values, suggesting that the selected electrode positioning and design provide good active range of useful current amplitude. The results of the subsequent analysis suggested that the stimuli intensity level of 200% of sensation threshold is the most probable value of the localization threshold. Furthermore, this level ensures low chance (i.e. 0.7%) of reaching the discomfort.

CONCLUSIONS

We believe that envisioned electrotactile system could serve as a high bandwidth feedback channel that can be easily setup to provide proprioceptive and sensory feedback from supernumerary limbs.

Design and Development of OECT Logic Circuits for Electrical Stimulation Applications

This paper presents the first successful implementation of fully printed electronics for flexible and wearable smart multi-pad stimulation electrodes intended for use in medical, sports and lifestyle applications. The smart multi-pad electrodes with the electronic circuits based on organic electrochemical transistor (OECT)-based electronic circuits comprising the 3–8 decoder for active pad selection and high current throughput transistors for switching were produced by multi-layer screen printing. Devices with different architectures of switching transistors were tested in relevant conditions for electrical stimulation applications. An automated testbed with a configurable stimulation source and an adjustable human model equivalent circuit was developed for this purpose. Three of the proposed architectures successfully routed electrical currents of up to 15 mA at an output voltage of 30 V, while one was reliably performing even at 40 V. The presented results demonstrate feasibility of the concept in a range of conditions relevant to several applications of electrical stimulation.

The Impact of Size and Position of Reference Electrode on the Localization of Biphasic Electrotactile Stimulation on the Fingertips

Development of haptic interfaces to enrich augmented and virtual reality with the sense of touch is the next frontier for technological advancement of these systems. Among available technologies, electrotactile stimulation enables design of high-density interfaces that can provide natural-like sensation of touch in interaction with virtual objects. The present study investigates the human perception of electrotactile sensations on fingertips, focusing on the sensation localization in function of the size and position of reference electrode. Ten healthy subjects participated in the study, with the task to mark the sensations elicited by stimulating the index fingertip using an 8-pad electrode. The test systematically explored several configurations of the active (position) and reference (position and size) electrode pads. The results indicated that there was a spreading of perceived sensations across the fingertip, but that they were mostly localized below the active pad. The position and size of the reference electrode were shown to affect the location of the perceived sensations, which can potentially be exploited as an additional parameter to modulate the feedback. The present study demonstrates that the fingertip is a promising target for the delivery of high-resolution feedback.

Electrotactile and Vibrotactile Feedback Enable Similar Performance in Psychometric Tests and Closed-Loop Control

Electro- and vibro-tactile stimulation are commonly employed for feedback in closed-loop human-machine interfacing. Although these feedback systems have been extensively investigated individually, they are rarely objectively compared. In this study, two state-of-the-art stimulation units (concentric electrode and C2-tactor) similar in shape and size were compared in psychometric and online control tests. The just noticeable difference and number of discriminable levels for intensity and frequency modulation were determined across values of carrier frequency and intensity, respectively. Next, subjects performed a compensatory tracking task, in which the feedback encoded the momentary tracking error. In the psychometric tests, intensity modulation outperformed frequency modulation and electrotactile stimulation enabled significantly higher resolution than vibrotactile stimulation, for the same carrier frequency. However, for the best-case settings (eletro-tactile: 100 Hz; vibro-tactile: 200 Hz), the two stimulation modalities were equivalent in the psychometric tests and in the online control tests, where the two stimulation methods resulted in similar correlation and deviation between the target and the generated trajectory. Time delay was slightly but significantly lower for the vibrotactile modality. Overall, the present assessment shows that despite psychometric differences between the two stimulation methods, they enable similar online control performance when parameters are optimally selected for each modality.

Quantifying the Generation Process of Multi-Level Tactile Sensations via ERP Component Investigation

Humans obtain characteristic information such as texture and weight of external objects, relying on the brain's integration and classification of tactile information; however, the decoding mechanism of multi-level tactile information is relatively elusive from the temporal sequence. In this paper, nonvariant frequency, along with the variant pulse width of electrotactile stimulus, was performed to generate multi-level pressure sensation. Event-related potentials (ERPs) were measured to investigate the mechanism of whole temporal tactile processing. Five ERP components, containing P100-N140-P200-N200-P300, were observed. By establishing the relationship between stimulation parameters and ERP component amplitudes, we found the following: (1) P200 is the most significant component for distinguishing multi-level tactile sensations; (2) P300 is correlated well with the subjective judgment of tactile sensation. The temporal sequence of brain topographies was implemented to clarify the spatiotemporal characteristics of the tactile process, which conformed to the serial processing model in neurophysiology and cortical network response area described by fMRI. Our results can help further clarify the mechanism of tactile sequential processing, which can be applied to improve the tactile BCI performance, sensory enhancement, and clinical diagnosis for doctors to evaluate the tactile process disorders by examining the temporal ERP components.

Embedded Electrotactile Feedback System for Hand Prostheses Using Matrix Electrode and Electronic Skin

As the technology moves towards more human-like bionic limbs, it is necessary to develop a feedback system that provides active touch feedback to a user of a prosthetic hand. Most of the contemporary sensory substitution methods comprise simple position and force sensors combined with few discrete stimulation units, and hence they are characterized with a limited amount of information that can be transmitted by the feedback. The present study describes a novel system for tactile feedback integrating advanced multipoint sensing (electronic skin) and stimulation (matrix electrodes). The system comprises a flexible sensing array (16 sensors) integrated on the index finger of a Michelangelo prosthetic hand mockup, embedded interface electronics and multichannel stimulator connected to a flexible matrix electrode (24 pads). The developed system conveys contact information (binary detections) to the user. To demonstrate the feasibility, the system was tested in six able-bodied subjects who were asked to recognize static patterns (contact position) with two different spatial resolutions and dynamic movement patterns (i.e., sliding along and/or across the finger) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into electrotactile profiles, which the subjects could recognize with good performance. The success rates (mean ± standard deviation) for the static patterns were 91 ± 4% and 58 ± 10% for low and high spatial resolution, respectively, while the success rate for sliding touch was 94 ± 4%. These results demonstrate that the developed system is an important step towards a new generation of tactile feedback interfaces that can provide high-bandwidth connection between the user and his/her bionic limb. Such systems would allow mimicking spatially distributed natural feedback, thereby facilitating the control and embodiment of the artificial device into the user body scheme.

A compact system for simultaneous stimulation and recording for closed-loop myoelectric control

Background

Despite important advancements in control and mechatronics of myoelectric prostheses, the communication between the user and his/her bionic limb is still unidirectional, as these systems do not provide somatosensory feedback. Electrotactile stimulation is an attractive technology to close the control loop since it allows flexible modulation of multiple parameters and compact interface design via multi-pad electrodes. However, the stimulation interferes with the recording of myoelectric signals and this can be detrimental to control.

Methods

We present a novel compact solution for simultaneous recording and stimulation through dynamic blanking of stimulation artefacts. To test the system, a feedback coding scheme communicating wrist rotation and hand aperture was developed specifically to stress the myoelectric control while still providing meaningful information to the subjects. Ten subjects participated in an experiment, where the quality of closed-loop myoelectric control was assessed by controlling a cursor in a two degrees of freedom target-reaching task. The benchmark performance with visual feedback was compared to that achieved by combining visual feedback and electrotactile stimulation as well as by using electrotactile feedback only.

Results

There was no significant difference in performance between visual and combined feedback condition with regards to successfully reached targets, time to reach a target, path efficiency and the number of overshoots. Therefore, the quality of myoelectric control was preserved in spite of the stimulation. As expected, the tactile condition was significantly poorer in completion rate (100/4% and 78/25% for combined and tactile condition, respectively) and time to reach a target (9/2 s and 13/4 s for combined and tactile condition, respectively). However, the performance in the tactile condition was still good, with no significant difference in path efficiency (38/8%) and the number of overshoots (0.5/0.4 overshoots), indicating that the stimulation was meaningful for the subjects and useful for closed-loop control.

Conclusions

Overall, the results demonstrated that the developed system can provide robust closed-loop control using electrotactile stimulation. The system supports different encoding schemes and allows placing the recording and stimulation electrodes next to each other. This is an important step towards an integrated solution where the developed unit will be embedded into a prosthetic socket.

Development of a Tactile Actuator with Non-Contact and Trans-Object Characteristics Using a Time-Varying Magnetic Field

A non-contact tactile stimulation system using a time-varying magnetic field was developed. The system comprises a control unit, power unit, output unit, and actuator. The control unit adjusts stimulation parameters, particularly the signal intensity and frequency. The power unit produces high voltages for generating the magnetic field, whereas the output unit transmits the energy generated according to the signal from the control unit to the actuator. A spiral coil actuator generates the magnetic field. To validate the effectiveness of the system, preliminary experiments on 10 male adults without neurological disorders (23.2 ± 3.05 years) were conducted. Magnetic field stimuli were presented to the right palm of the subjects at three different frequencies (10, 30, and 50 Hz), and corresponding electroencephalogram (EEG) signals were measured simultaneously. Event-related potential (ERP) analysis showed that N100 and P300 components were identified in somatosensory areas. Subjective evaluations revealed that feelings such as “tingling,” “trembling,” “tapping,” and “percussing” were induced. Moreover, as the stimulus frequency changes, differences may occur in induced feeling. The system uses a time-varying magnetic field, which not only induces tactile stimulation without contact but also has trans-object characteristics that can present tactile sensations, even when there is an obstacle between an actuator and skin.

Non-informative vision improves spatial tactile discrimination on the shoulder but does not influence detection sensitivity

Vision of the body has been reported to improve tactile acuity even when vision is not informative about the actual tactile stimulation. However, it is currently unclear whether this effect is limited to body parts such as hand, forearm or foot that can be normally viewed, or it also generalizes to body locations, such as the shoulder, that are rarely before our own eyes. In this study, subjects consecutively performed a detection threshold task and a numerosity judgment task of tactile stimuli on the shoulder. Meanwhile, they watched either a real-time video showing their shoulder or simply a fixation cross as control condition. We show that non-informative vision improves tactile numerosity judgment which might involve tactile acuity, but not tactile sensitivity. Furthermore, the improvement in tactile accuracy modulated by vision seems to be due to an enhanced ability in discriminating the number of adjacent active electrodes. These results are consistent with the view that bimodal visuotactile neurons sharp tactile receptive fields in an early somatosensory map, probably via top-down modulation of lateral inhibition.

Recent Developments in Prosthesis Sensors, Texture Recognition, and Sensory Stimulation for Upper Limb Prostheses

Current developments being made in upper limb prostheses are focused on replacing lost sensory information to the amputees. Providing sensory stimulation from the prosthesis can directly improve control over the prosthetic and provide a sense of body ownership. The focus of this review article is on recent developments while including foundational knowledge for some of the critical concepts in neural prostheses. Reported concepts follow the flow of information from sensors to signal processing, with emphasis on texture recognition, and then to sensory stimulation strategies that reestablish the lost sensory feedback loop. Prosthetic sensors are used to detect the physical environment, converting pressure, force, and position into electrical signals. The electrical signals can then be processed in an effort to identify the surrounding environment using distinctive characteristics such as stiffness and texture. In order for the amputee to use this information in a natural manner, there must be real-time sensory stimulation, perception, and motor control of the prosthesis. Although truly complete sensory replacement has not yet been realized, some basic percepts can be partially restored, allowing progress towards a more realistic prosthesis with natural sensations.

Wearable multichannel haptic device for encoding proprioception in the upper limb

OBJECTIVE

We present the design, implementation, and evaluation of a wearable multichannel haptic system. The device is a wireless closed-loop armband driven by surface electromyography (EMG) and provides sensory feedback encoding proprioception. The study is motivated by restoring proprioception information in upper limb prostheses.

APPROACH

The armband comprises eight vibrotactile actuators that generate distributed patterns of mechanical waves around the limb to stimulate perception and to transfer proportional information on the arm motion. An experimental study was conducted to assess: the sensory threshold in eight locations around the forearm, the user adaptation to the sensation provided by the device, the user performance in discriminating multiple stimulation levels, and the device performance in coding proprioception using four spatial patterns of stimulation. Eight able-bodied individuals performed reaching tasks by controlling a cursor with an EMG interface in a virtual environment. Vibrotactile patterns were tested with and without visual information on the cursor position with the addition of a random rotation of the reference control system to disturb the natural control and proprioception.

MAIN RESULTS

The sensation threshold depended on the actuator position and increased over time. The maximum resolution for stimuli discrimination was four. Using this resolution, four patterns of vibrotactile activation with different spatial and magnitude properties were generated to evaluate their performance in enhancing proprioception. The optimal vibration pattern varied among the participants. When the feedback was used in closed-loop control with the EMG interface, the task success rate, completion time, execution efficiency, and average target-cursor distance improved for the optimal stimulation pattern compared to the condition without visual or haptic information on the cursor position.

SIGNIFICANCE

The results indicate that the vibrotactile device enhanced the participants' perceptual ability, suggesting that the proposed closed-loop system has the potential to code proprioception and enhance user performance in the presence of perceptual perturbation.

Amplitude versus spatially modulated electrotactile feedback for myoelectric control of two degrees of freedom

OBJECTIVE

Artificial proprioceptive feedback from a myoelectric prosthesis is an important aspect in enhancing embodiment and user satisfaction, possibly lowering the demand for visual attention while controlling a prosthesis in everyday tasks. Contemporary myoelectric prostheses are advanced mechatronic systems with multiple degrees of freedom, and therefore, to communicate the prosthesis state, the feedback interface needs to transmit several variables simultaneously. In the present study, two different configurations for conveying proprioceptive information of wrist rotation and hand aperture through multichannel electrotactile stimulation were developed and evaluated during online myoelectric control.

APPROACH

Myoelectric recordings were acquired from the dominant forearm and electrotactile stimulation was delivered on the non-dominant forearm using a compact interface. The first feedback configuration, which was based on spatial coding, transmitted the information using a moving tactile stimulus, whereas the second, amplitude-based configuration conveyed the position via sensation intensity. Thirteen able-bodied subjects used pattern classification-based myoelectric control with both feedback configurations to accomplish a target-reaching task.

MAIN RESULTS

High task performance (completion rate > 90%) was observed for both configurations, with no significant difference in completion rate, time to reach the target, distance error and path efficiency, respectively.

SIGNIFICANCE

Overall, the results demonstrated that both feedback configurations allowed subjects to perceive and interpret two feedback variables delivered simultaneously, despite using a compact stimulation interface. This is an encouraging result for the prospect of communicating the full state of a multifunctional hand prosthesis.

Temporal Asynchrony but Not Total Energy Nor Duration Improves the Judgment of Numerosity in Electrotactile Stimulation

Stroke patients suffer from impairments of both motor and somatosensory functions. The functional recovery of upper extremities is one of the primary goals of rehabilitation programs. Additional somatosensory deficits limit sensorimotor function and significantly affect its recovery after the neuromotor injury. Sensory substitution systems, providing tactile feedback, might facilitate manipulation capability, and improve patient's dexterity during grasping movements. As a first step toward this aim, we evaluated the ability of healthy subjects in exploiting electrotactile feedback on the shoulder to determine the number of perceived stimuli in numerosity judgment tasks. During the experiment, we compared four different stimulation patterns (two simultaneous: short and long, intermittent and sequential) differing in total duration, total energy, or temporal synchrony. The experiment confirmed that the subject ability to enumerate electrotactile stimuli decreased with increasing the number of active electrodes. Furthermore, we found that, in electrotactile stimulation, the temporal coding schemes, and not total energy or duration modulated the accuracy in numerosity judgment. More precisely, the sequential condition resulted in significantly better numerosity discrimination than intermittent and simultaneous stimulation. These findings, together with the fact that the shoulder appeared to be a feasible stimulation site to communicate tactile information via electrotactile feedback, can serve as a guide to deliver tactile feedback to proximal areas in stroke survivors who lack sensory integrity in distal areas of their affected arm, but retain motor skills.

A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control

This manuscript reviews historical and recent studies that focus on supplementary sensory feedback for use in upper limb prostheses. It shows that the inability of many studies to speak to the issue of meaningful performance improvements in real-life scenarios is caused by the complexity of the interactions of supplementary sensory feedback with other types of feedback along with other portions of the motor control process. To do this, the present manuscript frames the question of supplementary feedback from the perspective of computational motor control, providing a brief review of the main advances in that field over the last 20 years. It then separates the studies on the closed-loop prosthesis control into distinct categories, which are defined by relating the impact of feedback to the relevant components of the motor control framework, and reviews the work that has been done over the last 50+ years in each of those categories. It ends with a discussion of the studies, along with suggestions for experimental construction and connections with other areas of research, such as machine learning.

Evoking haptic sensations in the foot through high-density transcutaneous electrical nerve stimulations

OBJECTIVE

Evoking haptic sensation on upper limb amputees via peripheral nerve stimulation has been investigated intensively in the past decade, but related studies involving lower limb amputees are limited. This study aimed to evaluate the feasibility of using non-invasive transcutaneous electrical nerve stimulation to evoke haptic sensation along the phantom limb of the amputated foot of transtibial amputees.

APPROACH

A high-density electrode grid (4 × 4) was placed over the skin surface above the distal branching of the sciatic, tibial, and common peroneal nerves. We hypothesized that electrical stimulation delivered to distinct electrode pairs created unique electric fields, which can activate selective sets of sensory axons innervating different skin regions of the foot. Five transtibial amputee subjects (three unilateral and two bilateral) and one able-bodied subject were tested by scanning all possible electrode pair combinations.

MAIN RESULTS

All subjects reported various haptic percepts at distinct regions along the foot with each corresponding to specific electrode pairs. These results demonstrated the capability of our non-invasive nerve stimulation method to evoke haptic sensations in the foot of transtibial amputees and the able-bodied subject.

SIGNIFICANCE

The outcomes contribute important knowledge and evidence regarding missing tactile sensation in the foot of lower limb amputees and might also facilitate future development of strategies to manage phantom pain and enhance embodiment of prosthetic legs in the future.

Electropneumotactile Stimulation: Multimodal Haptic Actuators Enabled by a Stretchable Conductive Polymer on Inflatable Pockets

This paper describes a type of haptic device that delivers two modes of stimulation simultaneously and at the same locations on the skin. The two modes of stimulation are mechanical (delivered pneumatically by inflatable air pockets embedded within a silicone elastomer) and electrical (delivered by a conductive polymer). The key enabling aspect of this work is the use of a highly plasticized conductive polymer based on poly(3,4-ethylenedioxythiphene) (PEDOT) blended with elastomeric polyurethane (PU). To fabricate the "electropneumotactile" device, the polymeric electrodes are overlaid directly on top of the elastomeric pneumatic actuator pockets. Co-placement of the pneumatic actuators and the electrotactile electrodes is enabled by the stretchability of the PEDOT:OTs/PU blend, allowing the electrotactiles to conform to underlying pneumatic pockets under deformation. The blend of PEDOT and PU has a Young's modulus of ~150 MPa with little degradation in conductivity following repeated inflation of the air pockets. The ability to perceive simultaneous delivery of two sensations to the same location on the skin are supported by experiments using human subjects. These results show that participants can successfully detect the location of pneumatic stimulation and whether electrotactile stimulation is delivered (yes/no) at a rate significantly above chance (mean accuracy = 94%).

Virtual Texture Generated using Elastomeric Conductive Block Copolymer in Wireless Multimodal Haptic Glove

Haptic devices are in general more adept at mimicking the bulk properties of materials than they are at mimicking the surface properties. This paper describes a haptic glove capable of producing sensations reminiscent of three types of near-surface properties: hardness, temperature, and roughness. To accomplish this mixed mode of stimulation, three types of haptic actuators were combined: vibrotactile motors, thermoelectric devices, and electrotactile electrodes made from a stretchable conductive polymer synthesized in our laboratory. This polymer consisted of a stretchable polyanion which served as a scaffold for the polymerization of poly(3,4-ethylenedioxythiophene) (PEDOT). The scaffold was synthesized using controlled radical polymerization to afford material of low dispersity, relatively high conductivity (0.1 S cm-1), and low impedance relative to metals. The glove was equipped with flex sensors to make it possible to control a robotic hand and a hand in virtual reality (VR). In psychophysical experiments, human participants were able to discern combinations of electrotactile, vibrotactile, and thermal stimulation in VR. Participants trained to associate these sensations with roughness, hardness, and temperature had an overall accuracy of 98%, while untrained participants had an accuracy of 85%. Sensations could similarly be conveyed using a robotic hand equipped with sensors for pressure and temperature.

Dual-Parameter Modulation Improves Stimulus Localization in Multichannel Electrotactile Stimulation

Among most challenging open issues in prosthetic research is the development of a robust bidirectional interface between a prosthesis and its user. Commercially available prosthetic systems are mechanically advanced, but they do not provide somatosensory feedback. Here, we present a novel non-invasive interface for multichannel electrotactile feedback, comprising a matrix of 24 pads, and we investigate the ability of able-bodied human subjects to localize the electrotactile stimulus delivered through the matrix. For this purpose, we tested conventional stimulation (same frequency for all pads) and a novel dual-parameter modulation scheme (interleaved frequency and intensity) designed to facilitate the spatial localization over the electrode. Electrotactile stimulation was also compared to mechanical stimulation of the same locations on the skin. Experimental results on eight able-bodied subjects demonstrated that the proposed interleaved coding substantially improved the spatial localization compared to same-frequency stimulation. The results also showed that same-frequency stimulation was equivalent to mechanical stimulation, whereas the performance with dual-parameter modulation was significantly better. These are encouraging outcomes for the application of a multichannel interface for the restoration of feedback in prosthetics. The high-resolution augmented interfaces might be used to explore novel scenarios for effective communication with the prosthesis user enabled by maximizing information transmission.

The Variability of Psychophysical Parameters Following Surface and Subdermal Stimulation: A Multiday Study in Amputees

Electrotactile stimulation has been suggested as a modality for providing sensory feedback in upper limb prostheses. This study investigates the multiday variability of subdermal and surface stimulation. Electrical stimulation was delivered using either surface or fine wire electrodes placed right under the skin in eight amputees for seven consecutive days. The variability of psychophysical measurements, including detection threshold (DT), pain threshold (PT), dynamic range (DR), just noticeable difference (JND), Weber fraction (WF) and quality of evoked sensations, was evaluated using the coefficient of variation (CoV). In addition, the systematic change in the mean of the parameters across days was assessed in both stimulation modalities. In the case of DT, PT, DR, and perceived intensity at 100 Hz, the CoV of surface stimulation was significantly smaller than that of subdermal stimulation. Only PT showed a significant systematic change in the mean value across days for both modalities. The outcome of this study has implications for the choice of modality in delivering sensory feedback, though the significance of the quantified variability needs to be evaluated using usability tests with user feedback.

Simulated robotic device malfunctions resembling malicious cyberattacks impact human perception of trust, satisfaction, and frustration

Robot assistants and wearable devices are highly useful; however, these artificial systems are susceptible to hackers. In this article, two sets of experiments were conducted. The first part of this study simulated a malicious attack on a prosthetic arm system to adversely affect the operation of the prosthetic system, while the perception of 10 human subjects was surveyed. These 10 able-bodied subjects controlled the prosthetic arm and hand with electromyogram signals, while an artificial sensation of touch was conveyed to their arms as they operated the system, which enabled them to feel what the prosthetic hand was grasping as they were asked to transport an object from one location to another. This haptic feedback was provided in both the normal and abnormal operational modes but was disabled in the extremely abnormal mode. The electromyogram control signals for the arm were reversed in both the abnormal and extremely abnormal modes. Results from the simulated malicious attack on a prosthetic arm system showed that the subjects found the haptic feedback helpful in both the normal and abnormal modes of operation. Both the abnormal and extremely abnormal modes of operation negatively impacted the self-reported levels of trust, satisfaction, and frustration with the prosthetic system as the subjects grasped and transported an object. While these metrics were negatively impacted by system malfunctions resembling a malicious attack on the control functionality, it was possible to rebuild them to their former higher levels after the functionality of the prosthetic system was restored. A parallel study in this article involved simulating a malicious attack on a robot assistant to unfavorably affect the delivery operation modes, while the perception of 20 human subjects was surveyed. Results showed that the simulated malfunctions unfavorably impacted the perception of trust, satisfaction, and frustration, but it was possible to restore these metrics in two different ways as the device functionality was restored.

Optimization of Semiautomated Calibration Algorithm of Multichannel Electrotactile Feedback for Myoelectric Hand Prosthesis

The main drawback of the commercially available myoelectric hand prostheses is the absence of somatosensory feedback. We recently developed a feedback interface for multiple degrees of freedom myoelectric prosthesis that allows proprioceptive and sensory information (i.e., grasping force) to be transmitted to the wearer instantaneously. High information bandwidth is achieved through intelligent control of spatiotemporal distribution of electrical pulses over a custom-designed electrode array. As electrotactile sensations are location-dependent and the developed interface requires that electrical stimuli are perceived to be of the same intensity on all locations, a calibration procedure is of high importance. The aim of this study was to gain more insight into the calibration procedure and optimize this process by leveraging a priori knowledge. For this purpose, we conducted a study with 9 able-bodied subjects performing 10 sessions of the array electrode calibration. Based on the collected data, we optimized and simplified the calibration procedure by adapting the initial (baseline) amplitude values in the calibration algorithm. The results suggest there is an individual pattern of stimulation amplitudes across 16 electrode pads for each subject, which is not affected by the initial amplitudes. Moreover, the number of user actions performed and the time needed for the calibration procedure are significantly reduced by the proposed methodology.

A thin-film multichannel electrode for muscle recording and stimulation in neuroprosthetics applications

OBJECTIVE

We propose, design and test a novel thin-film multichannel electrode that can be used for both recording from and stimulating a muscle in acute implants.

APPROACH

The system is built on a substrate of polyimide and contains 12 recording and three stimulation sites made of platinum. The structure is 420 µm wide, 20 µm thick and embeds the recording and stimulation contacts on the two sides of the polyimide over an approximate length of 2 cm. We show representative applications in healthy individuals as well as tremor patients. The designed system was tested by a psychometric characterization of the stimulation contacts in six tremor patients and three healthy individuals determining the perception threshold and current limit as well as the success rate in discriminating elicited sensations (electrotactile feedback). Also, we investigated the possibility of using the intramuscular electrode for reducing tremor in one patient by electrical stimulation delivered with timing based on the electromyographic activity recorded with the same electrode.

MAIN RESULTS

In the tremor patients, the current corresponding to the perception threshold and the current limit were 0.7  ±  0.2 and 1.4  ±  0.7 mA for the wrist flexor muscles and 0.4  ±  0.2 and 1.5  ±  0.7 mA for the extensors. In one patient, closed-loop stimulation resulted in a decrease of the tremor power  >50%. In healthy individuals the perception threshold and current limits were 0.9  ±  0.6 and 2.1  ±  0.6 mA for the extensor carpi radialis muscle. The subjects could distinguish four or six stimulation patterns (two or three stimulation sites  ×  two stimulation current amplitudes) with true positive rate  >80% (two subjects) and  >60% (one subject), respectively.

SIGNIFICANCE

The proposed electrode provides a compact multichannel interface for recording electromyogram and delivering electrical stimulation in applications such as neuroprostheses for tremor suppression and closed-loop myoelectric prostheses.

Evoked Haptic Sensation in the Hand With Concurrent Non-Invasive Nerve Stimulation

OBJECTIVE

Haptic perception is critical for prosthetic users to control their prosthetic hand intuitively. In this study, we seek to evaluate the haptic perception evoked from concurrent stimulation trains through multiple channels using transcutaneous nerve stimulation.

METHODS

A 2 × 8 electrode grid was used to deliver current to the median and ulnar nerves in the upper arm. Different electrodes were first selected to activate the sensory axons, which can elicit sensations at different locations of the hand. Charge-balanced bipolar stimulation was then delivered to two sets of electrodes concurrently with a phase delay (dual stimulation) to determine whether the evoked sensation can be constructed from sensations of single stimulation delivered separately at different locations (single stimulation) along the electrode grid. The temporal delay between the two stimulation trains was altered to evaluate potential interference. The short-term stability of the haptic sensation within a testing session was also evaluated.

RESULTS

The evoked sensation during dual stimulation was largely a direct summation of the sensation from single stimulations. The delay between the two stimulation locations had minimal effect on the evoked sensations, which was also stable over repeated testing within a session.

CONCLUSION

Our results indicated that the haptic sensations at different regions of the hand can be constructed by combining the response from multiple stimulation trains directly. The interference between stimulations were minimal.

SIGNIFICANCE

The outcomes will allow us to construct specific haptic sensation patterns when the prosthesis interacts with different objects, which may help improve user embodiment of the prosthesis.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices – such as prostheses, orthoses, exoskeletons, and devices for teleoperation–often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Reusable Flexible Concentric Electrodes Coated With a Conductive Graphene Ink for Electrotactile Stimulation

Electrotactile stimulation is a highly promising technique for providing sensory feedback information for prosthetics. To this aim, disposable electrodes which are predominantly used result in a high environmental and financial cost when used over a long period of time. In addition, disposable electrodes are limited in their size and configurations. This paper presents an alternative approach based on a 3D printed reusable flexible concentric electrode coated with a conductive graphene ink. Here, we have characterized the electrode and demonstrated its effective performance in electrotactile stimulation and sensory feedback for robotic prosthetic hands.

Robotic interfaces for cognitive psychology and embodiment research: A research roadmap

Advanced human-machine interfaces render robotic devices applicable to study and enhance human cognition. This turns robots into formidable neuroscientific tools to study processes such as the adaptation between a human operator and the operated robotic device and how this adaptation modulates human embodiment and embodied cognition. We analyze bidirectional human-machine interface (bHMI) technologies for transparent information transfer between a human and a robot via efferent and afferent channels. Even if such interfaces have a tremendous positive impact on feedback loops and embodiment, advanced bHMIs face immense technological challenges. We critically discuss existing technical approaches, mainly focusing on haptics, and suggest extensions thereof, which include other aspects of touch. Moreover, we point out other potential constraints such as limited functionality, semi-autonomy, intent-detection, and feedback methods. From this, we develop a research roadmap to guide understanding and development of bidirectional human-machine interfaces that enable robotic experiments to empirically study the human mind and embodiment. We conclude the integration of dexterous control and multisensory feedback to be a promising roadmap towards future robotic interfaces, especially regarding applications in the cognitive sciences. This article is categorized under: Computer Science > Robotics Psychology > Motor Skill and Performance Neuroscience > Plasticity.

Merged Haptic Sensation in the Hand during Concurrent Non-Invasive Proximal Nerve Stimulation

When individuals interact with the environment, sensory feedback is a critical aspect of the experience. Individuals using prosthesis often have difficulty controlling their device, partly due to a lack of sensory information. Transcutaneous nerve stimulation has the potential to elicit focal haptic sensation when controlled electrical current was delivered to a pair of electrodes in proximity to the nerve. The objective of this preliminary study was to evaluate how different elicited focal haptic sensation were altered, when multiple concurrent electrical stimuli were delivered to different portions of the median and ulnar nerve bundles. The delay between the individual stimulation during concurrent stimuli was also varied to identify if this parameter could alter the resulting sensation region. Lastly, the stability/repeatability of the perceived sensation during concurrent stimuli was determined. Our preliminary results showed that the spatial distribution of the haptic sensation was largely a direct summation/merge of the sensation regions from the individual nerve stimulation when comparing the regions to that of the concurrent double stimulation. Our results also showed that merged sensation region was not sensitive to different time delays the two concurrent stimuli. Lastly, the sensation regions remained stable and showed repeatable sensation in the hand even with 20-60 minutes between repeated stimulations.

Object discrimination using electrotactile feedback

OBJECTIVE

A variety of bioengineering systems are being developed to restore tactile sensations in individuals who have lost somatosensory feedback because of spinal cord injury, stroke, or amputation. These systems typically detect tactile force with sensors placed on an insensate hand (or prosthetic hand in the case of amputees) and deliver touch information by electrically or mechanically stimulating sensate skin above the site of injury. Successful object manipulation, however, also requires proprioceptive feedback representing the configuration and movements of the hand and digits.

APPROACH

Therefore, we developed a simple system that simultaneously provides information about tactile grip force and hand aperture using current amplitude-modulated electrotactile feedback. We evaluated the utility of this system by testing the ability of eight healthy human subjects to distinguish among 27 objects of varying sizes, weights, and compliances based entirely on electrotactile feedback. The feedback was modulated by grip-force and hand-aperture sensors placed on the hand of an experimenter (not visible to the subject) grasping and lifting the test objects. We were also interested to determine the degree to which subjects could learn to use such feedback when tested over five consecutive sessions.

MAIN RESULTS

The average percentage correct identifications on day 1 (28.5%  ±  8.2% correct) was well above chance (3.7%) and increased significantly with training to 49.2%  ±  10.6% on day 5. Furthermore, this training transferred reasonably well to a set of novel objects.

SIGNIFICANCE

These results suggest that simple, non-invasive methods can provide useful multisensory feedback that might prove beneficial in improving the control over prosthetic limbs.

Feel-Good Robotics: Requirements on Touch for Embodiment in Assistive Robotics

The feeling of embodiment, i.e., experiencing the body as belonging to oneself and being able to integrate objects into one's bodily self-representation, is a key aspect of human self-consciousness and has been shown to importantly shape human cognition. An extension of such feelings toward robots has been argued as being crucial for assistive technologies aiming at restoring, extending, or simulating sensorimotor functions. Empirical and theoretical work illustrates the importance of sensory feedback for the feeling of embodiment and also immersion; we focus on the the perceptual level of touch and the role of tactile feedback in various assistive robotic devices. We critically review how different facets of tactile perception in humans, i.e., affective, social, and self-touch, might influence embodiment. This is particularly important as current assistive robotic devices - such as prostheses, orthoses, exoskeletons, and devices for teleoperation-often limit touch low-density and spatially constrained haptic feedback, i.e., the mere touch sensation linked to an action. Here, we analyze, discuss, and propose how and to what degree tactile feedback might increase the embodiment of certain robotic devices, e.g., prostheses, and the feeling of immersion in human-robot interaction, e.g., in teleoperation. Based on recent findings from cognitive psychology on interactive processes between touch and embodiment, we discuss technical solutions for specific applications, which might be used to enhance embodiment, and facilitate the study of how embodiment might alter human-robot interactions. We postulate that high-density and large surface sensing and stimulation are required to foster embodiment of such assistive devices.

Selectivity and Longevity of Peripheral-Nerve and Machine Interfaces: A Review

For those individuals with upper-extremity amputation, a daily normal living activity is no longer possible or it requires additional effort and time. With the aim of restoring their sensory and motor functions, theoretical and technological investigations have been carried out in the field of neuroprosthetic systems. For transmission of sensory feedback, several interfacing modalities including indirect (non-invasive), direct-to-peripheral-nerve (invasive), and cortical stimulation have been applied. Peripheral nerve interfaces demonstrate an edge over the cortical interfaces due to the sensitivity in attaining cortical brain signals. The peripheral nerve interfaces are highly dependent on interface designs and are required to be biocompatible with the nerves to achieve prolonged stability and longevity. Another criterion is the selection of nerves that allows minimal invasiveness and damages as well as high selectivity for a large number of nerve fascicles. In this paper, we review the nerve-machine interface modalities noted above with more focus on peripheral nerve interfaces, which are responsible for provision of sensory feedback. The invasive interfaces for recording and stimulation of electro-neurographic signals include intra-fascicular, regenerative-type interfaces that provide multiple contact channels to a group of axons inside the nerve and the extra-neural-cuff-type interfaces that enable interaction with many axons around the periphery of the nerve. Section Current Prosthetic Technology summarizes the advancements made to date in the field of neuroprosthetics toward the achievement of a bidirectional nerve-machine interface with more focus on sensory feedback. In the Discussion section, the authors propose a hybrid interface technique for achieving better selectivity and long-term stability using the available nerve interfacing techniques.

Short- and Long-Term Learning of Feedforward Control of a Myoelectric Prosthesis with Sensory Feedback by Amputees

Human motor control relies on a combination of feedback and feedforward strategies. The aim of this study was to longitudinally investigate artificial somatosensory feedback and feedforward control in the context of grasping with myoelectric prosthesis. Nine amputee subjects performed routine grasping trials, with the aim to produce four levels of force during four blocks of 60 trials across five days. The electrotactile force feedback was provided in the second and third block using multipad electrode and spatial coding. The first baseline and last validation block (open-loop control) evaluated the effects of long- (across sessions) and short-term (within session) learning, respectively. The outcome measures were the absolute error between the generated and target force, and the number of force saturations. The results demonstrated that the electrotactile feedback improved the performance both within and across sessions. In the validation block, the performance did not significantly decrease and the quality of open-loop control (baseline) improved across days, converging to the performance characterizing closed-loop control. This paper provides important insights into the feedback and feedforward processes in prosthesis control, contributing to the better understanding of the role and design of feedback in prosthetic systems.

A System for Electrotactile Feedback Using Electronic Skin and Flexible Matrix Electrodes: Experimental Evaluation

Myoelectric prostheses are successfully controlled using muscle electrical activity, thereby restoring lost motor functions. However, the somatosensory feedback from the prosthesis to the user is still missing. The sensory substitution methods described in the literature comprise mostly simple position and force sensors combined with discrete stimulation units. The present study describes a novel system for sophisticated electrotactile feedback integrating advanced distributed sensing (electronic skin) and stimulation (matrix electrodes). The system was tested in eight healthy subjects who were asked to recognize the shape, trajectory, and direction of a set of dynamic movement patterns (single lines, geometrical objects, letters) presented on the electronic skin. The experiments demonstrated that the system successfully translated the mechanical interaction into the moving electrotactile profiles, which the subjects could recognize with a good performance (shape recognition: 86±8% lines, 73±13% geometries, 72±12% letters). In particular, the subjects could identify the movement direction with a high confidence. These results are in accordance with previous studies investigating the recognition of moving stimuli in human subjects. This is an important development towards closed-loop prostheses providing comprehensive and sophisticated tactile feedback to the user, facilitating the control and the embodiment of the artificial device into the user body scheme.

Electrotactile Feedback Improves Performance and Facilitates Learning in the Routine Grasping Task

Aim of this study was to investigate the feasibility of electrotactile feedback in closed loop training of force control during the routine grasping task. The feedback was provided using an array electrode and a simple six-level spatial coding, and the experiment was conducted in three amputee subjects. The psychometric tests confirmed that the subjects could perceive and interpret the electrotactile feedback with a high success rate. The subjects performed the routine grasping task comprising 4 blocks of 60 grasping trials. In each trial, the subjects employed feedforward control to close the hand and produce the desired grasping force (four levels). First (baseline) and the last (validation) session were performed in open loop, while the second and the third session (training) included electrotactile feedback. The obtained results confirmed that using the feedback improved the accuracy and precision of the force control. In addition, the subjects performed significantly better in the validation vs. baseline session, therefore suggesting that electrotactile feedback can be used for learning and training of myoelectric control.