The rubber hand illusion evaluated using different stimulation modalities

Tactile feedback plays a vital role in inducing ownership and improving motor control of prosthetic hands. However, commercially available prosthetic hands typically do not provide tactile feedback and because of that the prosthetic user must rely on visual input to adjust the grip. The classical rubber hand illusion (RHI) where a brush is stroking the rubber hand, and the user's hidden hand synchronously can induce ownership of a rubber hand. In the classic RHI the stimulation is modality-matched, meaning that the stimulus on the real hand matches the stimulus on the rubber hand. The RHI has also been used in previous studies with a prosthetic hand as the "rubber hand," suggesting that a hand prosthesis can be incorporated within the amputee's body scheme. Interestingly, previous studies have shown that stimulation with a mismatched modality, where the rubber hand was brushed, and vibrations were felt on the hidden hand also induced the RHI. The aim of this study was to compare how well mechanotactile, vibrotactile, and electrotactile feedback induced the RHI in able-bodied participants and forearm amputees. 27 participants with intact hands and three transradial amputees took part in a modified RHI experiment. The rubber hand was stroked with a brush, and the participant's hidden hand/residual limb received stimulation with either brush stroking, electricity, pressure, or vibration. The three latter stimulations were modality mismatched with regard to the brushstroke. Participants were tested for ten different combinations (stimulation blocks) where the stimulations were applied on the volar (glabrous skin), and dorsal (hairy skin) sides of the hand. Outcome was assessed using two standard tests (questionnaire and proprioceptive drift). All types of stimulation induced RHI but electrical and vibration stimulation induced a stronger RHI than pressure. After completing more stimulation blocks, the proprioceptive drift test showed that the difference between pre- and post-test was reduced. This indicates that the illusion was drifting toward the rubber hand further into the session.

Somatosensory Event-Related Potential as an Electrophysiological Correlate of Endogenous Spatial Tactile Attention: Prospects for Electrotactile Brain-Computer Interface for Sensory Training

Tactile attention tasks are used in the diagnosis and treatment of neurological and sensory processing disorders, while somatosensory event-related potentials (ERP) measured by electroencephalography (EEG) are used as neural correlates of attention processes. Brain-computer interface (BCI) technology provides an opportunity for the training of mental task execution via providing online feedback based on ERP measures. Our recent work introduced a novel electrotactile BCI for sensory training, based on somatosensory ERP; however, no previous studies have addressed specific somatosensory ERP morphological features as measures of sustained endogenous spatial tactile attention in the context of BCI control. Here we show the morphology of somatosensory ERP responses induced by a novel task introduced within our electrotactile BCI platform i.e., the sustained endogenous spatial electrotactile attention task. By applying pulsed electrical stimuli to the two proximal stimulation hotspots at the user's forearm, stimulating sequentially the mixed branches of radial and median nerves with equal probability of stimuli occurrence, we successfully recorded somatosensory ERPs for both stimulation locations, in the attended and unattended conditions. Waveforms of somatosensory ERP responses for both mixed nerve branches showed similar morphology in line with previous reports on somatosensory ERP components obtained by stimulation of exclusively sensory nerves. Moreover, we found statistically significant increases in ERP amplitude on several components, at both stimulation hotspots, while sustained endogenous spatial electrotactile attention task is performed. Our results revealed the existence of general ERP windows of interest and signal features that can be used to detect sustained endogenous tactile attention and classify between spatial attention locations in 11 healthy subjects. The current results show that features of N140, P3a and P3b somatosensory ERP components are the most prominent global markers of sustained spatial electrotactile attention, over all subjects, within our novel electrotactile BCI task/paradigm, and this work proposes the features of those components as markers of sustained endogenous spatial tactile attention in online BCI control. Immediate implications of this work are the possible improvement of online BCI control within our novel electrotactile BCI system, while these finding can be used for other tactile BCI applications in the diagnosis and treatment of neurological disorders by employing mixed nerve somatosensory ERPs and sustained endogenous electrotactile attention task as control paradigms.

Substitutive proprioception feedback of a prosthetic wrist by electrotactile stimulation

Objective

Sensory feedback of upper-limb prostheses is widely desired and studied. As important components of proprioception, position, and movement feedback help users to control prostheses better. Among various feedback methods, electrotactile stimulation is a potential method for coding proprioceptive information of a prosthesis. This study was motivated by the need for proprioception information for a prosthetic wrist. The flexion-extension (FE) position and movement information of the prosthetic wrist are transmitted back to the human body through multichannel electrotactile stimulation.

Approach

We developed an electrotactile scheme to encode the FE position and movement of the prosthetic wrist and designed an integrated experimental platform. A preliminary experiment on the sensory threshold and discomfort threshold was performed. Then, two proprioceptive feedback experiments were performed: a position sense experiment (Exp 1) and a movement sense experiment (Exp 2). Each experiment included a learning session and a test session. The success rate (SR) and discrimination reaction time (DRT) were analyzed to evaluate the recognition effect. The acceptance of the electrotactile scheme was evaluated by a questionnaire.

Main results

Our results showed that the average position SRs of five able-bodied subjects, amputee 1, and amputee 2 were 83.78, 97.78, and 84.44%, respectively. The average movement SR, and the direction and range SR of wrist movement in five able-bodied subjects were 76.25, 96.67%, respectively. Amputee 1 and amputee 2 had movement SRs of 87.78 and 90.00% and direction and range SRs of 64.58 and 77.08%, respectively. The average DRT of five able-bodied subjects was less than 1.5 s and that of amputees was less than 3.5 s.

Conclusion

The results indicate that after a short period of learning, the subjects can sense the position and movement of wrist FE. The proposed substitutive scheme has the potential for amputees to sense a prosthetic wrist, thus enhancing the human-machine interaction.

Encoding of spatial patterns using electrotactile stimulation via a multi-pad electrode placed on the torso

BACKGROUND

Tactile stimulation can be used to convey information to a user in different scenarios while avoiding overloading other senses. Tactile messages can be transmitted as spatial patterns, potentially allowing for a high information throughput. The aim of the present study was to design and test different encoding schemes to determine the best approach for conveying spatial patterns.

METHODS

Encoding schemes with simultaneous (SIM) and sequential pad activation (SEQ) were evaluated, including four SEQ variants designed to potentially facilitate the recognition. In SEQ-col and SEQ-row, the column and row of the activated pad were signified using different frequencies, while SEQ-all and SEQ-all-fast included the activation of all pads where those belonging to the pattern were indicated by changes in frequency (ON pads). The success rate (SR) of the pattern identification and the response time were quantified in 15 participants who recognized 20 patterns delivered through a 3 × 2 pad matrix placed on the lateral torso.

RESULTS

SIM was not a feasible method to present the patterns (median, 15%; IQR, 5%). The SR improved with SEQ (median, 60%; IQR, 20%) and further increased with additional cues, particularly with SEQ-row (median, 78.3%; IQR, 23.3%) and SEQ-all (median, 96.7%; IQR, 5%). Importantly, the stimulation time of SEQ-all could be decreased without a substantial drop in accuracy (SEQ-all-fast: median, 89.2%; IQR, 19.2%).

CONCLUSIONS

The spatiotemporal stimulation with sequential activation of all pads (SEQ-all) seems to be the method of choice when conveying tactile messages as spatial patterns. This is an important outcome for increasing the information bandwidth of communication through the tactile channel.

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Electrotactile feedback outweighs natural feedback in sensory integration during control of grasp force

Objective.The nervous system subconsciously estimates the state of the body as a weighted average of the information from various sensory sources, where the weights reflect the perceived reliability of each source. Loss of motor functions can be partially compensated using assistive systems (e.g. prostheses), which may also restore somatosensory feedback through tactile stimulation. Whether such artificial feedback is integrated in the neural state estimation process is not known.Approach.In this study, able-bodied subjects performed a grasp force matching task with supplementary non-invasive electrotactile stimulation with a frequency proportional to grasp force magnitude. Before the task, a brief training session taught the subjects to associate the sensation of electrotactile stimulation with the generated grasp force. In some trials, the force-frequency mapping was biased to introduce an unnoticeable mismatch between natural and electrotactile force feedback, thereby provoking the subject to subconsciously estimate the force as a compromise between the two sources of information.Main results.The outcome of this compromise revealed the weights assigned to each feedback type. The grasp forces were significantly affected by the biased mappings, as indicated by the average estimated relative weights (electrotactile: 0.69 ± 0.29; natural: 0.31 ± 0.29). Across subjects, this weight was correlatedr2=0.75) with the improvement in force matching precision when adding the unbiased electrotactile feedback to the natural force feedback, as predicted by maximum likelihood estimation. This shows that even after minimal training the nervous system adopts electrotactile stimulation as a highly reliable source of information that can improve the precision in the estimation of the grip force.Significance.This result has important implications for the restoration of sensory feedback in upper limb prostheses as it indicates that even non-invasive stimulation can be integrated naturally (i.e. subconsciously and effectively) in the motor controlloop.

Temporal Asynchrony but Not Total Energy Nor Duration Improves the Judgment of Numerosity in Electrotactile Stimulation

Stroke patients suffer from impairments of both motor and somatosensory functions. The functional recovery of upper extremities is one of the primary goals of rehabilitation programs. Additional somatosensory deficits limit sensorimotor function and significantly affect its recovery after the neuromotor injury. Sensory substitution systems, providing tactile feedback, might facilitate manipulation capability, and improve patient's dexterity during grasping movements. As a first step toward this aim, we evaluated the ability of healthy subjects in exploiting electrotactile feedback on the shoulder to determine the number of perceived stimuli in numerosity judgment tasks. During the experiment, we compared four different stimulation patterns (two simultaneous: short and long, intermittent and sequential) differing in total duration, total energy, or temporal synchrony. The experiment confirmed that the subject ability to enumerate electrotactile stimuli decreased with increasing the number of active electrodes. Furthermore, we found that, in electrotactile stimulation, the temporal coding schemes, and not total energy or duration modulated the accuracy in numerosity judgment. More precisely, the sequential condition resulted in significantly better numerosity discrimination than intermittent and simultaneous stimulation. These findings, together with the fact that the shoulder appeared to be a feasible stimulation site to communicate tactile information via electrotactile feedback, can serve as a guide to deliver tactile feedback to proximal areas in stroke survivors who lack sensory integrity in distal areas of their affected arm, but retain motor skills.

Electrotactile Feedback Improves Performance and Facilitates Learning in the Routine Grasping Task

Aim of this study was to investigate the feasibility of electrotactile feedback in closed loop training of force control during the routine grasping task. The feedback was provided using an array electrode and a simple six-level spatial coding, and the experiment was conducted in three amputee subjects. The psychometric tests confirmed that the subjects could perceive and interpret the electrotactile feedback with a high success rate. The subjects performed the routine grasping task comprising 4 blocks of 60 grasping trials. In each trial, the subjects employed feedforward control to close the hand and produce the desired grasping force (four levels). First (baseline) and the last (validation) session were performed in open loop, while the second and the third session (training) included electrotactile feedback. The obtained results confirmed that using the feedback improved the accuracy and precision of the force control. In addition, the subjects performed significantly better in the validation vs. baseline session, therefore suggesting that electrotactile feedback can be used for learning and training of myoelectric control.

The impact of the stimulation frequency on closed-loop control with electrotactile feedback

BACKGROUND

Electrocutaneous stimulation can restore the missing sensory information to prosthetic users. In electrotactile feedback, the information about the prosthesis state is transmitted in the form of pulse trains. The stimulation frequency is an important parameter since it influences the data transmission rate over the feedback channel as well as the form of the elicited tactile sensations.

METHODS

We evaluated the influence of the stimulation frequency on the subject's ability to utilize the feedback information during electrotactile closed-loop control. Ten healthy subjects performed a real-time compensatory tracking (standard test bench) of sinusoids and pseudorandom signals using either visual feedback (benchmark) or electrocutaneous feedback in seven conditions characterized by different combinations of the stimulation frequency (FSTIM) and tracking error sampling rate (FTE). The tracking error was transmitted using two concentric electrodes placed on the forearm. The quality of tracking was assessed using the Squared Pearson Correlation Coefficient (SPCC), the Normalized Root Mean Square Tracking Error (NRMSTE) and the time delay between the reference and generated trajectories (TDIO).

RESULTS

The results demonstrated that FSTIM was more important for the control performance than FTE. The quality of tracking deteriorated with a decrease in the stimulation frequency, SPCC and NRMSTE (mean) were 87.5% and 9.4% in the condition 100/100 (FTE/FSTIM), respectively, and deteriorated to 61.1% and 15.3% in 5/5, respectively, while the TDIO increased from 359.8 ms in 100/100 to 1009 ms in 5/5. However, the performance recovered when the tracking error sampled at a low rate was delivered using a high stimulation frequency (SPCC = 83.6%, NRMSTE = 10.3%, TDIO = 415.6 ms, in 5/100).

CONCLUSIONS

The likely reason for the performance decrease and recovery was that the stimulation frequency critically influenced the tactile perception quality and thereby the effective rate of information transfer through the feedback channel. The outcome of this study can facilitate the selection of optimal system parameters for somatosensory feedback in upper limb prostheses. The results imply that the feedback variables (e.g., grasping force) should be transmitted at relatively high frequencies of stimulation (>25 Hz), but that they can be sampled at much lower rates (e.g., 5 Hz).

Efficacy of electrotactile vestibular substitution in patients with peripheral and central vestibular loss

Vestibular dysfunction of either central or peripheral origin can significantly affect balance, posture, and gait. We conducted a pilot study to test the effectiveness of training with the BrainPort balance device in subjects with a balance dysfunction due to peripheral or central vestibular loss. The BrainPort balance device transmits information about the patient's head position via electrotactile stimulation of the tongue. Head position data is sensed by an accelerometer and displayed on the tongue as a pattern of stimulation. This pattern of stimulation moves forward, backward, and laterally on the tongue in direct response to head movements. Users of the device were trained to use this stimulation to adjust their position in order to maintain their balance. Twenty-eight subjects with peripheral or central vestibular loss were trained with the BrainPort balance device and tested using the following standardized quantitative measurements of the treatment effects: Computerized Dynamic Posturography (CDP) using the Sensory Organization Test (SOT), Dynamic Gait Index (DGI), Activities-specific Balance Confidence Scale (ABC), and Dizziness Handicap Inventory (DHI). All subjects had chronic balance problems and all but one had previously participated in vestibular rehabilitation therapy. The scores on the clinical tests upon entry into the study were compared to their scores following training with the BrainPort balance device. Our results exhibit consistent positive and statistically significant improvements in balance, posture and gait. These results exceed what could normally be achieved in three to five days of traditional balance training alone. Since this was not a controlled study, we are unable to distinguish the degree to which these improvements are attributable to training with the BrainPort balance device versus the balance exercises performed by all subjects as a part of the BrainPort training sessions. Nonetheless, after training with the BrainPort balance device, all subjects demonstrated significant improvements in performance beyond what might be expected from conventional vestibular rehabilitation therapy.