Kinesthetic Feedback During 2DOF Wrist Movements via a Novel MR-Compatible Robot

Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal Device

A goal of wearable haptic devices has been to enable haptic communication, where individuals learn to map information typically processed visually or aurally to haptic cues via a process of cross-modal associative learning. Neural correlates have been used to evaluate haptic perception and may provide a more objective approach to assess association performance than more commonly used behavioral measures of performance. In this article, we examine Representational Similarity Analysis (RSA) of electroencephalography (EEG) as a framework to evaluate how the neural representation of multifeatured haptic cues changes with association training. We focus on the first phase of cross-modal associative learning, perception of multimodal cues. A participant learned to map phonemes to multimodal haptic cues, and EEG data were acquired before and after training to create neural representational spaces that were compared to theoretical models. Our perceptual model showed better correlations to the neural representational space before training, while the feature-based model showed better correlations with the post-training data. These results suggest that training may lead to a sharpening of the sensory response to haptic cues. Our results show promise that an EEG-RSA approach can capture a shift in the representational space of cues, as a means to track haptic learning.

Changes in Resting State Functional Connectivity Associated with Dynamic Adaptation of Wrist Movements

Dynamic adaptation is an error-driven process of adjusting planned motor actions to changes in task dynamics (Shadmehr, 2017). Adapted motor plans are consolidated into memories that contribute to better performance on re-exposure. Consolidation begins within 15 min following training (Criscimagna-Hemminger and Shadmehr, 2008), and can be measured via changes in resting state functional connectivity (rsFC). For dynamic adaptation, rsFC has not been quantified on this timescale, nor has its relationship to adaptative behavior been established. We used a functional magnetic resonance imaging (fMRI)-compatible robot, the MR-SoftWrist (Erwin et al., 2017), to quantify rsFC specific to dynamic adaptation of wrist movements and subsequent memory formation in a mixed-sex cohort of human participants. We acquired fMRI during a motor execution and a dynamic adaptation task to localize brain networks of interest, and quantified rsFC within these networks in three 10-min windows occurring immediately before and after each task. The next day, we assessed behavioral retention. We used a mixed model of rsFC measured in each time window to identify changes in rsFC with task performance, and linear regression to identify the relationship between rsFC and behavior. Following the dynamic adaptation task, rsFC increased within the cortico-cerebellar network and decreased interhemispherically within the cortical sensorimotor network. Increases within the cortico-cerebellar network were specific to dynamic adaptation, as they were associated with behavioral measures of adaptation and retention, indicating that this network has a functional role in consolidation. Instead, decreases in rsFC within the cortical sensorimotor network were associated with motor control processes independent from adaptation and retention.SIGNIFICANCE STATEMENT Motor memory consolidation processes have been studied via functional magnetic resonance imaging (fMRI) by analyzing changes in resting state functional connectivity (rsFC) occurring more than 30 min after adaptation. However, it is unknown whether consolidation processes are detectable immediately (<15 min) following dynamic adaptation. We used an fMRI-compatible wrist robot to localize brain regions involved in dynamic adaptation in the cortico-thalamic-cerebellar (CTC) and cortical sensorimotor networks and quantified changes in rsFC within each network immediately after adaptation. Different patterns of change in rsFC were observed compared with studies conducted at longer latencies. Increases in rsFC in the cortico-cerebellar network were specific to adaptation and retention, while interhemispheric decreases in the cortical sensorimotor network were associated with alternate motor control processes but not with memory formation.

Concurrent Contribution of Co-contraction to Error Reduction during Dynamic Adaptation of the Wrist

MRI-compatible robots provide a means of studying brain function involved in complex sensorimotor learning processes, such as adaptation. To properly interpret the neural correlates of behavior measured using MRI-compatible robots, it is critical to validate the measurements of motor performance obtained via such devices. Previously, we characterized adaptation of the wrist in response to a force field applied via an MRI-compatible robot, the MR-SoftWrist. Compared to arm reaching tasks, we observed lower end magnitude of adaptation, and reductions in trajectory errors beyond those explained by adaptation. Thus, we formed two hypotheses: that the observed differences were due to measurement errors of the MR-SoftWrist; or that impedance control plays a significant role in control of wrist movements during dynamic perturbations. To test both hypotheses, we performed a two-session counterbalanced crossover study. In both sessions, participants performed wrist pointing in three force field conditions (zero force, constant, random). Participants used either the MR-SoftWrist or the UDiffWrist, a non-MRI-compatible wrist robot, for task execution in session one, and the other device in session two. To measure anticipatory co-contraction associated with impedance control, we collected surface EMG of four forearm muscles. We found no significant effect of device on behavior, validating the measurements of adaptation obtained with the MR-SoftWrist. EMG measures of co-contraction explained a significant portion of the variance in excess error reduction not attributable to adaptation. These results support the hypothesis that for the wrist, impedance control significantly contributes to reductions in trajectory errors in excess of those explained by adaptation.

Human Perception of Wrist Flexion and Extension Torque During Upper and Lower Extremity Movement

Real-world application of haptic feedback from kinesthetic devices is implemented while the user is in motion, but human wrist torque magnitude discrimination has previously only been characterized while users are stationary. In this study, we measured wrist torque discrimination in conditions relevant to activities of daily living, using a previously developed backdrivable wrist exoskeleton capable of applying wrist flexion and extension torque. We implemented a torque comparison test using a two-alternative forced-choice paradigm while participants were both seated and walking on a treadmill, with both a stationary and a moving wrist. Like most kinesthetic haptic devices, the wrist exoskeleton output torque is commanded in an open-loop manner. Thus, the study design was informed by Monte Carlo simulations to verify that the errors in the wrist exoskeleton output torque would not significantly affect the results. Results from ten participants show that although both walking and moving wrist conditions result in higher Weber Fractions (worse perception), participants were able to detect relatively small changes in torque of 12-19% on average in all grouped conditions. The results provide insight regarding the torque magnitudes necessary to make wrist-worn kinesthetic haptic devices noticeable and meaningful to the user in various conditions relevant to activities of daily living.

Full-hand electrotactile feedback using electronic skin and matrix electrodes for high-bandwidth human-machine interfacing

Tactile feedback is relevant in a broad range of human-machine interaction systems (e.g. teleoperation, virtual reality and prosthetics). The available tactile feedback interfaces comprise few sensing and stimulation units, which limits the amount of information conveyed to the user. The present study describes a novel technology that relies on distributed sensing and stimulation to convey comprehensive tactile feedback to the user of a robotic end effector. The system comprises six flexible sensing arrays (57 sensors) integrated on the fingers and palm of a robotic hand, embedded electronics (64 recording channels), a multichannel stimulator and seven flexible electrodes (64 stimulation pads) placed on the volar side of the subject's hand. The system was tested in seven subjects asked to recognize contact positions and identify contact sliding on the electronic skin, using distributed anode configuration (DAC) and single dedicated anode configuration. The experiments demonstrated that DAC resulted in substantially better performance. Using DAC, the system successfully translated the contact patterns into electrotactile profiles that the subjects could recognize with satisfactory accuracy ([Formula: see text] for static and [Formula: see text] for dynamic patterns). The proposed system is an important step towards the development of a high-density human-machine interfacing between the user and a robotic hand. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Evaluation of a Portable fMRI Compatible Robotic Wrist Interface

This paper presents evaluation of a portable fMRI compatible haptic interface to study the brain correlates of sensorimotor control during wrist motion. The interface is actuated by a shielded DC motor located more than 2 m away from the 3T MR scanner's bore. The achievable wrist torque of the interface is up to 2 Nm, and the interface provides sufficient bandwidth for human motor control experiments. Ergonomic and fMRI compatibility testing with a 3T MR scanner showed that the interface is MR safe, compatible with a strong static magnetic field and radio frequency emission, and its operation does not affect the quality of the acquired images. Clinical Relevance- We present and evaluate an fMRI compatible robotic interface to study human wrist joint motor function.

Development and placement accuracy evaluation of an MR conditional robot for prostate intervention

Robot-assisted prostate intervention under magnetic resonance imaging (MRI) guidance is a promising method to improve the clinical performance compared with the manual method. An MR conditional 6-DOF prostate intervention serial robot is developed and a binocular vision system (BVS) is established to evaluate the needle placement accuracy and located the penetration point precisely. The robot is designed by the MR conditional criteria. The serial configuration of the robot provides adequate flexibility and large workspace and excellent friendliness to the physicians. The kinematics are deduced and the needle placement control flow is proposed according to the configuration of the robot. The robot-assisted prostate intervention is divided into two phases including needle placement and needle penetration. A custom-made robust BVS is developed to obtain the needle tip position automatically in the needle placement phase where the needle cannot be detected by the MRI for lack of hydrogen atom. A simple and general algorithm used for needle tip camera coordinate estimation is proposed. Experiments on the BVS validation and robot accuracy evaluation are performed. The experiment results show that the errors of the BVS are under 0.3621 mm and the position error of the proposed robot is 2.815 mm which indicate the adequate accuracy for the prostate intervention. Graphical abstract.

The SE-AssessWrist for robot-aided assessment of wrist stiffness and range of motion: Development and experimental validation

Introduction

Physical human-robot interaction offers a compelling platform for assessing recovery from neurological injury; however, robots currently used for assessment have typically been designed for the requirements of rehabilitation, not assessment. In this work, we present the design, control, and experimental validation of the SE-AssessWrist, which extends the capabilities of prior robotic devices to include complete wrist range of motion assessment in addition to stiffness evaluation.

Methods

The SE-AssessWrist uses a Bowden cable-based transmission in conjunction with series elastic actuation to increase device range of motion while not sacrificing torque output. Experimental validation of robot-aided wrist range of motion and stiffness assessment was carried out with five able-bodied individuals.

Results

The SE-AssessWrist achieves the desired maximum wrist range of motion, while having sufficient position and zero force control performance for wrist biomechanical assessment. Measurements of two-degree-of-freedom wrist range of motion and stiffness envelopes revealed that the axis of greatest range of motion and least stiffness were oblique to the conventional anatomical axes, and approximately parallel to each other.

Conclusions

Such an assessment could be beneficial in the clinic, where standard clinical measures of recovery after neurological injury are subjective, labor intensive, and graded on an ordinal scale.

MR-Conditional Actuations: A Review

Magnetic resonance imaging (MRI) is one of the most prevailing technologies to enable noninvasive and radiation-free soft tissue imaging. Operating a robotic device under MRI guidance is an active research area that has the potential to provide efficient and precise surgical therapies. MR-conditional actuators that can safely drive these robotic devices without causing safety hazards or adversely affecting the image quality are crucial for the development of MR-guided robotic devices. This paper aims to summarize recent advances in actuation methods for MR-guided robots and each MR-conditional actuator was reviewed based on its working principles, construction materials, the noteworthy features, and corresponding robotic application systems, if any. Primary characteristics, such as torque, force, accuracy, and signal-to-noise ratio (SNR) variation due to the variance of the actuator, are also covered. This paper concludes with a perspective on the current development and future of MR-conditional actuators.

StretchfMRI: a novel technique to quantify the contribution of the reticular formation to long-latency responses via fMRI

Increased reticulospinal (RS) function has been observed to cause both positive and negative outcomes in the recovery of motor function after corticospinal lesions such as stroke. Current knowledge of RS function is limited by the lack of accurate, noninvasive methods for measuring RS function. Recent studies suggest that the RS tract may be involved in processing and generating Long Latency Responses (LLRs). As such, LLRs, elicited by applying precisely controlled perturbations, can thus act as a reliable stimulus to measure brainstem function using fMRI with high signal-to-noise ratio.In this paper, we present StretchfMRI, a novel technique that enables simultaneous recording of neural and muscular activity during motor responses conditioned by robotic perturbations, which allows direct investigation of the neural correlates of LLRs.Via preliminary validation experiments, we demonstrate that our technique can reliably elicit and identify LLRs in two wrist muscles-Flexor Carpi Radialis and Extensor Carpi Ulnaris. Moreover, via a single-subject pilot experiment, we show that the occurrence of an LLR in a flexor and extensor muscles modulates neural activity in distinct regions of the brainstem. The observed somatotopic organization is in agreement with the double reciprocal model of RS function observed in animal models, in which the right medullary and left pontine reticular formation are responsible for control of the motor activity in flexors and extensors, respectively.

A Soft Robotic Wearable Wrist Device for Kinesthetic Haptic Feedback

Advances in soft robotics provide a unique approach for delivering haptic feedback to a user by a soft wearable device. Such devices can apply forces directly on the human joints, while still maintaining the safety and flexibility necessary for use in close proximity to the human body. To take advantage of these properties, we present a new haptic wrist device using pressure-driven soft actuators called reverse pneumatic artificial muscles (rPAMs) mounted on four sides of the wrist. These actuators are originally pre-strained and release compressive stress under pressure, applying a safe torque around the wrist joints while being compact and portable, representing the first soft haptic device capable of real-time feedback. To demonstrate the functional utility of this device, we created a virtual path-following task, wherein the user employs the motion of their wrist to control their embodied agent. We used the haptic wrist device to assist the user in following the path and study their performance with and without haptic feedback in multiple scenarios. Our results quantify the effect of wearable soft robotic haptic feedback on user performance. Specifically, we observed that our haptic feedback system improved the performance of users following complicated paths in a statistically significant manner, but did not show improvement for simple linear paths. Based on our findings, we anticipate broader applications of wearable soft robotic haptic devices toward intuitive user interactions with robots, computers, and other users.

MR-Compatible Haptic Display of Membrane Puncture in Robot-Assisted Needle Procedures

Multilayer electroactive polymer films actuate a small hand-held device that can display tool tip forces during MR-guided interventions. The display produces localized skin stretch at the thumb and index fingertips. Tests confirm that the device does not significantly affect MR imaging and produces detectable stimuli in response to forces measured by a biopsy needle instrumented with optical fibers. Tests with human subjects explored robotic and teleoperated paradigms to detect when the needle contacted a membrane embedded at variable depth in a tissue phantom that approximated the properties of porcine liver. In the first case, naive users detected membranes with a 98.9% success rate as the needle was driven at fixed speed. In the second case, users with experience in needle-based procedures controlled the needle insertion and detected membranes embedded in tissue phantoms with a 98% success rate. In the second experiment, some users detected membranes with very light contact forces, but there was greater subject-to-subject variation.

Effects of Assist-As-Needed Upper Extremity Robotic Therapy after Incomplete Spinal Cord Injury: A Parallel-Group Controlled Trial

BACKGROUND

Robotic rehabilitation of the upper limb following neurological injury has been supported through several large clinical studies for individuals with chronic stroke. The application of robotic rehabilitation to the treatment of other neurological injuries is less developed, despite indications that strategies successful for restoration of motor capability following stroke may benefit individuals with incomplete spinal cord injury (SCI) as well. Although recent studies suggest that robot-aided rehabilitation might be beneficial after incomplete SCI, it is still unclear what type of robot-aided intervention contributes to motor recovery.

METHODS

We developed a novel assist-as-needed (AAN) robotic controller to adjust challenge and robotic assistance continuously during rehabilitation therapy delivered via an upper extremity exoskeleton, the MAHI Exo-II, to train independent elbow and wrist joint movements. We further enrolled seventeen patients with incomplete spinal cord injury (AIS C and D levels) in a parallel-group balanced controlled trial to test the efficacy of the AAN controller, compared to a subject-triggered (ST) controller that does not adjust assistance or challenge levels continuously during therapy. The conducted study is a stage two, development-of-concept pilot study.

RESULTS

We validated the AAN controller in its capability of modulating assistance and challenge during therapy via analysis of longitudinal robotic metrics. For the selected primary outcome measure, the pre-post difference in ARAT score, no statistically significant change was measured in either group of subjects. Ancillary analysis of secondary outcome measures obtained via robotic testing indicates gradual improvement in movement quality during the therapy program in both groups, with the AAN controller affording greater increases in movement quality over the ST controller.

CONCLUSION

The present study demonstrates feasibility of subject-adaptive robotic therapy after incomplete spinal cord injury, but does not demonstrate gains in arm function occurring as a result of the robot-assisted rehabilitation program, nor differential gains obtained as a result of the developed AAN controller. Further research is warranted to better quantify the recovery potential provided by AAN control strategies for robotic rehabilitation of the upper limb following incomplete SCI. ClinicalTrials.gov registration number: NCT02803255.