Myoelectric prosthesis users and non-disabled individuals wearing a simulated prosthesis exhibit similar compensatory movement strategies

A multifaceted suite of metrics for comparative myoelectric prosthesis controller research

Upper limb robotic (myoelectric) prostheses are technologically advanced, but challenging to use. In response, substantial research is being done to develop person-specific prosthesis controllers that can predict a user's intended movements. Most studies that test and compare new controllers rely on simple assessment measures such as task scores (e.g., number of objects moved across a barrier) or duration-based measures (e.g., overall task completion time). These assessment measures, however, fail to capture valuable details about: the quality of device arm movements; whether these movements match users' intentions; the timing of specific wrist and hand control functions; and users' opinions regarding overall device reliability and controller training requirements. In this work, we present a comprehensive and novel suite of myoelectric prosthesis control evaluation metrics that better facilitates analysis of device movement details-spanning measures of task performance, control characteristics, and user experience. As a case example of their use and research viability, we applied these metrics in real-time control experimentation. Here, eight participants without upper limb impairment compared device control offered by a deep learning-based controller (recurrent convolutional neural network-based classification with transfer learning, or RCNN-TL) to that of a commonly used controller (linear discriminant analysis, or LDA). The participants wore a simulated prosthesis and performed complex functional tasks across multiple limb positions. Analysis resulting from our suite of metrics identified 16 instances of a user-facing problem known as the "limb position effect". We determined that RCNN-TL performed the same as or significantly better than LDA in four such problem instances. We also confirmed that transfer learning can minimize user training burden. Overall, this study contributes a multifaceted new suite of control evaluation metrics, along with a guide to their application, for use in research and testing of myoelectric controllers today, and potentially for use in broader rehabilitation technologies of the future.

Restoring natural upper limb movement through a wrist prosthetic module for partial hand amputees

BACKGROUND

Most partial hand amputees experience limited wrist movement. The limited rotational wrist movement deteriorates natural upper limb system related to hand use and the usability of the prosthetic hand, which may cause secondary damage to the musculoskeletal system due to overuse of the upper limb affected by repetitive compensatory movement patterns. Nevertheless, partial hand prosthetics, in common, have only been proposed without rotational wrist movement because patients have various hand shapes, and a prosthetic hand should be attached to a narrow space.

METHODS

We hypothesized that partial hand amputees, when using a prosthetic hand with a wrist rotation module, would achieve natural upper limb movement muscle synergy and motion analysis comparable to a control group. To validate the proposed prototype design with the wrist rotation module and verify our hypothesis, we compared a control group with partial hand amputees wearing hand prostheses, both with and without the wrist rotation module prototype. The study contained muscle synergy analysis through non-negative matrix factorization (NMF) using surface electromyography (sEMG) and motion analyses employing a motion capture system during the reach-to-grasp task. Additionally, we assessed the usability of the prototype design for partial hand amputees using the Jebsen-Taylor hand function test (JHFT).

RESULTS

The results showed that the number of muscle synergies identified through NMF remained consistent at 3 for both the control group and amputees using a hand prosthesis with a wrist rotation module. In the motion analysis, a statistically significant difference was observed between the control group and the prosthetic hand without the wrist rotation module, indicating the presence of compensatory movements when utilizing a prosthetic hand lacking this module. Furthermore, among the amputees, the JHFT demonstrated a greater improvement in total score when using the prosthetic hand equipped with a wrist rotation module compared to the prosthetic hand without this module.

CONCLUSION

In conclusion, integrating a wrist rotation module in prosthetic hand designs for partial hand amputees restores natural upper limb movement patterns, reduces compensatory movements, and prevent the secondary musculoskeletal. This highlights the importance of this module in enhancing overall functionality and quality of life.

A Case Series in Position-Aware Myoelectric Prosthesis Control Using Recurrent Convolutional Neural Network Classification with Transfer Learning

Position-aware myoelectric prosthesis controllers require long, data-intensive training routines. Transfer Learning (TL) might reduce training burden. A TL model can be pre-trained using forearm muscle signal data from many individuals to become the starting point for a new user. A recurrent convolutional neural network (RCNN)-based classifier has already been shown to benefit from TL in offline analysis (95% accuracy). The present real-time study tested whether an RCNN-based classification controller with TL (RCNN-TL) could reduce training burden, offer improved device control (per functional task performance metrics), and mitigate what is known as the "limb position effect". 27 participants without amputation were recruited. 19 participants performed wrist/hand movements across multiple limb positions, with resulting forearm muscle signal data used to pre-train RCNN-TL. 8 other participants donned a simulated prosthesis, retrained (calibrated) and tested RCNN-TL, plus trained and tested a conventional linear discriminant analysis classification controller (LDA-Baseline). Results confirmed that TL reduces user training burden. RCNN-TL yielded improved task performance durations over LDA-Baseline (in specific Grasp and Release phases), yet other metrics worsened. Overall, this work contributes training condition factors necessary for TL success, identifies metrics needed for comprehensive control analysis, and contributes insights towards improved position-aware control.

A scoping review of eye tracking metrics used to assess visuomotor behaviours of upper limb prosthesis users

Advanced upper limb prostheses aim to restore coordinated hand and arm function. However, this objective can be difficult to quantify as coordinated movements require an intact visuomotor system. Eye tracking has recently been applied to study the visuomotor behaviours of upper limb prosthesis users by enabling the calculation of eye movement metrics. This scoping review aims to characterize the visuomotor behaviours of upper limb prosthesis users as described by eye tracking metrics, to summarize the eye tracking metrics used to describe prosthetic behaviour, and to identify gaps in the literature and potential areas for future research. A review of the literature was performed to identify articles that reported eye tracking metrics to evaluate the visual behaviours of individuals using an upper limb prosthesis. Data on the level of amputation, type of prosthetic device, type of eye tracker, primary eye metrics, secondary outcome metrics, experimental task, aims, and key findings were extracted. Seventeen studies were included in this scoping review. A consistently reported finding is that prosthesis users have a characteristic visuomotor behaviour that differs from that of individuals with intact arm function. Visual attention has been reported to be directed more towards the hand and less towards the target during object manipulation tasks. A gaze switching strategy and delay to disengage gaze from the current target has also been reported. Differences in the type of prosthetic device and experimental task have revealed some distinct gaze behaviours. Control factors have been shown to be related to gaze behaviour, while sensory feedback and training interventions have been demonstrated to reduce the visual attention associated with prosthesis use. Eye tracking metrics have also been used to assess the cognitive load and sense of agency of prosthesis users. Overall, there is evidence that eye tracking is an effective tool to quantitatively assess the visuomotor behaviour of prosthesis users and the recorded eye metrics are sensitive to change in response to various factors. Additional studies are needed to validate the eye metrics used to assess cognitive load and sense of agency in upper limb prosthesis users.

Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation

BACKGROUND

Virtual and augmented reality (AR) have become popular modalities for training myoelectric prosthesis control with upper-limb amputees. While some systems have shown moderate success, it is unclear how well the complex motor skills learned in an AR simulation transfer to completing the same tasks in physical reality. Limb loading is a possible dimension of motor skill execution that is absent in current AR solutions that may help to increase skill transfer between the virtual and physical domains.

METHODS

We implemented an immersive AR environment where individuals could operate a myoelectric virtual prosthesis to accomplish a variety of object relocation manipulations. Intact limb participants were separated into three groups, the load control (CG_LD; [Formula: see text]), the AR control (CG_AR; [Formula: see text]), and the experimental group (EG; [Formula: see text]). Both the CG_AR and EG completed a 5-session prosthesis training protocol in AR while the CG_LD performed simple muscle training. The EG attempted manipulations in AR while undergoing limb loading. The CG_AR attempted the same manipulations without loading. All participants performed the same manipulations in physical reality while operating a real prosthesis pre- and post-training. The main outcome measure was the change in the number of manipulations completed during the physical reality assessments (i.e. completion rate). Secondary outcomes included movement kinematics and visuomotor behavior.

RESULTS

The EG experienced a greater increase in completion rate post-training than both the CG_AR and CG_LD. This performance increase was accompanied by a shorter motor learning phase, the EG's performance saturating in less sessions of AR training than the CG_AR.

CONCLUSION

The results demonstrated that limb loading plays an important role in transferring complex motor skills learned in virtual spaces to their physical reality analogs. While participants who did not receive limb loading were able to receive some functional benefit from AR training, participants who received the loading experienced a greater positive change in motor performance with their performance saturating in fewer training sessions.

Wrist speed feedback improves elbow compensation and reaching accuracy for myoelectric transradial prosthesis users in hybrid virtual reaching task

BACKGROUND

Myoelectric prostheses are a popular choice for restoring motor capability following the loss of a limb, but they do not provide direct feedback to the user about the movements of the device-in other words, kinesthesia. The outcomes of studies providing artificial sensory feedback are often influenced by the availability of incidental feedback. When subjects are blindfolded and disconnected from the prosthesis, artificial sensory feedback consistently improves control; however, when subjects wear a prosthesis and can see the task, benefits often deteriorate or become inconsistent. We theorize that providing artificial sensory feedback about prosthesis speed, which cannot be precisely estimated via vision, will improve the learning and control of a myoelectric prosthesis.

METHODS

In this study, we test a joint-speed feedback system with six transradial amputee subjects to evaluate how it affects myoelectric control and adaptation behavior during a virtual reaching task.

RESULTS

Our results showed that joint-speed feedback lowered reaching errors and compensatory movements during steady-state reaches. However, the same feedback provided no improvement when control was perturbed.

CONCLUSIONS

These outcomes suggest that the benefit of joint speed feedback may be dependent on the complexity of the myoelectric control and the context of the task.

Communicative capital: a key resource for human-machine shared agency and collaborative capacity

In this work, we present a perspective on the role machine intelligence can play in supporting human abilities. In particular, we consider research in rehabilitation technologies such as prosthetic devices, as this domain requires tight coupling between human and machine. Taking an agent-based view of such devices, we propose that human-machine collaborations have a capacity to perform tasks which is a result of the combined agency of the human and the machine. We introduce communicative capital as a resource developed by a human and a machine working together in ongoing interactions. Development of this resource enables the partnership to eventually perform tasks at a capacity greater than either individual could achieve alone. We then examine the benefits and challenges of increasing the agency of prostheses by surveying literature which demonstrates that building communicative resources enables more complex, task-directed interactions. The viewpoint developed in this article extends current thinking on how best to support the functional use of increasingly complex prostheses, and establishes insight toward creating more fruitful interactions between humans and supportive, assistive, and augmentative technologies.

Estimating speed-accuracy trade-offs to evaluate and understand closed-loop prosthesis interfaces

OBJECTIVE

Closed-loop prosthesis interfaces, which combine electromyography (EMG)-based control with supplementary feedback, represent a promising direction for developing the next generation of bionic limbs. However, we still lack an understanding of how users utilize these interfaces and how to evaluate competing solutions. In this study, we used the framework of speed-accuracy trade-off functions (SAF) to understand, evaluate, and compare the performance of two closed-loop user-prosthesis interfaces.

APPROACH

Ten able-bodied participants and an amputee performed a force-matching task in a functional box-and-block setup at three different speeds. All participants were subjected to both interfaces in a crossover study design with a one-week washout period. Importantly, both interfaces used (identical) direct proportional control but differed in the feedback provided to the participant (EMG feedback vs. Force feedback). Therefore, we estimated the SAFs afforded by the two interfaces and sought to understand how the participants planned and executed the task under the various conditions.

MAIN RESULTS

We found that execution speed significantly influenced performance, and that EMG feedback afforded better overall performance, especially at medium speeds. Notably, we found that there was a difference in the SAF between the two interfaces, with EMG feedback enabling participants to attain higher accuracies faster than Force feedback. Furthermore, both interfaces enabled participants to develop flexible control policies, while EMG feedback also afforded participants the ability to generate smoother, more repeatable EMG commands.

SIGNIFICANCE

Overall, the results indicate that the performance of closed-loop prosthesis interfaces depends critically on the feedback approach and execution speed. This study showed that the SAF framework could be used to reveal the differences between feedback approaches, which might not have been detected if the assessment was performed at a single speed. Therefore, we argue that it is important to consider the speed-accuracy trade-offs to rigorously evaluate and compare user-prosthesis interfaces.

Spatiotemporal Coupling of Hand and Eye Movements When Using a Myoelectric Prosthetic Hand

Upper limb prosthesis users have disruptions in hand-eye coordination, with increased fixations towards the hand and less visual allocation for feedforward planning. The purpose of this study was to explore whether improved motor planning, as reflected by eye gaze behaviour, was associated with more efficient hand movement patterns. Able-bodied participants wore a simulated prosthesis while performing a functional object movement task. Motion and eye tracking data were collected to quantify the eye gaze and hand movement during object interaction. The results of this study demonstrated that the latency of the eye to precede the hand at pick-up was correlated with measures of hand function, including hand variability, movement units, and grasp time, but not reach time. During transport and release, longer latency to disengage gaze from the grasped object and look ahead towards the target was correlated to hand kinematics of hand variability, distance travelled, and transport time. In addition, the latency of the eye to disengage the drop-off location was correlated to release time. Together these may point to control issues with opening and closing the prosthetic hand. Overall, increased feedforward fixations towards the target and reduced feedback fixations towards the hand were related to improved measures of hand function. Hence, coordination between eye and hand movements when using a myoelectric prosthesis may prove to be a useful metric to assess motor planning.

Recurrent Convolutional Neural Networks as an Approach to Position-Aware Myoelectric Prosthesis Control

OBJECTIVE

Persons with normal arm function can perform complex wrist and hand movements over a wide range of limb positions. However, for those with transradial amputation who use myoelectric prostheses, control across multiple limb positions can be challenging, frustrating, and can increase the likelihood of device abandonment. In response, the goal of this research was to investigate recurrent convolutional neural network (RCNN)-based position-aware myoelectric prosthesis control strategies.

METHODS

Surface electromyographic (EMG) and inertial measurement unit (IMU) signals, obtained from 16 non-disabled participants wearing two Myo armbands, served as inputs to RCNN classification and regression models. Such models predicted movements (wrist flexion/extension and forearm pronation/supination), based on a multi-limb-position training routine. RCNN classifiers and RCNN regressors were compared to linear discriminant analysis (LDA) classifiers and support vector regression (SVR) regressors, respectively. Outcomes were examined to determine whether RCNN-based control strategies could yield accurate movement predictions, while using the fewest number of available Myo armband data streams.

RESULTS

An RCNN classifier (trained with forearm EMG data, and forearm and upper arm IMU data) predicted movements with 99.00% accuracy (versus the LDAs 97.67%). An RCNN regressor (trained with forearm EMG and IMU data) predicted movements with R2 values of 84.93% for wrist flexion/extension and 84.97% for forearm pronation/supination (versus the SVRs 77.26% and 60.73%, respectively). The control strategies that employed these models required fewer than all available data streams.

CONCLUSION

RCNN-based control strategies offer novel means of mitigating limb position challenges.

SIGNIFICANCE

This research furthers the development of improved position-aware myoelectric prosthesis control.