Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

DistaNet: grasp-specific distance biofeedback promotes the retention of myoelectric skills

Objective.An active myoelectric interface responds to the user's muscle signals to enable movements. Machine learning can decode user intentions from myoelectric signals. However, machine learning-based interface control lacks continuous, intuitive feedback about task performance, needed to facilitate the acquisition and retention of myoelectric control skills.Approach.We propose DistaNet as a neural network-based framework that extracts smooth, continuous, and low-dimensional signatures of the hand grasps from multi-channel myoelectric signals and provides grasp-specific biofeedback to the users.Main results.Experimental results show its effectiveness in decoding user gestures and providing biofeedback, helping users retain the acquired motor skills.Significance.We demonstrates myoelectric skill retention in a pattern recognition setting for the first time.

Biomimetic versus arbitrary motor control strategies for bionic hand skill learning

A long-standing engineering ambition has been to design anthropomorphic bionic limbs: devices that look like and are controlled in the same way as the biological body (biomimetic). The untested assumption is that biomimetic motor control enhances device embodiment, learning, generalization and automaticity. To test this, we compared biomimetic and non-biomimetic control strategies for non-disabled participants when learning to control a wearable myoelectric bionic hand operated by an eight-channel electromyography pattern-recognition system. We compared motor learning across days and behavioural tasks for two training groups: biomimetic (mimicking the desired bionic hand gesture with biological hand) and arbitrary control (mapping an unrelated biological hand gesture with the desired bionic gesture). For both trained groups, training improved bionic limb control, reduced cognitive reliance and increased embodiment over the bionic hand. Biomimetic users had more intuitive and faster control early in training. Arbitrary users matched biomimetic performance later in training. Furthermore, arbitrary users showed increased generalization to a new control strategy. Collectively, our findings suggest that biomimetic and arbitrary control strategies provide different benefits. The optimal strategy is probably not strictly biomimetic, but rather a flexible strategy within the biomimetic-to-arbitrary spectrum, depending on the user, available training opportunities and user requirements.

Progressive unsupervised control of myoelectric upper limbs

Objective.Unsupervised myocontrol methods aim to create control models for myoelectric prostheses while avoiding the complications of acquiring reliable, regular, and sufficient labeled training data. A limitation of current unsupervised methods is that they fix the number of controlled prosthetic functions a priori, thus requiring an initial assessment of the user's motor skills and neglecting the development of novel motor skills over time.Approach.We developed a progressive unsupervised myocontrol (PUM) paradigm in which the user and the control model coadaptively identify distinct muscle synergies, which are then used to control arbitrarily associated myocontrol functions, each corresponding to a hand or wrist movement. The interaction starts with learning a single function and the user may request additional functions after mastering the available ones, which aligns the evolution of their motor skills with an increment in system complexity. We conducted a multi-session user study to evaluate PUM and compare it against a state-of-the-art non-progressive unsupervised alternative. Two participants with congenital upper-limb differences tested PUM, while ten non-disabled control participants tested either PUM or the non-progressive baseline. All participants engaged in myoelectric control of a virtual hand and wrist.Main results.PUM enabled autonomous learning of three myocontrol functions for participants with limb differences, and of all four available functions for non-disabled subjects, using both existing or newly identified muscle synergies. Participants with limb differences achieved similar success rates to non-disabled ones on myocontrol tests, but faced greater difficulties in internalizing new motor skills and exhibited slightly inferior movement quality. The performance was comparable with either PUM or the non-progressive baseline for the group of non-disabled participants.Significance.The PUM paradigm enables users to autonomously learn to operate the myocontrol system, adapts to the users' varied preexisting motor skills, and supports the further development of those skills throughout practice.

Learning to operate a high-dimensional hand via a low-dimensional controller

Dimensionality reduction techniques have proven useful in simplifying complex hand kinematics. They may allow for a low-dimensional kinematic or myoelectric interface to be used to control a high-dimensional hand. Controlling a high-dimensional hand, however, is difficult to learn since the relationship between the low-dimensional controls and the high-dimensional system can be hard to perceive. In this manuscript, we explore how training practices that make this relationship more explicit can aid learning. We outline three studies that explore different factors which affect learning of an autoencoder-based controller, in which a user is able to operate a high-dimensional virtual hand via a low-dimensional control space. We compare computer mouse and myoelectric control as one factor contributing to learning difficulty. We also compare training paradigms in which the dimensionality of the training task matched or did not match the true dimensionality of the low-dimensional controller (both 2D). The training paradigms were a) a full-dimensional task, in which the user was unaware of the underlying controller dimensionality, b) an implicit 2D training, which allowed the user to practice on a simple 2D reaching task before attempting the full-dimensional one, without establishing an explicit connection between the two, and c) an explicit 2D training, during which the user was able to observe the relationship between their 2D movements and the higher-dimensional hand. We found that operating a myoelectric interface did not pose a big challenge to learning the low-dimensional controller and was not the main reason for the poor performance. Implicit 2D training was found to be as good, but not better, as training directly on the high-dimensional hand. What truly aided the user's ability to learn the controller was the 2D training that established an explicit connection between the low-dimensional control space and the high-dimensional hand movements.

Virtual regression-based myoelectric hand-wrist prosthesis control and electrode site selection using no force feedback

Most transradial prosthesis users with conventional "Sequential" myoelectric control have two electrode sites which control one degree of freedom (DoF) at a time. Rapid EMG co-activation toggles control between DoFs (e.g., hand and wrist), providing limited function. We implemented a regression-based EMG control method which achieved simultaneous and proportional control of two DoFs in a virtual task. We automated electrode site selection using a short-duration (90 s) calibration period, without force feedback. Backward stepwise selection located the best electrodes for either six or 12 electrodes (selected from a pool of 16). We additionally studied two, 2-DoF controllers: "Intuitive" control (hand open-close and wrist pronation-supination controlled virtual target size and rotation, respectively) and "Mapping" control (wrist flexion-extension and ulnar-radial deviation controlled virtual target left-right and up-down movement, respectively). In practice, a Mapping controller would be mapped to control prosthesis hand open-close and wrist pronation-supination. Eleven able-bodied subjects and 4 limb-absent subjects completed virtual target matching tasks (fixed target moves to a new location after being "matched," and subject immediately pursues) and fixed (static) target tasks. For all subjects, both 2-DoF controllers with 6 optimally-sited electrodes had statistically better target matching performance than Sequential control in number of matches (average of 4-7 vs. 2 matches, p< 0.001) and throughput (average of 0.75-1.25 vs. 0.4 bits/s, p< 0.001), but not overshoot rate and path efficiency. There were no statistical differences between 6 and 12 optimally-sited electrodes for both 2-DoF controllers. These results support the feasibility of 2-DoF simultaneous, proportional myoelectric control.

Should bionic limb control mimic the human body? Impact of control strategy on bionic hand skill learning

A longstanding engineering ambition has been to design anthropomorphic bionic limbs: devices that look like and are controlled in the same way as the biological body (biomimetic). The untested assumption is that biomimetic motor control enhances device embodiment, learning, generalization, and automaticity. To test this, we compared biomimetic and non-biomimetic control strategies for able-bodied participants when learning to operate a wearable myoelectric bionic hand. We compared motor learning across days and behavioural tasks for two training groups: Biomimetic (mimicking the desired bionic hand gesture with biological hand) and Arbitrary control (mapping an unrelated biological hand gesture with the desired bionic gesture). For both trained groups, training improved bionic limb control, reduced cognitive reliance, and increased embodiment over the bionic hand. Biomimetic users had more intuitive and faster control early in training. Arbitrary users matched biomimetic performance later in training. Further, arbitrary users showed increased generalization to a novel control strategy. Collectively, our findings suggest that biomimetic and arbitrary control strategies provide different benefits. The optimal strategy is likely not strictly biomimetic, but rather a flexible strategy within the biomimetic to arbitrary spectrum, depending on the user, available training opportunities and user requirements.

Recalibration of myoelectric control with active learning

Introduction

Improving the robustness of myoelectric control to work over many months without the need for recalibration could reduce prosthesis abandonment. Current approaches rely on post-hoc error detection to verify the certainty of a decoder's prediction using predefined threshold value. Since the decoder is fixed, the performance decline over time is inevitable. Other approaches such as supervised recalibration and unsupervised self-recalibration entail limitations in scaling up and computational resources. The objective of this paper is to study active learning as a scalable, human-in-the-loop framework, to improve the robustness of myoelectric control.

Method

Active learning and linear discriminate analysis methods were used to create an iterative learning process, to modify decision boundaries based on changes in the data. We simulated a real-time scenario. We exploited least confidence, smallest margin and entropy reduction sampling strategies in single and batch-mode sample selection. Optimal batch-mode sampling was considered using ranked batch-mode active learning.

Results

With only 3.2 min of data carefully selected by the active learner, the decoder outperforms random sampling by 4-5 and ~2% for able-bodied and people with limb difference, respectively. We observed active learning strategies to systematically and significantly enhance the decoders adaptation while optimizing the amount of training data on a class-specific basis. Smallest margin and least confidence uncertainty were shown to be the most supreme.

Discussion

We introduce for the first time active learning framework for long term adaptation in myoelectric control. This study simulates closed-loop environment in an offline manner and proposes a pipeline for future real-time deployment.

Increasing Voluntary Myoelectric Training Time Through Game Design

In virtual prosthetic training research, serious games have been investigated for over 30 years. However, few game design elements are used and assessed for their effect on the voluntary adherence and repetition of the performed task. We compared two game-based versions of an established myoelectric-controlled virtual prosthetic training task with an interface without game elements of the same task [for video, see (Garske, 2022)]. Twelve limb-intact participants were sorted into three groups of comparable ability and asked to perform the task as long as they were motivated. Following the task, they completed a questionnaire regarding their motivation and engagement in the task. The investigation established that participants in the game-based groups performed the task significantly longer when more game design elements were implemented in the task (medians of 6 vs. 9.5 vs. 14 blocks for groups with increasing number of different game design elements). The participants in the game-based versions were also more likely to end the task out of fatigue than for reasons of boredom or frustration, which was verified by a fatigue analysis of the myoelectric signal. We demonstrated that the utilization of game design methodically in virtual myoelectric training tasks can support adherence and duration of a virtual training, in the short-term. Whether such short-term enhanced engagement would lead to long-term adherence remains an open question.

Internet of Things for beyond-the-laboratory prosthetics research

Research on upper-limb prostheses is typically laboratory-based. Evidence indicates that research has not yet led to prostheses that meet user needs. Inefficient communication loops between users, clinicians and manufacturers limit the amount of quantitative and qualitative data that researchers can use in refining their innovations. This paper offers a first demonstration of an alternative paradigm by which remote, beyond-the-laboratory prosthesis research according to user needs is feasible. Specifically, the proposed Internet of Things setting allows remote data collection, real-time visualization and prosthesis reprogramming through Wi-Fi and a commercial cloud portal. Via a dashboard, the user can adjust the configuration of the device and append contextual information to the prosthetic data. We evaluated this demonstrator in real-time experiments with three able-bodied participants. Results promise the potential of contextual data collection and system update through the internet, which may provide real-life data for algorithm training and reduce the complexity of send-home trials. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Spatio-temporal feature extraction in sensory electroneurographic signals

The recording and analysis of peripheral neural signal can provide insight for various prosthetic and bioelectronics medicine applications. However, there are few studies that investigate how informative features can be extracted from population activity electroneurographic (ENG) signals. In this study, five feature extraction frameworks were implemented on sensory ENG datasets and their classification performance was compared. The datasets were collected in acute rat experiments where multi-channel nerve cuffs recorded from the sciatic nerve in response to proprioceptive stimulation of the hindlimb. A novel feature extraction framework, which incorporates spatio-temporal focus and dynamic time warping, achieved classification accuracies above 90% while keeping a low computational cost. This framework outperformed the remaining frameworks tested in this study and has improved the discrimination accuracy of the sensory signals. Thus, this study has extended the tools available to extract features from sensory population activity ENG signals. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

Myoelectric Control Performance of Two Degree of Freedom Hand-Wrist Prosthesis by Able-Bodied and Limb-Absent Subjects

Recent research has advanced two degree-of-freedom (DoF), simultaneous, independent and proportional control of hand-wrist prostheses using surface electromyogram signals from remnant muscles as the control input. We evaluated two such regression-based controllers, along with conventional, sequential two-site control with co-contraction mode switching (SeqCon), in box-block, refined-clothespin and door-knob tasks, on 10 able-bodied and 4 limb-absent subjects. Subjects operated a commercial hand and wrist using a socket bypass harness. One 2-DoF controller (DirCon) related the intuitive hand actions of open-close and pronation-supination to the associated prosthesis hand-wrist actions, respectively. The other (MapCon) mapped myoelectrically more distinct, but less intuitive, actions of wrist flexion-extension and ulnar-radial deviation. Each 2-DoF controller was calibrated from separate 90 s calibration contractions. SeqCon performed better statistically than MapCon in the predominantly 1-DoF box-block task (>20 blocks/minute vs. 8-18 blocks/minute, on average). In this task, SeqCon likely benefited from an ability to easily focus on 1-DoF and not inadvertently trigger co-contraction for mode switching. The remaining two tasks require 2-DoFs, and both 2-DoF controllers each performed better (factor of 2-4) than SeqCon. We also compared the use of 12 vs. 6 optimally-selected EMG electrodes as inputs, finding no statistical difference. Overall, we provide further evidence of the benefits of regression-based EMG prosthesis control of 2-DoFs in the hand-wrist.

Spatio-temporal warping for myoelectric control: an offline, feasibility study

Objective.The efficacy of an adopted feature extraction method directly affects the classification of the electromyographic (EMG) signals in myoelectric control applications. Most methods attempt to extract the dynamics of the multi-channel EMG signals in the time domain and on a channel-by-channel, or at best pairs of channels, basis. However, considering multi-channel information to build a similarity matrix has not been taken into account.Approach.Combining methods of long and short-term memory (LSTM) and dynamic temporal warping, we developed a new feature, called spatio-temporal warping (STW), for myoelectric signals. This method captures the spatio-temporal relationships of multi-channels EMG signals.Main results. Across four online databases, we show that in terms of average classification error and standard deviation values, the STW feature outperforms traditional features by 5%-17%. In comparison to the more recent deep learning models, e.g. convolutional neural networks (CNNs), STW outperformed by 5%-18%. Also, STW showed enhanced performance when compared to the CNN + LSTM model by 2%-14%. All differences were statistically significant with a large effect size.Significance.This feasibility study provides evidence supporting the hypothesis that the STW feature of the EMG signals can enhance the classification accuracy in an explainable way when compared to recent deep learning methods. Future work includes real-time implementation of the method and testing for prosthesis control.

Remote creation of clinical-standard myoelectric trans-radial bypass sockets during COVID-19

To enable the progression of research during the COVID-19 lockdown, a novel remote method of creating clinical standard trans-radial bypass sockets was devised as a collaboration between an engineering team and a clinical research group. The engineering team recruited two able-bodied participants, marked areas of interest on the participant's limb and captured limb geometry and electrode sites with a high definition optical scanner. The resulting 3D scan was modified to make electrode sites and areas of interest recessed and tactile. Models were 3D printed to scale and posted to the clinical team to manufacture the sockets. A modified lamination process was used, comprising plaster casting and rectifiying the model by hand. The recessed areas of the 3D printed model were used to guide the process. The bypass sockets were returned to the engineering team for testing. A simple electromyography (EMG) tracking task was performed using clinical electrodes to validate the skin-electrode contact and alignment. This paper demonstrates a validated method for remotely creating transradial bypass sockets. There is potential to extrapolate this method to standard socket fittings with further research.

Long-Term Myoelectric Training with Delayed Feedback in the Home Environment

Myoelectric prosthesis users typically do not receive immediate feedback from their device. They must be able to consistently produce distinct muscle activations in the absence of augmented feedback. In previous experiments, abstract decoding has provided real-time visual feedback for closed loop control. It is unclear if the performance in those experiments was due to short-term adaptation or motor learning. To test if similar performance could be reached without short-term adaptation, we trained participants with a delayed feedback paradigm. Feedback was delayed until after the ~1.5 s trial was completed. Three participants trained for five days in their home environments, completing a cumulative total of 4920 trials. Participants became highly accurate while receiving no real-time feedback of their control input. They were also able to retain performance gains across days. This strongly suggests that abstract decoding with delayed feedback facilitates motor learning, enabling four class control without immediate feedback.

A Network-Enabled Myoelectric Platform for Prototyping Research Outside of the Lab

We present a network-enabled myoelectric platform for performing research outside of the laboratory environment. A low-cost, flexible, modular design based on common Internet of Things connectivity technology allows home-based research to be piloted. An outline of the platform is presented followed by technical results obtained from ten days of home-based tests with three participants. Results show the system enabled collection of close to 12,000 trials during around 28 cumulative hours of use. Home-based testing of multiple participants in parallel offers efficiency gains and provides a intuitive route toward long-term testing of upper-limb prosthetic devices in more naturalistic settings.Clinical relevance- In-home myoelectric training reduces clinician time. Network-enabled systems with back-end dashboards allow clinicians to monitor patients myoelectric ability over time and will provide a new way of accessing information about how upper-limb prosthetics are commonly used.

A more human prosthetic hand

Co-creation leads the way for bioinspired prosthetics with improved design and performance.

Arduino-Based Myoelectric Control: Towards Longitudinal Study of Prosthesis Use

Understanding how upper-limb prostheses are used in daily life helps to improve the design and robustness of prosthesis control algorithms and prosthetic components. However, only a very small fraction of published research includes prosthesis use in community settings. The cost, limited battery life, and poor generalisation may be the main reasons limiting the implementation of home-based applications. In this work, we introduce the design of a cost-effective Arduino-based myoelectric control system with wearable electromyogram (EMG) sensors. The design considerations focused on home studies, so the robustness, user-friendly control adjustments, and user supports were the main concerns. Three control algorithms, namely, direct control, abstract control, and linear discriminant analysis (LDA) classification, were implemented in the system. In this paper, we will share our design principles and report the robustness of the system in continuous operation in the laboratory. In addition, we will show a first real-time implementation of the abstract decoder for prosthesis control with an able-bodied participant.

SenseBack - An Implantable System for Bidirectional Neural Interfacing

Chronic in-vivo neurophysiology experiments require highly miniaturized, remotely powered multi-channel neural interfaces which are currently lacking in power or flexibility post implantation. To resolve this problem we present the SenseBack system, a post-implantation reprogrammable wireless 32-channel bidirectional neural interfacing device that can enable chronic peripheral electrophysiology experiments in freely behaving small animals. The large number of channels for a peripheral neural interface, coupled with fully implantable hardware and complete software flexibility enable complex in-vivo studies where the system can adapt to evolving study needs as they arise. In complementary \textit{ex-vivo} and \textit{in-vivo} preparations, we demonstrate that this system can record neural signals and perform high-voltage, bipolar stimulation on any channel. In addition, we demonstrate transcutaneous power delivery and Bluetooth 5 data communication with a PC. The SenseBack system is capable of stimulation on any channel with 20 V of compliance and up to 315 A of current, and highly configurable recording with per-channel adjustable gain and filtering with 8 sets of 10-bit ADCs to sample data at 20 kHz for each channel. To our knowledge this is the first such implantable research platform offering this level of performance and flexibility post-implantation (including complete reprogramming even after encapsulation) for small animal electrophysiology. Here we present initial acute trials, demonstrations and progress towards a system that we expect to enable a wide range of electrophysiology experiments in freely behaving animals.