Activities of daily living with bionic arm improved by combination training and latching filter in prosthesis control comparison

Validity and Impact of Methods for Collecting Training Data for Myoelectric Prosthetic Control Algorithms

Intuitive regression control of prostheses relies on training algorithms to correlate biological recordings to motor intent. The quality of the training dataset is critical to run-time regression performance, but accurately labeling intended hand kinematics after hand amputation is challenging. In this study, we quantified the accuracy and precision of labeling hand kinematics using two common training paradigms: 1) mimic training, where participants mimic predetermined motions of a prosthesis, and 2) mirror training, where participants mirror their contralateral intact hand during synchronized bilateral movements. We first explored this question in healthy non-amputee individuals where the ground-truth kinematics could be readily determined using motion capture. Kinematic data showed that mimic training fails to account for biomechanical coupling and temporal changes in hand posture. Additionally, mirror training exhibited significantly higher accuracy and precision in labeling hand kinematics. These findings suggest that the mirror training approach generates a more faithful, albeit more complex, dataset. Accordingly, mirror training resulted in significantly better offline regression performance when using a large amount of training data and a non-linear neural network. Next, we explored these different training paradigms online, with a cohort of unilateral transradial amputees actively controlling a prosthesis in real-time to complete a functional task. Overall, we found that mirror training resulted in significantly faster task completion speeds and similar subjective workload. These results demonstrate that mirror training can potentially provide more dexterous control through the utilization of task-specific, user-selected training data. Consequently, these findings serve as a valuable guide for the next generation of myoelectric and neuroprostheses leveraging machine learning to provide more dexterous and intuitive control.

Correcting Temporal Inaccuracies in Labeled Training Data for Electromyographic Control Algorithms

Electromyographic (EMG) control relies on supervised-learning algorithms that correlate EMG to motor intent. The quality of the training dataset is critical to the runtime performance of the algorithm, but labeling motor intent is imprecise and imperfect. Traditional EMG training data is collected while participants mimic predetermined movements of a virtual hand with their own hand. This assumes participants are perfectly synchronized with the predetermined movements, which is unlikely due to reaction time and signal-processing delays. Prior work has used cross-correlation to globally shift and re-align kinematic data and EMG. Here, we quantify the impact of this global re-alignment on both classification algorithms and regression algorithms with and without a human in the loop. We also introduce a novel trial-by-trial re-alignment method to re-align EMG with kinematics on a per-movement basis. We show that EMG and kinematic data are inherently misaligned, and that reaction time is inconsistent throughout data collection. Both global and trial-by-trial re-alignment significantly improved offline performance for classification and regression. Our trial-by-trial re-alignment further improved offline classification performance relative to global realignment. However, online performance, with a human actively in the loop, was no different with or without re-alignment. This work highlights inaccuracies in labeled EMG data and has broad implications for EMG-control applications.

Automated Quantifiable Assessments of Sensorimotor Function Using an Instrumented Fragile Object

Accurate assessment of hand dexterity plays a critical role in informing rehabilitation and care of upper-limb hemiparetic stroke patients. Common upper-limb assessments, such as the Box and Blocks Test and Nine Hole Peg Test, primarily evaluate gross motor function in terms of speed. These assessments neglect an individual's ability to finely regulate grip force, which is critical in activities of daily living, such as manipulating fragile objects. Here we present the Electronic Grip Gauge (EGG), an instrumented fragile object that assesses both gross and fine motor function. Embedded with a load cell, accelerometer, and Hall-effect sensor, the EGG measures grip force, acceleration, and relative position (via magnetic fields) in real time. The EGG can emit an audible "break" sound when the applied grip force exceeds a threshold. The number of breaks, transfer duration, and applied forces are automatically logged in real-time. Using the EGG, we evaluated sensorimotor function in implicit grasping and gentle grasping for the non-paretic and paretic hands of 3 hemiparetic stroke patients. For all participants, the paretic hand took longer to transfer the EGG during implicit grasping. For 2 of 3 participants, grip forces were significantly greater for the paretic hand during gentle grasping. Differences in implicit grasping forces were unique to each participant. This work constitutes an important step towards more widespread and quantitative measures of sensorimotor function, which may ultimately lead to improved personalized rehabilitation and better patient outcomes.

Advances in artificial muscles: A brief literature and patent review

Background: Artificial muscles are an active research area now. Methods: A bibliometric analysis was performed to evaluate the development of artificial muscles based on research papers and patents. A detailed overview of artificial muscles' scientific and technological innovation was presented from aspects of productive countries/regions, institutions, journals, researchers, highly cited papers, and emerging topics. Results: 1,743 papers and 1,925 patents were identified after retrieval in Science Citation Index-Expanded (SCI-E) and Derwent Innovations Index (DII). The results show that China, the United States, and Japan are leading in the scientific and technological innovation of artificial muscles. The University of Wollongong has the most publications and Spinks is the most productive author in artificial muscle research. Smart Materials and Structures is the journal most productive in this field. Materials science, mechanical and automation, and robotics are the three fields related to artificial muscles most. Types of artificial muscles like pneumatic artificial muscles (PAMs) and dielectric elastomer actuator (DEA) are maturing. Shape memory alloy (SMA), carbon nanotubes (CNTs), graphene, and other novel materials have shown promising applications in this field. Conclusion: Along with the development of new materials and processes, researchers are paying more attention to the performance improvement and cost reduction of artificial muscles.

A Biorealistic Computational Model Unfolds Human-Like Compliant Properties for Control of Hand Prosthesis

Objective: Human neuromuscular reflex control provides a biological model for a compliant hand prosthesis. Here we present a computational approach to understanding the emerging human-like compliance, force and position control, and stiffness adaptation in a prosthetic hand with a replica of human neuromuscular reflex. Methods: A virtual twin of prosthetic hand was constructed in the MuJoCo environment with a tendon-driven anthropomorphic hand structure. Biorealistic mathematic models of muscle, spindle, spiking-neurons and monosynaptic reflex were implemented in neuromorphic chips to drive the virtual hand for real-time control. Results: Simulation showed that the virtual hand acquired human-like ability to control fingertip position, force and stiffness for grasp, as well as the capacity to interact with soft objects by adaptively adjusting hand stiffness. Conclusion: The biorealistic neuromorphic reflex model restores human-like neuromuscular properties for hand prosthesis to interact with soft objects.

A Multi-User Transradial Functional-Test Socket for Validation of New Myoelectric Prosthetic Control Strategies

The validation of myoelectric prosthetic control strategies for individuals experiencing upper-limb loss is hindered by the time and cost affiliated with traditional custom-fabricated sockets. Consequently, researchers often rely upon virtual reality or robotic arms to validate novel control strategies, which limits end-user involvement. Prosthetists fabricate diagnostic check sockets to assess and refine socket fit, but these clinical techniques are not readily available to researchers and are not intended to assess functionality for control strategies. Here we present a multi-user, low-cost, transradial, functional-test socket for short-term research use that can be custom-fit and donned rapidly, used in conjunction with various electromyography configurations, and adapted for use with various residual limbs and terminal devices. In this study, participants with upper-limb amputation completed functional tasks in physical and virtual environments both with and without the socket, and they reported on their perceived comfort level over time. The functional-test socket was fabricated prior to participants' arrival, iteratively fitted by the researchers within 10 mins, and donned in under 1 min (excluding electrode placement, which will vary for different use cases). It accommodated multiple individuals and terminal devices and had a total cost of materials under $10 USD. Across all participants, the socket did not significantly impede functional task performance or reduce the electromyography signal-to-noise ratio. The socket was rated as comfortable enough for at least 2 h of use, though it was expectedly perceived as less comfortable than a clinically-prescribed daily-use socket. The development of this multi-user, transradial, functional-test socket constitutes an important step toward increased end-user participation in advanced myoelectric prosthetic research. The socket design has been open-sourced and is available for other researchers.

Robust Torque Predictions From Electromyography Across Multiple Levels of Active Exoskeleton Assistance Despite Non-linear Reorganization of Locomotor Output

Robotic exoskeletons can assist humans with walking by providing supplemental torque in proportion to the user's joint torque. Electromyographic (EMG) control algorithms can estimate a user's joint torque directly using real-time EMG recordings from the muscles that generate the torque. However, EMG signals change as a result of supplemental torque from an exoskeleton, resulting in unreliable estimates of the user's joint torque during active exoskeleton assistance. Here, we present an EMG control framework for robotic exoskeletons that provides consistent joint torque predictions across varying levels of assistance. Experiments with three healthy human participants showed that using diverse training data (from different levels of assistance) enables robust torque predictions, and that a convolutional neural network (CNN), but not a Kalman filter (KF), can capture the non-linear transformations in EMG due to exoskeleton assistance. With diverse training, the CNN could reliably predict joint torque from EMG during zero, low, medium, and high levels of exoskeleton assistance [root mean squared error (RMSE) below 0.096 N-m/kg]. In contrast, without diverse training, RMSE of the CNN ranged from 0.106 to 0.144 N-m/kg. RMSE of the KF ranged from 0.137 to 0.182 N-m/kg without diverse training, and did not improve with diverse training. When participant time is limited, training data should emphasize the highest levels of assistance first and utilize at least 35 full gait cycles for the CNN. The results presented here constitute an important step toward adaptive and robust human augmentation via robotic exoskeletons. This work also highlights the non-linear reorganization of locomotor output when using assistive exoskeletons; significant reductions in EMG activity were observed for the soleus and gastrocnemius, and a significant increase in EMG activity was observed for the erector spinae. Control algorithms that can accommodate spatiotemporal changes in muscle activity have broad implications for exoskeleton-based assistance and rehabilitation following neuromuscular injury.

A Recurrent Neural Network Provides Stable Across-Day Prosthetic Control for a Human Amputee with Implanted Intramuscular Electromyographic Recording Leads

Upper-limb prosthetic control is often challenging and non-intuitive, leading to up to 50% of prostheses users abandoning their prostheses. Convolutional neural networks (CNN) and recurrent long short-term memory (LSTM) networks have shown promise in extracting high-degree-of-freedom motor intent from myoelectric signals, thereby providing more intuitive and dexterous prosthetic control. An important next consideration for these algorithms is if performance remains stable over multiple days. Here we introduce a new LSTM network and compare its performance to previously established state-of-the-art algorithms-a CNN and a modified Kalman filter (MKF)-in offline analyses using 76 days of intramuscular recordings from one amputee participant collected over 425 calendar days. Specifically, we assessed the robustness of each algorithm over time by training on data from the first (one, five, ten, 30, or 60) days and then testing on myoelectric signals on the last 16 days. Results indicate that training on additional datasets from prior days generally decreases the Root Mean Squared Error (RMSE) of intended and unintended movements for all algorithms. Across all algorithms trained with 60 days of data, the lowest RMSE for unintended movements was achieved with the LSTM. The LSTM also showed less across-day variance in RMSE of unintended movements relative to the other algorithms. Altogether this work suggests that the LSTM algorithm introduced here can provide more intuitive and dexterous control for prosthetic users, and that training on multiple days of data improves overall performance on subsequent days, at least for offline analyses.

Research on Joint-Angle Prediction Based on Artificial Neural Network for Above-Knee Amputees

In the current study, our research group proposed an asymmetric lower extremity exoskeleton to enable above-knee amputees to walk with a load. Due to the absence of shank and foot, the knee and ankle joint at the amputation side of the exoskeleton lack tracking targets, so it is difficult to realize the function of assisted walking when going up and downstairs. Currently, the use of lower-limb electromyography to predict the angles of lower limb joints has achieved remarkable results. However, the prediction effect was poor when only using electromyography from the thigh. Therefore, this paper introduces hip-angle and plantar pressure signals for improving prediction effect and puts forward a joint prediction method of knee- and ankle-joint angles by electromyography of the thigh, hip-joint angle, and plantar pressure signals. The generalized regression neural network optimized by the golden section method is used to predict the joint angles. Finally, the parameters (the maximum error, the Root-Mean-Square error (RMSE), and correlation coefficient (γ)) were calculated to verify the feasibility of the prediction method.