Using Reinforcement Learning to Estimate Human Joint Moments From Electromyography or Joint Kinematics: An Alternative Solution to Musculoskeletal-Based Biomechanics

Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Context-informed incremental learning improves both the performance and resilience of myoelectric control

Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.

Continuous Motion Intention Prediction Using sEMG for Upper-Limb Rehabilitation: A Systematic Review of Model-Based and Model-Free Approaches

Upper limb functional impairments persisting after stroke significantly affect patients' quality of life. Precise adjustment of robotic assistance levels based on patients' motion intentions using sEMG signals is crucial for active rehabilitation. This paper systematically reviews studies on continuous prediction of upper limb single joints and multi-joint combinations motion intention using Model-Based (MB) and Model-Free (MF) approaches over the past decade, based on 186 relevant studies screened from six major electronic databases. The findings indicate ongoing challenges in terms of subject composition, algorithm robustness and generalization, and algorithm feasibility for practical applications. Moreover, it suggests integrating the strengths of both MB and MF approaches to improve existing algorithms. Therefore, future research should further explore personalized MB-MF combination methods incorporating deep learning, attention mechanisms, muscle synergy features, motor unit features, and closed-loop feedback to achieve precise, real-time, and long-duration prediction of multi-joint complex movements, while further refining the transfer learning strategy for rapid algorithm deployment across days and subjects. Overall, this review summarizes the current research status, significant findings, and challenges, aiming to inspire future research on predicting upper limb motion intentions based on sEMG.

NSF DARE-Transforming modeling in neurorehabilitation: Four threads for catalyzing progress

We present an overview of the Conference on Transformative Opportunities for Modeling in Neurorehabilitation held in March 2023. It was supported by the Disability and Rehabilitation Engineering (DARE) program from the National Science Foundation's Engineering Biology and Health Cluster. The conference brought together experts and trainees from around the world to discuss critical questions, challenges, and opportunities at the intersection of computational modeling and neurorehabilitation to understand, optimize, and improve clinical translation of neurorehabilitation. We organized the conference around four key, relevant, and promising Focus Areas for modeling: Adaptation & Plasticity, Personalization, Human-Device Interactions, and Modeling 'In-the-Wild'. We identified four common threads across the Focus Areas that, if addressed, can catalyze progress in the short, medium, and long terms. These were: (i) the need to capture and curate appropriate and useful data necessary to develop, validate, and deploy useful computational models (ii) the need to create multi-scale models that span the personalization spectrum from individuals to populations, and from cellular to behavioral levels (iii) the need for algorithms that extract as much information from available data, while requiring as little data as possible from each client (iv) the insistence on leveraging readily available sensors and data systems to push model-driven treatments from the lab, and into the clinic, home, workplace, and community. The conference archive can be found at (dare2023.usc.edu). These topics are also extended by three perspective papers prepared by trainees and junior faculty, clinician researchers, and federal funding agency representatives who attended the conference.

A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Walking Kinematics Via an Artificial Neural Network

Lower-limb powered prostheses can provide users with volitional control of ambulation. To accomplish this goal, they require a sensing modality that reliably interprets user intention to move. Surface electromyography (EMG) has been previously proposed to measure muscle excitation and provide volitional control to upper- and lower-limb powered prosthesis users. Unfortunately, EMG suffers from a low signal to noise ratio and crosstalk between neighboring muscles, often limiting the performance of EMG-based controllers. Ultrasound has been shown to have better resolution and specificity than surface EMG. However, this technology has yet to be integrated into lower-limb prostheses. Here we show that A-mode ultrasound sensing can reliably predict the prosthesis walking kinematics of individuals with a transfemoral amputation. Ultrasound features from the residual limb of 9 transfemoral amputee subjects were recorded with A-mode ultrasound during walking with their passive prosthesis. The ultrasound features were mapped to joint kinematics through a regression neural network. Testing of the trained model against untrained kinematics from an altered walking speed show accurate predictions of knee position, knee velocity, ankle position, and ankle velocity, with a normalized RMSE of 9.0 ± 3.1%, 7.3 ± 1.6%, 8.3 ± 2.3%, and 10.0 ± 2.5% respectively. This ultrasound-based prediction suggests that A-mode ultrasound is a viable sensing technology for recognizing user intent. This study is the first necessary step towards implementation of volitional prosthesis controller based on A-mode ultrasound for individuals with transfemoral amputation.

A Feature-Encoded Physics-Informed Parameter Identification Neural Network for Musculoskeletal Systems

Identification of muscle-tendon force generation properties and muscle activities from physiological measurements, e.g., motion data and raw surface electromyography (sEMG), offers opportunities to construct a subject-specific musculoskeletal (MSK) digital twin system for health condition assessment and motion prediction. While machine learning approaches with capabilities in extracting complex features and patterns from a large amount of data have been applied to motion prediction given sEMG signals, the learned data-driven mapping is black-box and may not satisfy the underlying physics and has reduced generality. In this work, we propose a feature-encoded physics-informed parameter identification neural network (FEPI-PINN) for simultaneous prediction of motion and parameter identification of human MSK systems. In this approach, features of high-dimensional noisy sEMG signals are projected onto a low-dimensional noise-filtered embedding space for the enhancement of forwarding dynamics prediction. This FEPI-PINN model can be trained to relate sEMG signals to joint motion and simultaneously identify key MSK parameters. The numerical examples demonstrate that the proposed framework can effectively identify subject-specific muscle parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion predictions of elbow flexion-extension motion that are in good agreement with the measured joint motion data.

Myoelectric Control Systems for Upper Limb Wearable Robotic Exoskeletons and Exosuits-A Systematic Review

In recent years, myoelectric control systems have emerged for upper limb wearable robotic exoskeletons to provide movement assistance and/or to restore motor functions in people with motor disabilities and to augment human performance in able-bodied individuals. In myoelectric control, electromyographic (EMG) signals from muscles are utilized to implement control strategies in exoskeletons and exosuits, improving adaptability and human-robot interactions during various motion tasks. This paper reviews the state-of-the-art myoelectric control systems designed for upper-limb wearable robotic exoskeletons and exosuits, and highlights the key focus areas for future research directions. Here, different modalities of existing myoelectric control systems were described in detail, and their advantages and disadvantages were summarized. Furthermore, key design aspects (i.e., supported degrees of freedom, portability, and intended application scenario) and the type of experiments conducted to validate the efficacy of the proposed myoelectric controllers were also discussed. Finally, the challenges and limitations of current myoelectric control systems were analyzed, and future research directions were suggested.

Personalizing exoskeleton assistance while walking in the real world

Personalized exoskeleton assistance provides users with the largest improvements in walking speed1 and energy economy2-4 but requires lengthy tests under unnatural laboratory conditions. Here we show that exoskeleton optimization can be performed rapidly and under real-world conditions. We designed a portable ankle exoskeleton based on insights from tests with a versatile laboratory testbed. We developed a data-driven method for optimizing exoskeleton assistance outdoors using wearable sensors and found that it was equally effective as laboratory methods, but identified optimal parameters four times faster. We performed real-world optimization using data collected during many short bouts of walking at varying speeds. Assistance optimized during one hour of naturalistic walking in a public setting increased self-selected speed by 9 ± 4% and reduced the energy used to travel a given distance by 17 ± 5% compared with normal shoes. This assistance reduced metabolic energy consumption by 23 ± 8% when participants walked on a treadmill at a standard speed of 1.5 m s-1. Human movements encode information that can be used to personalize assistive devices and enhance performance.

Foot contact forces can be used to personalize a wearable robot during human walking

Individuals with below-knee amputation (BKA) experience increased physical effort when walking, and the use of a robotic ankle-foot prosthesis (AFP) can reduce such effort. The walking effort could be further reduced if the robot is personalized to the wearer using human-in-the-loop (HIL) optimization of wearable robot parameters. The conventional physiological measurement, however, requires a long estimation time, hampering real-time optimization due to the limited experimental time budget. This study hypothesized that a function of foot contact force, the symmetric foot force-time integral (FFTI), could be used as a cost function for HIL optimization to rapidly estimate the physical effort of walking. We found that the new cost function presents a reasonable correlation with measured metabolic cost. When we employed the new cost function in HIL ankle-foot prosthesis stiffness parameter optimization, 8 individuals with simulated amputation reduced their metabolic cost of walking, greater than 15% (p < 0.02), compared to the weight-based and control-off conditions. The symmetry cost using the FFTI percentage was lower for the optimal condition, compared to all other conditions (p < 0.05). This study suggests that foot force-time integral symmetry using foot pressure sensors can be used as a cost function when optimizing a wearable robot parameter.

Comparing Reinforcement Learning Agents and Supervised Learning Neural Networks for EMG-Based Decoding of Continuous Movements

Recent work on electromyography (EMG)-based decoding of continuous joint kinematics has included model-based approaches, such as musculoskeletal modeling, as well as model-free approaches such as supervised learning neural networks (SLNN). This study aimed to present a new kinematics decoding framework based on reinforcement learning (RL), which combines machine learning and model-based approaches together. We compared the performance and robustness of our new method with those of the SLNN approach. EMG and kinematic data were collected from 5 able-bodied subjects while they performed flexion and extension of the metacarpophalangeal (MCP) and wrist joints simultaneously at both a slow and fast tempo. The data were used to train an RL agent and a SLNN for each of the 2 tempos. All the trained agents and SLNNs were tested with both fast and slow kinematic data. Pearson's correlation coefficient (r) and normalized root mean square error (NRMSE) between measured and estimated joint angles were used to determine performance. Our results suggest that the RL-based kinematics decoder is more robust to changes in movement speeds between training and testing data and has better performance than the SLNN.