Deep Multi-scale Fusion of Convolutional Neural Networks for EMG-based Movement Estimation

Influence of spatio-temporal filtering on hand kinematics estimation from high-density EMG signals. J Neural Eng 2024;21:026014. [PMID: 38525843 DOI: 10.1088/1741-2552/ad3498] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Objective.Surface electromyography (sEMG) is a non-invasive technique that records the electrical signals generated by muscles through electrodes placed on the skin. sEMG is the state-of-the-art method used to control active upper limb prostheses because of the association between its amplitude and the neural drive sent from the spinal cord to muscles. However, accurately estimating the kinematics of a freely moving human hand using sEMG from extrinsic hand muscles remains a challenge. Deep learning has been recently successfully applied to this problem by mapping raw sEMG signals into kinematics. Nonetheless, the optimal number of EMG signals and the type of pre-processing that would maximize performance have not been investigated yet.Approach.Here, we analyze the impact of these factors on the accuracy in kinematics estimates. For this purpose, we processed monopolar sEMG signals that were originally recorded from 320 electrodes over the forearm muscles of 13 subjects. We used a previously published deep learning method that can map the kinematics of the human hand with real-time resolution.Main results.While myocontrol algorithms essentially use the temporal envelope of the EMG signal as the only EMG feature, we show that our approach requires the full bandwidth of the signal in the temporal domain for accurate estimates. Spatial filtering however, had a smaller impact and low-order spatial filters may be suitable. Moreover, reducing the number of channels by ablation resulted in large performance losses. The highest accuracy was reached with the highest number of available sensors (n = 320). Importantly and unexpected, our results suggest that increasing the number of channels above those used in this study may further enhance the accuracy in predicting the kinematics of the human hand.Significance.We conclude that full bandwidth high-density EMG systems of hundreds of electrodes are needed for accurate kinematic estimates of the human hand.

Generalizing Upper Limb Force Modeling With Transfer Learning: A Multimodal Approach Using EMG and IMU for New Users and Conditions

In the field of EMG-based force modeling, the ability to generalize models across individuals could play a significant role in its adoption across a range of applications, including assistive devices, robotic and rehabilitation devices. However, current studies have predominately focused on intra-subject modeling, largely neglecting the burden of end-user data acquisition. In this work, we propose the use of transfer learning (TL) to generalize force modeling to a new user by first establishing a baseline model trained using other users' data, and then adapting to the end-user using a small amount of new data (only 10% , 20% , and 40% of the new user data). Using a deep multimodal convolutional neural network, consisting of two CNN models, one with high-density (HD) EMG and one with motion data recorded by an Inertial Measurement Unit (IMU), our proposed TL technique significantly improved force modeling compared to leave-one-subject-out (LOSO) and even intra-subject scenarios. The TL approach increased the average R squared values of the force modeling task by 60.81%, 190.53%, and 199.79% compared to the LOSO case, and by 13.4%, 36.88%, and 45.51% compared to the intra-subject case for isotonic, isokinetic and dynamic conditions, respectively. These results show that it is possible to adapt to a new user with minimal data while improving performance significantly compared to the intra-subject scenario. We also show that TL can be used to generalize on a new experimental condition for a new user.

A Feature-Encoded Physics-Informed Parameter Identification Neural Network for Musculoskeletal Systems

Identification of muscle-tendon force generation properties and muscle activities from physiological measurements, e.g., motion data and raw surface electromyography (sEMG), offers opportunities to construct a subject-specific musculoskeletal (MSK) digital twin system for health condition assessment and motion prediction. While machine learning approaches with capabilities in extracting complex features and patterns from a large amount of data have been applied to motion prediction given sEMG signals, the learned data-driven mapping is black-box and may not satisfy the underlying physics and has reduced generality. In this work, we propose a feature-encoded physics-informed parameter identification neural network (FEPI-PINN) for simultaneous prediction of motion and parameter identification of human MSK systems. In this approach, features of high-dimensional noisy sEMG signals are projected onto a low-dimensional noise-filtered embedding space for the enhancement of forwarding dynamics prediction. This FEPI-PINN model can be trained to relate sEMG signals to joint motion and simultaneously identify key MSK parameters. The numerical examples demonstrate that the proposed framework can effectively identify subject-specific muscle parameters and the trained physics-informed forward-dynamics surrogate yields accurate motion predictions of elbow flexion-extension motion that are in good agreement with the measured joint motion data.