Generalized EMG-based isometric contact force estimation using a deep learning approach

Towards Efficient Neural Decoder for Dexterous Finger Force Predictions

OBJECTIVE

Dexterous control of robot hands requires a robust neural-machine interface capable of accurately decoding multiple finger movements. Existing studies primarily focus on single-finger movement or rely heavily on multi-finger data for decoder training, which requires large datasets and high computation demand. In this study, we investigated the feasibility of using limited single-finger surface electromyogram (sEMG) data to train a neural decoder capable of predicting the forces of unseen multi-finger combinations.

METHODS

We developed a deep forest-based neural decoder to concurrently predict the extension and flexion forces of three fingers (index, middle, and ring-pinky). We trained the model using varying amounts of high-density EMG data in a limited condition (i.e., single-finger data).

RESULTS

We showed that the deep forest decoder could achieve consistently commendable performance with 7.0% of force prediction errors and R2 value of 0.874, significantly surpassing the conventional EMG amplitude method and convolutional neural network approach. However, the deep forest decoder accuracy degraded when a smaller amount of data was used for training and when the testing data became noisy.

CONCLUSION

The deep forest decoder shows accurate performance in multi-finger force prediction tasks. The efficiency aspect of the deep forest lies in the short training time and small volume of training data, which are two critical factors in current neural decoding applications.

SIGNIFICANCE

This study offers insights into efficient and accurate neural decoder training for advanced robotic hand control, which has the potential for real-life applications during human-machine interactions.

An early force prediction control scheme using multimodal sensing of electromyography and digit force signals

Different grasping gestures result in the change of muscular activity of the forearm muscles. Similarly, the muscular activity changes with a change in grip force while grasping the object. This change in muscular activity, measured by a technique called Electromyography (EMG) is used in the upper limb bionic devices to select the grasping gesture. Previous research studies have shown gesture classification using pattern recognition control schemes. However, the use of EMG signals for force manipulation is less focused, especially during precision grasping. In this study, an early predictive control scheme is designed for the efficient determination of grip force using EMG signals from forearm muscles and digit force signals. The optimal pattern recognition (PR) control schemes are investigated using three different inputs of two signals: EMG signals, digit force signals and a combination of EMG and digit force signals. The features extracted from EMG signals included Slope Sign Change, Willison Amplitude, Auto Regressive Coefficient and Waveform Length. The classifiers used to predict force levels are Random Forest, Gradient Boosting, Linear Discriminant Analysis, Support Vector Machines, k-nearest Neighbors and Decision Tree. The two-fold objectives of early prediction and high classification accuracy of grip force level were obtained using EMG signals and digit force signals as inputs and Random Forest as a classifier. The earliest prediction was possible at 1000 ms from the onset of the gripping of the object with a mean classification accuracy of 90 % for different grasping gestures. Using this approach to study, an early prediction will result in the determination of force level before the object is lifted from the surface. This approach will also result in better biomimetic regulation of the grip force during precision grasp, especially for a population facing vision deficiency.

Generalizing Upper Limb Force Modeling With Transfer Learning: A Multimodal Approach Using EMG and IMU for New Users and Conditions

In the field of EMG-based force modeling, the ability to generalize models across individuals could play a significant role in its adoption across a range of applications, including assistive devices, robotic and rehabilitation devices. However, current studies have predominately focused on intra-subject modeling, largely neglecting the burden of end-user data acquisition. In this work, we propose the use of transfer learning (TL) to generalize force modeling to a new user by first establishing a baseline model trained using other users' data, and then adapting to the end-user using a small amount of new data (only 10% , 20% , and 40% of the new user data). Using a deep multimodal convolutional neural network, consisting of two CNN models, one with high-density (HD) EMG and one with motion data recorded by an Inertial Measurement Unit (IMU), our proposed TL technique significantly improved force modeling compared to leave-one-subject-out (LOSO) and even intra-subject scenarios. The TL approach increased the average R squared values of the force modeling task by 60.81%, 190.53%, and 199.79% compared to the LOSO case, and by 13.4%, 36.88%, and 45.51% compared to the intra-subject case for isotonic, isokinetic and dynamic conditions, respectively. These results show that it is possible to adapt to a new user with minimal data while improving performance significantly compared to the intra-subject scenario. We also show that TL can be used to generalize on a new experimental condition for a new user.

A membership-function-based broad learning system for human-robot interaction force estimation under drawing task

Estimating interaction force is of great significance in the field of human-robot interaction (HRI) thanks to its guarantee of interaction safety. To this end, this paper proposes a novel estimation method by leveraging broad learning system (BLS) and human surface electromyography (sEMG) signals. Since the previous sEMG may also contain valuable information of human muscle force, it would cause the estimation to be incomplete and abate the estimation accuracy in the case of neglecting the previous sEMG. To remedy this thorn, a new linear membership function is first developed to calculate contributions of sEMG at different sampling times in the proposed method. Subsequently, the contribution values calculated by the membership function are integrated with features of sEMG to be considered as the input layer of BLS. For extensive studies, five different features extracted from sEMG signals and their combination are explored to estimate the interaction force by the proposed method. Lastly, the performance of the proposed method is compared with those of three well-known methods through experimental test regarding the drawing task. The experimental results confirm that combining the time domain (TD) with frequency domain (FD) features of sEMG can enhance the estimation quality. Moreover, the proposed method outperforms its contenders with respect to estimation accuracy.

Deep Multi-scale Fusion of Convolutional Neural Networks for EMG-based Movement Estimation

EMG-based motion estimation is required for applications such as myoelectric control, where the simultaneous estimation of kinematic information, namely joint angle and velocity, is challenging and critical. We propose a novel method for accurately modelling the generated joint angle and velocity simultaneously under isotonic, isokinetic (quasi-dynamic), and fully dynamic conditions. Our solution uses two streams of CNN, called TS-CNN to learn informative features from raw EMG data using different scales and estimate the generated motion during elbow flexion and extension. The experimental results show the robustness of our approach in comparison to conventional CNN as well as some methods used in the literature. The best obtained R2 values, are 0.81±0.06, 0.71±0.06, and 0.80±0.13 for joint angle estimation and 0.78±0.05, 0.79±0.07, and 0.71±0.13 for the velocity estimation, during isotonic, isokinetic, and dynamic contractions, respectively. Additionally, our results indicate that the experimental condition can have an impact on the model's performance for motion prediction. EMG-based velocity estimation obtains higher performance than joint angle estimation under isokinetic conditions. Under dynamic conditions, joint angle estimation is more accurate than velocity estimation, and there is no difference between joint angle and velocity estimation in the isotonic case.

A Mirror Bilateral Neuro-Rehabilitation Robot System with the sEMG-Based Real-Time Patient Active Participant Assessment

In this paper, a novel mirror visual feedback-based (MVF) bilateral neurorehabilitation system with surface electromyography (sEMG)-based patient active force assessment was proposed for upper limb motor recovery and improvement of limb inter-coordination. A mirror visual feedback-based human–robot interface was designed to facilitate the bilateral isometric force output training task. To achieve patient active participant assessment, an sEMG signals-based elbow joint isometric force estimation method was implemented into the proposed system for real-time affected side force assessment and participation evaluation. To assist the affected side limb efficiently and precisely, a mirror bilateral control framework was presented for bilateral limb coordination. Preliminary experiments were conducted to evaluate the estimation accuracy of force estimation method and force tracking accuracy of system performance. The experimental results show the proposed force estimation method can efficiently calculate the elbow joint force in real-time, and the affected side limb of patients can be assisted to track output force of the non-paretic side limb for better limb coordination by the proposed bilateral rehabilitation system.