Estimation and Correlation Analysis of Lower Limb Joint Angles Based on Surface Electromyography

AI-Enabled Soft Sensing Array for Simultaneous Detection of Muscle Deformation and Mechanomyography for Metaverse Somatosensory Interaction

Motion recognition (MR)-based somatosensory interaction technology, which interprets user movements as input instructions, presents a natural approach for promoting human-computer interaction, a critical element for advancing metaverse applications. Herein, this work introduces a non-intrusive muscle-sensing wearable device, that in conjunction with machine learning, enables motion-control-based somatosensory interaction with metaverse avatars. To facilitate MR, the proposed device simultaneously detects muscle mechanical activities, including dynamic muscle shape changes and vibrational mechanomyogram signals, utilizing a flexible 16-channel pressure sensor array (weighing ≈0.38 g). Leveraging the rich information from multiple channels, a recognition accuracy of ≈96.06% is achieved by classifying ten lower-limb motions executed by ten human subjects. In addition, this work demonstrates the practical application of muscle-sensing-based somatosensory interaction, using the proposed wearable device, for enabling the real-time control of avatars in a virtual space. This study provides an alternative approach to traditional rigid inertial measurement units and electromyography-based methods for achieving accurate human motion capture, which can further broaden the applications of motion-interactive wearable devices for the coming metaverse age.

Technological advances in lower-limb tele-rehabilitation: A review of literature

Tele-rehabilitation is a healthcare practice that leverages technology to provide rehabilitation services remotely to individuals in their own homes or other locations. With advancements in remote monitoring and Artificial Intelligence, automatic tele-rehabilitation systems that can measure joint angles, recognize exercises, and provide feedback based on movement analysis are being developed. Such platforms can offer valuable information to clinicians for improved care planning. However, with various methods and sensors being used, understanding their pros, cons, and performance is important. This paper reviews and compares the performance of recent vision-based, wearable, and pressure-sensing technologies used in lower limb tele-rehabilitation systems over the past 10 years (from 2014 to 2023). We selected studies that were published in English and focused on joint angle estimation, activity recognition, and exercise assessment. Vision-based approaches were the most common, accounting for 42% of studies. Wearable technology followed at approximately 37%, and pressure-sensing technology appeared in 21% of studies. Identified gaps include a lack of uniformity in reported performance metrics and evaluation methods, a need for cross-subject validation, inadequate testing with patients and older adults, restricted sets of exercises evaluated, and a scarcity of comprehensive datasets on lower limb exercises, especially those involving movements while lying down.

Continuous Estimation of Human Knee Joint Angles by Fusing Kinematic and Myoelectric Signals

Exoskeleton robot is an essential tool in active rehabilitation training for patients with lower limb motor dysfunctions. Accurate and real-time recognition in human motion intention is a great challenge in exoskeleton robot, which can be implemented by continues estimation of human joint angles. In this study, we innovatively proposed a novel feature-based convolutional neural network-bi-directional long-short term memory network (CNN-BiLSTM) model to predict the knee joint angles more accurately and in real time. We validated our method on a public dataset, including surface electromyography(sEMG) and inertial measurement unit (IMU) data of 10 healthy subjects during normal walking. Initially, features extraction from each modal data achieved feature-level fusion. Then the importance of each sEMG and IMU signal feature for knee joint angle prediction was quantified by ensemble feature scorer (EFS) and the number of features required for prediction while ensuring accuracy was simplified by profile likelihood maximization (PLM) algorithm. Finally, the CNN-BiLSTM model was created by using the determined simplest features to further fuse the spatio-temporal correlation of signals. The results indicated that the EFS and PLM algorithm could remove the feature redundancy perfectly and estimation performance would become better when bi-modal gait data were fused. For the estimation performance, the average root mean square error (RMSE), adjusted [Formula: see text] and pearson correlation coefficient (CC) of our algorithm were 4.07, 0.95, and 0.98, respectively, which was better than CNN, BiLSTM and other three traditional machine learning methods. In addition, the model test time was 62.47 ± 0.29 ms, which was less than the predicted horizon of 100 ms. The real-time performance and accuracy are satisfactory. Compared with previous works, our method has great advantages in feature selection and model design, which further improves the prediction accuracy. These promising results demonstrate that the proposed method has considerable potential to be applied to exoskeleton robot control.

Human Motion Pattern Recognition and Feature Extraction: An Approach Using Multi-Information Fusion

An exoskeleton is a kind of intelligent wearable device with bioelectronics and biomechanics. To realize its effective assistance to the human body, an exoskeleton needs to recognize the real time movement pattern of the human body in order to make corresponding movements at the right time. However, it is of great difficulty for an exoskeleton to fully identify human motion patterns, which are mainly manifested as incomplete acquisition of lower limb motion information, poor feature extraction ability, and complicated steps. Aiming at the above consideration, the motion mechanisms of human lower limbs have been analyzed in this paper, and a set of wearable bioelectronics devices are introduced based on an electromyography (EMG) sensor and inertial measurement unit (IMU), which help to obtain biological and kinematic information of the lower limb. Then, the Dual Stream convolutional neural network (CNN)-ReliefF was presented to extract features from the fusion sensors’ data, which were input into four different classifiers to obtain the recognition accuracy of human motion patterns. Compared with a single sensor (EMG or IMU) and single stream CNN or manual designed feature extraction methods, the feature extraction based on Dual Stream CNN-ReliefF shows better performance in terms of visualization performance and recognition accuracy. This method was used to extract features from EMG and IMU data of six subjects and input these features into four different classifiers. The motion pattern recognition accuracy of each subject under the four classifiers is above 97%, with the highest average recognition accuracy reaching 99.12%. It can be concluded that the wearable bioelectronics device and Dual Stream CNN-ReliefF feature extraction method proposed in this paper enhanced an exoskeleton’s ability to capture human movement patterns, thus providing optimal assistance to the human body at the appropriate time. Therefore, it can provide a novel approach for improving the human-machine interaction of exoskeletons.

Estimation of Lower Limb Kinematics during Squat Task in Different Loading Using sEMG Activity and Deep Recurrent Neural Networks

The aim of the present study was to predict the kinematics of the knee and the ankle joints during a squat training task of different intensities. Lower limb surface electromyographic (sEMG) signals and the 3-D kinematics of lower extremity joints were recorded from 19 body builders during squat training at four loading conditions. A long-short term memory (LSTM) was used to estimate the kinematics of the knee and the ankle joints. The accuracy, in terms root-mean-square error (RMSE) metric, of the LSTM network for the knee and ankle joints were 6.774 ± 1.197 and 6.961 ± 1.200, respectively. The LSTM network with inputs processed by cross-correlation (CC) method showed 3.8% and 4.7% better performance in the knee and ankle joints, respectively, compared to when the CC method was not used. Our results showed that in the prediction, regardless of the intensity of movement and inter-subject variability, an off-the-shelf LSTM decoder outperforms conventional fully connected neural networks.

Continuous Myoelectric Prediction of Future Ankle Angle and Moment Across Ambulation Conditions and Their Transitions

A hallmark of human locomotion is that it continuously adapts to changes in the environment and predictively adjusts to changes in the terrain, both of which are major challenges to lower limb amputees due to the limitations in prostheses and control algorithms. Here, the ability of a single-network nonlinear autoregressive model to continuously predict future ankle kinematics and kinetics simultaneously across ambulation conditions using lower limb surface electromyography (EMG) signals was examined. Ankle plantarflexor and dorsiflexor EMG from ten healthy young adults were mapped to normal ranges of ankle angle and ankle moment during level overground walking, stair ascent, and stair descent, including transitions between terrains (i.e., transitions to/from staircase). Prediction performance was characterized as a function of the time between current EMG/angle/moment inputs and future angle/moment model predictions (prediction interval), the number of past EMG/angle/moment input values over time (sampling window), and the number of units in the network hidden layer that minimized error between experimentally measured values (targets) and model predictions of ankle angle and moment. Ankle angle and moment predictions were robust across ambulation conditions with root mean squared errors less than 1° and 0.04 Nm/kg, respectively, and cross-correlations (R²) greater than 0.99 for prediction intervals of 58 ms. Model predictions at critical points of trip-related fall risk fell within the variability of the ankle angle and moment targets (Benjamini-Hochberg adjusted p > 0.065). EMG contribution to ankle angle and moment predictions occurred consistently across ambulation conditions and model outputs. EMG signals had the greatest impact on noncyclic regions of gait such as double limb support, transitions between terrains, and around plantarflexion and moment peaks. The use of natural muscle activation patterns to continuously predict variations in normal gait and the model's predictive capabilities to counteract electromechanical inherent delays suggest that this approach could provide robust and intuitive user-driven real-time control of a wide variety of lower limb robotic devices, including active powered ankle-foot prostheses.

Gesture Recognition Based on Multiscale Singular Value Entropy and Deep Belief Network

As an important research direction of human–computer interaction technology, gesture recognition is the key to realizing sign language translation. To improve the accuracy of gesture recognition, a new gesture recognition method based on four channel surface electromyography (sEMG) signals is proposed. First, the S-transform is applied to four channel sEMG signals to enhance the time-frequency detail characteristics of the signals. Then, multiscale singular value decomposition is applied to the multiple time-frequency matrix output of S-transform to obtain the time-frequency joint features with better robustness. The corresponding singular value permutation entropy is calculated as the eigenvalue to effectively reduce the dimension of multiple eigenvectors. The gesture features are used as input into the deep belief network for classification, and nine kinds of gestures are recognized with an average accuracy of 93.33%. Experimental results show that the multiscale singular value permutation entropy feature is especially suitable for the pattern classification of the deep belief network.

Pulmonary Vein Activity Organization to Determine Atrial Fibrillation Recurrence: Preliminary Data from a Pilot Study

Ablation of pulmonary veins has emerged as a key procedure for normal rhythm restoration in atrial fibrillation patients. However, up to half of ablated Atrial fibrillation (AF) patients suffer recurrences during the first year. In this article, simultaneous intra-atrial recordings registered at pulmonary veins previous to the ablation procedure were analyzed. Spatial cross-correlation and transfer entropy were computed in order to estimate spatial organization. Results showed that, in patients with arrhythmia recurrence, pulmonary vein electrical activity was less correlated than in patients that maintained sinus rhythm. Moreover, correlation function between dipoles showed higher delays in patients with AF recurrence. Results with transfer entropy were consistent with spatial cross-correlation measurements. These results show that arrhythmia drivers located at the pulmonary veins are associated with a higher organization of the electrical activations after the ablation of these sites.