Real-time continuous recognition of knee motion using multi-channel mechanomyography signals detected on clothes

MMG-Based Knee Dynamic Extension Force Estimation Using Cross-Talk and IGWO-LSTM

Mechanomyography (MMG) is an important muscle physiological activity signal that can reflect the amount of motor units recruited as well as the contraction frequency. As a result, MMG can be utilized to estimate the force produced by skeletal muscle. However, cross-talk and time-series correlation severely affect MMG signal recognition in the real world. These restrict the accuracy of dynamic muscle force estimation and their interaction ability in wearable devices. To address these issues, a hypothesis that the accuracy of knee dynamic extension force estimation can be improved by using MMG signals from a single muscle with less cross-talk is first proposed. The hypothesis is then confirmed using the estimation results from different muscle signal feature combinations. Finally, a novel model (improved grey wolf optimizer optimized long short-term memory networks, i.e., IGWO-LSTM) is proposed for further improving the performance of knee dynamic extension force estimation. The experimental results demonstrate that MMG signals from a single muscle with less cross-talk have a superior ability to estimate dynamic knee extension force. In addition, the proposed IGWO-LSTM provides the best performance metrics in comparison to other state-of-the-art models. Our research is expected to not only improve the understanding of the mechanisms of quadriceps contraction but also enhance the flexibility and interaction capabilities of future rehabilitation and assistive devices.

Mechanomyography signals pattern recognition in hand movements using swarm intelligence algorithm optimized support vector machine based on acceleration sensors

On the basis of extracting mechanomyography (MMG) signal features, the classification of hand movements has certain application values in human-machine interaction systems and wearable devices. In this paper, pattern recognition of hand movements based on MMG signal is studied with swarm intelligence algorithms introduced to optimize support vector machine (SVM). Time domain (TD) features, wavelet packet node energy (WPNE) features, frequency domain (FD) features, convolution neural network (CNN) features were extracted from each channel to constitute different feature sets. Three novel swarm intelligence algorithms (i.e., bald eagle search (BES), sparrow search algorithm (SSA), grey wolf optimization (GWO)) optimized SVM is proposed to train the models and recognition of hand movements are tested for each MMG feature extraction method. Using GWO as the optimization algorithm, time consumption is less than using the other two swarm algorithms. Using GWO with TD+FD features can obtain the classification accuracy of 93.55 %, which is higher than other methods while using CNN to extract features can be independent of domain knowledge. The results confirm GWO-SVM with TD + FD features is superior to some other methods in the classification problem for tiny samples based on MMG.

Mechanomyography Signal Pattern Recognition of Knee and Ankle Movements Using Swarm Intelligence Algorithm-Based Feature Selection Methods

Pattern recognition of lower-limb movements based on mechanomyography (MMG) signals has a certain application value in the study of wearable rehabilitation-training devices. In this paper, MMG feature selection methods based on a chameleon swarm algorithm (CSA) and a grasshopper optimization algorithm (GOA) are proposed for the pattern recognition of knee and ankle movements in the sitting and standing positions. Wireless multichannel MMG acquisition systems were designed and used to collect MMG movements from four sites on the subjects thighs. The relationship between the threshold values and classification accuracy was analyzed, and comparatively high recognition rates were obtained after redundant information was eliminated. When the threshold value rose, the recognition rates from the CSA fluctuated within a small range: up to 88.17% (sitting position) and 90.07% (standing position). However, the recognition rates from the GOA drop dramatically when increasing the threshold value. The comparison results demonstrated that using a GOA consumes less time and selects fewer features, while a CSA gives higher recognition rates of knee and ankle movements.

Estimation of Knee Extension Force Using Mechanomyography Signals Based on GRA and ICS-SVR

During lower-extremity rehabilitation training, muscle activity status needs to be monitored in real time to adjust the assisted force appropriately, but it is a challenging task to obtain muscle force noninvasively. Mechanomyography (MMG) signals offer unparalleled advantages over sEMG, reflecting the intention of human movement while being noninvasive. Therefore, in this paper, based on MMG, a combined scheme of gray relational analysis (GRA) and support vector regression optimized by an improved cuckoo search algorithm (ICS-SVR) is proposed to estimate the knee joint extension force. Firstly, the features reflecting muscle activity comprehensively, such as time-domain features, frequency-domain features, time–frequency-domain features, and nonlinear dynamics features, were extracted from MMG signals, and the relational degree was calculated using the GRA method to obtain the correlation features with high relatedness to the knee joint extension force sequence. Then, a combination of correlated features with high relational degree was input into the designed ICS-SVR model for muscle force estimation. The experimental results show that the evaluation indices of the knee joint extension force estimation obtained by the combined scheme of GRA and ICS-SVR were superior to other regression models and could estimate the muscle force with higher estimation accuracy. It is further demonstrated that the proposed scheme can meet the need of muscle force estimation required for rehabilitation devices, powered prostheses, etc.

Wearable Iontronic FMG for Classification of Muscular Locomotion

Human motion recognition with high accuracy, fast response speed has long been considered an essential component in human-machine interactive activities such as assistive robotics, medical prosthesis, and wearable electronics. This study proposed a novel human lower limb locomotion classification strategy based on flexible supercapacitive iontronic sensors. Benefiting from the ultrahigh sensitivity (up to 1 nF/mmHg) and low activation pressure (less than 3 mmHg) of the supercapacitive iontronic pressure sensor, force myography (FMG) signal was acquired more accurately from 5 iontronic sensors strapped to the thigh (5 percentage point improvement compared with force sensitive resistor (FSR) in low window length). In the experiment with 12 subjects, the real-time classification strategy based on sliding window and SVM model gave an average locomotion classification accuracy of 99% for seven categories, including sitting, standing, walking on level ground, ramp ascent, ramp descent, stair ascent, stair descent. Compared with traditional FSR sensors, the result showed that iontronic sensors improved the classification accuracy by 5 percentage points in the case of short time window. The implementation of the high sensitivity flexible iontronic sensors in the wearable system brings a valuable tool for detecting small human body pressure signals and has great potential to improve the performance of the human-machine interface in rehabilitation and medical applications.

A Framework for Maternal Physical Activities and Health Monitoring Using Wearable Sensors

We propose a physical activity recognition and monitoring framework based on wearable sensors during maternity. A physical activity can either create or prevent health issues during a given stage of pregnancy depending on its intensity. Thus, it becomes very important to provide continuous feedback by recognizing a physical activity and its intensity. However, such continuous monitoring is very challenging during the whole period of maternity. In addition, maintaining a record of each physical activity, and the time for which it was performed, is also a non-trivial task. We aim at such problems by first recognizing a physical activity via the data of wearable sensors that are put on various parts of body. We avoid the use of smartphones for such task due to the inconvenience caused by wearing it for activities such as “eating”. In our proposed framework, a module worn on body consists of three sensors: a 3-axis accelerometer, 3-axis gyroscope, and temperature sensor. The time-series data from these sensors are sent to a Raspberry-PI via Bluetooth Low Energy (BLE). Various statistical measures (features) of this data are then calculated and represented in features vectors. These feature vectors are then used to train a supervised machine learning algorithm called classifier for the recognition of physical activity from the sensors data. Based on such recognition, the proposed framework sends a message to the care-taker in case of unfavorable situation. We evaluated a number of well-known classifiers on various features developed from overlapped and non-overlapped window size of time-series data. Our novel dataset consists of 10 physical activities performed by 61 subjects at various stages of maternity. On the current dataset, we achieve the highest recognition rate of 89% which is encouraging for a monitoring and feedback system.

Recognition of basketball players’ action detection based on visual image and Harris corner extraction algorithm

At present, there are efficiency problems in related algorithms for athlete detection and recognition. Based on this, this study analyzes the characteristics of athletes’ sports process. In this study, the Otsu method was used to perform grayscale feature processing. At the same time, based on the Harris corner extraction algorithm, this study proposes that the multi-target tracking combined with the corner feature of the target can be used to track different parts of the athlete as different target areas. In addition, this study uses a sequential algorithm to perform connected component labeling. Finally, in order to test the performance and recognition efficiency of the proposed algorithm, the performance of the algorithm is explored through experimental analysis. The research shows that the algorithm has good performance and has certain practical effects, and it has certain reference significance for subsequent related research.

Electrical stimulator with mechanomyography-based real-time monitoring, muscle fatigue detection, and safety shut-off: a pilot study

Functional electrical stimulation (FES) has been used to produce force-related activities on the paralyzed muscle among spinal cord injury (SCI) individuals. Early muscle fatigue is an issue in all FES applications. If not properly monitored, overstimulation can occur, which can lead to muscle damage. A real-time mechanomyography (MMG)-based FES system was implemented on the quadriceps muscles of three individuals with SCI to generate an isometric force on both legs. Three threshold drop levels of MMG-root mean square (MMG-RMS) feature (thr50, thr60, and thr70; representing 50%, 60%, and 70% drop from initial MMG-RMS values, respectively) were used to terminate the stimulation session. The mean stimulation time increased when the MMG-RMS drop threshold increased (thr50: 22.7 s, thr60: 25.7 s, and thr70: 27.3 s), indicating longer sessions when lower performance drop was allowed. Moreover, at thr70, the torque dropped below 50% from the initial value in 14 trials, more than at thr50 and thr60. This is a clear indication of muscle fatigue detection using the MMG-RMS value. The stimulation time at thr70 was significantly longer (p = 0.013) than that at thr50. The results demonstrated that a real-time MMG-based FES monitoring system has the potential to prevent the onset of critical muscle fatigue in individuals with SCI in prolonged FES sessions.

A long short-term memory neural network model for knee joint acceleration estimation using mechanomyography signals

With the growth of the number of elderly and disabled with motor dysfunction, the demand for assisted exercise is increasing. Wearable power assistance robots are developed to provide athletic ability of limbs for the elderly or the disabled who have weakened limbs to better self-care ability. Existing wearable power-assisted robots generally use surface electromyography (sEMG) to obtain effective human motion intentions. Due to the characteristics of sEMG signals, it is limited in many applications. To solve the above problems, we design a long short-term memory (LSTM) neural network model based on human mechanomyography (MMG) signals to estimate the motion acceleration of knee joint. The acceleration can be further calculated by the torque required for movement control of the wearable power assistance robots for the lower limb. We detect MMG signals on the clothed thigh, extract features of the MMG signals, and then, use principal component analysis to reduce the features’ dimensions. Finally, the dimension-reduced features are inputted into the LSTM neural network model in time series for estimating the acceleration. The experimental results show that the average correlation coefficient ( R) is 94.48 ± 1.91% for the estimation of acceleration in the process of continuously performing under approximately π/4 rad/s. This approach can be applied in the practical applications of wearable field.

A PILOT STUDY OF MECHANOMYOGRAPHY-BASED HAND MOVEMENTS RECOGNITION EMPHASIZING ON THE INFLUENCE OF FABRICS BETWEEN SENSOR AND SKIN

Multi-channel mechanomyography (MMG) signals were acquired from the forearm when the subjects were performing eight classes of hand movements related to rehabilitation training. Ten time domain (TD) features and wavelet packet node energy (WPNE) features were extracted from each channel of MMG, and the hand movements were classified by support vector machine (SVM), extreme learning machine (ELM), linear discriminant analysis (LDA) and [Formula: see text]-nearest neighborhood (KNN) and the classifying results of three methods of collecting MMG (sensors directly on skin, sensors on cotton fabric and sensors on acrylic fiber) were compared. When all TD features were selected and SVM was adopted as the classifier, the total recognition rates of hand movements were 94.0%, 93.9% and 93.6%, respectively, of three collection methods. Using ELM can obtain similar results as SVM, with the recognition rates of 94.3%, 94.3% and 94.1%, respectively, better than using LDA (88.5%, 88.6% and 88.0%) or KNN (88.9%, 89.4% and 89.0%). For each algorithm, using TD features can acquire the highest recognition rates. Once the feature set and the classifier were selected, the total recognition rates were almost equally among three collection methods (especially for some feature sets, the differences are smaller than 1%). The results confirmed that satisfactory effects could be acquired even when the MMG was collected from sensors on fabrics with specific material, thus indicating that MMG has a unique potential value for developing wearable devices.

Research on GA-SVM Based Head-Motion Classification via Mechanomyography Feature Analysis

This study investigated classification of six types of head motions using mechanomyography (MMG) signals. An unequal segmenting algorithm was adopted to segment the MMG signals generated by head motions. Three types of features (time domain, time-frequency domain and nonlinear dynamics) were extracted to construct five feature sets as candidate datasets for classification analysis. Genetic algorithm optimized support vector machine (GA-SVM) was used to classify the MMG signals. Three different kernel functions, different combinations of feature sets, different number of signal channels and training samples were selected for comparative analysis to evaluate the classification accuracy. Experimental results showed that the classifier had the best overall classification accuracy when using the radial basis function (RBF). Any combination of three different types of feature sets guaranteed an average accuracy of over 80%. In the case of the best combination (feature set 2 + 3 + 5), the classification accuracy was up to 88.2%. Using four channels to acquire MMG signal and no less than 60 training samples can assure a satisfactory classification accuracy.

A CNN-SVM combined model for pattern recognition of knee motion using mechanomyography signals

The commonly used classifiers for pattern recognition of human motion, like backpropagation neural network (BPNN) and support vector machine (SVM), usually implement the classification by extracting some hand-crafted features from the human biological signals. These features generally require the domain knowledge for researchers to be designed and take a long time to be tested and selected for high classification performance. In contrast, convolutional neural network (CNN), which has been widely applied to computer vision, can learn to automatically extract features from the training data by means of convolution and subsampling, but CNN training usually requires large sample data and has the overfitting problem. On the other hand, SVM has good generalization ability and can solve the small sample problem. Therefore, we proposed a CNN-SVM combined model to make use of their advantages. In this paper, we detected 4-channel mechanomyography (MMG) signals from the thigh muscles and fed them in the form of time series signals to the CNN-SVM combined model for the pattern recognition of knee motion. Compared with the common classifier performing the classification with hand-crafted features, the CNN-SVM combined model could automatically extract features using CNN, and better improved the generalization ability of CNN and the classification accuracy by means of combining the SVM. This study would provide reference for human motion recognition using other time series signals and further expand the application fields of CNN.