A Machine Learning Processing Pipeline for Reliable Hand Gesture Classification of FMG Signals with Stochastic Variance

A survey on the state of the art of force myography technique (FMG): analysis and assessment

Precise feedback assures precise control commands especially for assistive or rehabilitation devices. Biofeedback systems integrated with assistive or rehabilitative robotic exoskeletons tend to increase its performance and effectiveness. Therefore, there has been plenty of research in the field of biofeedback covering different aspects such as signal acquisition, conditioning, feature extraction and integration with the control system. Among several types of biofeedback systems, Force myography (FMG) technique is a promising one in terms of affordability, high classification accuracies, ease to use, and low computational cost. Compared to traditional biofeedback systems such as electromyography (EMG) which offers some invasive techniques, FMG offers a completely non-invasive solution with much less effort for preprocessing with high accuracies. This work covers the whole aspects of FMG technique in terms of signal acquisition, feature extraction, signal processing, developing the machine learning model, evaluating tools for the performance of the model. Stating the difference between real-time and offline assessment, also highlighting the main uncovered points for further study, and thus enhancing the development of this technique.

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

Robust inter-session modeling of gestures is still an open learning challenge. A sleeve equipped with capacitive strap sensors was used to capture two gesture data sets from a convenience sample of eight subjects. Two pipelines were explored. In FILT a novel two-stage algorithm was introduced which uses an unsupervised learning algorithm to find samples representing gesture transitions and discards them prior to training and validating conventional models. In TSC a confusion matrix was used to automatically consolidate commonly confused class labels, resulting in a set of gestures tailored to an individual subject's abilities. The inter-session testing accuracy using the Time Series Consolidation (TSC) method increased from a baseline inter-session average of 42.47 ± 3.83% to 93.02% ± 4.97% while retaining an average of 5.29 ± 0.46 out of the 11 possible gesture categories. These pipelines used classic machine learning algorithms which require relatively small amounts of data and computational power compared to deep learning solutions. These methods may also offer more flexibility in interface design for users suffering from handicaps limiting their manual dexterity or ability to reliably make gestures, and be possible to implement on edge devices with low computational power.

Building Effective Machine Learning Models for Ankle Joint Power Estimation During Walking Using FMG Sensors

Ankle joint power is usually determined by a complex process that involves heavy equipment and complex biomechanical models. Instead of using heavy equipment, we proposed effective machine learning (ML) and deep learning (DL) models to estimate the ankle joint power using force myography (FMG) sensors. In this study, FMG signals were collected from nine young, healthy participants. The task was to walk on a special treadmill for five different velocities with a respective duration of 1 min. FMG signals were collected from an FMG strap that consists of 8 force resisting sensor (FSR) sensors. The strap was positioned around the lower leg. The ground truth value for ankle joint power was determined with the help of a complex biomechanical model. At first, the predictors' value was preprocessed using a rolling mean filter. Following, three sets of features were formed where the first set includes raw FMG signals, and the other two sets contained time-domain and frequency-domain features extracted using the first set. Cat Boost Regressor (CBR), Long-Short Term Memory (LSTM), and Convolutional Neural Network (CNN) were trained and tested using these three features sets. The results presented in this study showed a correlation coefficient of R = 0.91 ± 0.07 for intrasubject testing and were found acceptable when compared to other similar studies. The CNN on raw features and the LSTM on time-domain features outperformed the other variations. Aside from that, a performance gap between the slowest and fastest walking distance was observed. The results from this study showed that it was possible to achieve an acceptable correlation coefficient in the prediction of ankle joint power using FMG sensors with an appropriate combination of feature set and ML model.

Phase-Based Grasp Classification for Prosthetic Hand Control Using sEMG

Pattern recognition using surface Electromyography (sEMG) applied on prosthesis control has attracted much attention in these years. In most of the existing methods, the sEMG signal during the firmly grasped period is used for grasp classification because good performance can be achieved due to its relatively stable signal. However, using the only the firmly grasped period may cause a delay to control the prosthetic hand gestures. Regarding this issue, we explored how grasp classification accuracy changes during the reaching and grasping process, and identified the period that can leverage the grasp classification accuracy and the earlier grasp detection. We found that the grasp classification accuracy increased along the hand gradually grasping the object till firmly grasped, and there is a sweet period before firmly grasped period, which could be suitable for early grasp classification with reduced delay. On top of this, we also explored corresponding training strategies for better grasp classification in real-time applications.