Hilbert sEMG data scanning for hand gesture recognition based on deep learning

Hand gesture recognition with deep residual network using Semg signal

OBJECTIVES

To design and develop a classifier, named Sewing Driving Training based Optimization-Deep Residual Network (SDTO_DRN) for hand gesture recognition.

METHODS

The electrical activity of forearm muscles generates the signals that can be captured with Surface Electromyography (sEMG) sensors and includes meaningful data for decoding both muscle actions and hand movement. This research develops an efficacious scheme for hand gesture recognition using SDTO_DRN. Here, signal pre-processing is done through Gaussian filtering. Thereafter, desired and appropriate features are extracted. Following that, effective features are chosen using SDTO. At last, hand gesture identification is accomplished based on DRN and this network is effectively fine-tuned by SDTO, which is a combination of Sewing Training Based Optimization (STBO) and Driving Training Based Optimization (DTBO). The datasets employed for the implementation of this work are MyoUP Dataset and putEMG: sEMG Gesture and Force Recognition Dataset.

RESULTS

The designed SDTO_DRN model has gained superior performance with magnificent results by delivering a maximum accuracy of 0.943, True Positive Rate (TPR) of 0.929, True Negative Rate (TNR) of 0.919, Positive Predictive Value (PPV) of 0.924, and Negative Predictive Value (NPV) of 0.924.

CONCLUSIONS

The hand gesture recognition using the proposed model is accurate and improves the effectiveness of the recognition.

The Impact of Feature Extraction on Classification Accuracy Examined by Employing a Signal Transformer to Classify Hand Gestures Using Surface Electromyography Signals

Interest in developing techniques for acquiring and decoding biological signals is on the rise in the research community. This interest spans various applications, with a particular focus on prosthetic control and rehabilitation, where achieving precise hand gesture recognition using surface electromyography signals is crucial due to the complexity and variability of surface electromyography data. Advanced signal processing and data analysis techniques are required to effectively extract meaningful information from these signals. In our study, we utilized three datasets: NinaPro Database 1, CapgMyo Database A, and CapgMyo Database B. These datasets were chosen for their open-source availability and established role in evaluating surface electromyography classifiers. Hand gesture recognition using surface electromyography signals draws inspiration from image classification algorithms, leading to the introduction and development of the Novel Signal Transformer. We systematically investigated two feature extraction techniques for surface electromyography signals: the Fast Fourier Transform and wavelet-based feature extraction. Our study demonstrated significant advancements in surface electromyography signal classification, particularly in the Ninapro database 1 and CapgMyo dataset A, surpassing existing results in the literature. The newly introduced Signal Transformer outperformed traditional Convolutional Neural Networks by excelling in capturing structural details and incorporating global information from image-like signals through robust basis functions. Additionally, the inclusion of an attention mechanism within the Signal Transformer highlighted the significance of electrode readings, improving classification accuracy. These findings underscore the potential of the Signal Transformer as a powerful tool for precise and effective surface electromyography signal classification, promising applications in prosthetic control and rehabilitation.

Efficient entry point encoding and decoding algorithms on 2D Hilbert space filling curve

The Hilbert curve is an important method for mapping high-dimensional spatial information into one-dimensional spatial information while preserving the locality in the high-dimensional space. Entry points of a Hilbert curve can be used for image compression, dimensionality reduction, corrupted image detection and many other applications. As far as we know, there is no specific algorithms developed for entry points. To address this issue, in this paper we present an efficient entry point encoding algorithm (EP-HE) and a corresponding decoding algorithm (EP-HD). These two algorithms are efficient by exploiting the m consecutive 0s in the rear part of an entry point. We further found that the outputs of these two algorithms are a certain multiple of a certain bit of s, where s is the starting state of these m levels. Therefore, the results of these m levels can be directly calculated without iteratively encoding and decoding. The experimental results show that these two algorithms outperform their counterparts in terms of processing entry points.

An Extended Spatial Transformer Convolutional Neural Network for Gesture Recognition and Self-Calibration Based on Sparse sEMG Electrodes

sEMG-based gesture recognition is widely applied in human-machine interaction system by its unique advantages. However, the accuracy of recognition drops significantly as electrodes shift. Besides, in applications such as VR, virtual hands should be shown in reasonable posture by self-calibration. We propose an armband fusing sEMG and IMU with autonomously adjustable gain, and an extended spatial transformer convolutional neural network (EST-CNN) with feature enhanced pretreatment (FEP) to accomplish both gesture recognition and self-calibration via a one-shot processing. Different from anthropogenic calibration methods, spatial transformer layers (STL) in EST-CNN automatically learn the transformation relation, and explicitly express the rotational angle for coarse correction. Due to the shape change of feature pattern as rotational shift, we design the fine tuning layer (FTL) which is able to regulate rotational angle within 45°. By combining STL, FTL and IMU-based posture, EST-CNN is able to calculate non-discretized angle, and achieves high resolution of posture estimation based on sparse sEMG electrodes. Experiments collect frequently-used 3 gestures of 4 subjects in equidistant angles to evaluate EST-CNN. The results under electrodes shift show that the accuracy of gesture recognition is 97.06%, which is 5.81% higher than CNN, the fitness between estimated and true rotational angle is 99.44%.

Looseness Identification of Track Fasteners Based on Ultra-Weak FBG Sensing Technology and Convolutional Autoencoder Network

Changes in the geological environment and track wear, and deterioration of train bogies may lead to the looseness of subway fasteners. Identifying loose fasteners randomly distributed along the subway line is of great significance to avoid train derailment. This paper presents a convolutional autoencoder (CAE) network-based method for identifying fastener loosening features from the distributed vibration responses of track beds detected by an ultra-weak fiber Bragg grating sensing array. For an actual subway tunnel monitoring system, a field experiment used to collect the samples of fastener looseness was designed and implemented, where a crowbar was used to loosen or tighten three pairs of fasteners symmetrical on both sides of the track within the common track bed area and the moving load of a rail inspection vehicle was employed to generate 12 groups of distributed vibration signals of the track bed. The original vibration signals obtained from the on-site test were converted into two-dimensional images through the pseudo-Hilbert scan to facilitate the proposed two-stage CAE network with acceptable capabilities in feature extraction and recognition. The performance of the proposed methodology was quantified by accuracy, precision, recall, and F1-score, and displayed intuitively by t-distributed stochastic neighbor embedding (t-SNE). The raster scan and the Hilbert scan were selected to compare with the pseudo-Hilbert scan under a similar CAE network architecture. The identification performance results represented by the four quantification indicators (accuracy, precision, recall, and F1-score) based on the scan strategy in this paper were at least 23.8%, 9.5%, 20.0%, and 21.1% higher than those of the two common scan methods. As well as that, the clustering visualization by t-SNE further verified that the proposed approach had a stronger ability in distinguishing the feature of fastener looseness.