Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier

Millimeter wave gesture recognition using multi-feature fusion models in complex scenes

As a form of body language, the gesture plays an important role in smart homes, game interactions, and sign language communication, etc. The gesture recognition methods have been carried out extensively. The existing methods have inherent limitations regarding user experience, visual environment, and recognition granularity. Millimeter wave radar provides an effective method for the problems lie ahead gesture recognition because of the advantage of considerable bandwidth and high precision perception. Interfering factors and the complexity of the model raise an enormous challenge to the practical application of gesture recognition methods as the millimeter wave radar is applied to complex scenes. Based on multi-feature fusion, a gesture recognition method for complex scenes is proposed in this work. We collected data in variety places to improve sample reliability, filtered clutters to improve the signal's signal-to-noise ratio (SNR), and then obtained multi features involves range-time map (RTM), Doppler-time map (DTM) and angle-time map (ATM) and fused them to enhance the richness and expression ability of the features. A lightweight neural network model multi-CNN-LSTM is designed to gestures recognition. This model consists of three convolutional neural network (CNN) for three obtained features and one long short-term memory (LSTM) for temporal features. We analyzed the performance and complexity of the model and verified the effectiveness of feature extraction. Numerous experiments have shown that this method has generalization ability, adaptability, and high robustness in complex scenarios. The recognition accuracy of 14 experimental gestures reached 97.28%.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Evaluation of Hand Action Classification Performance Using Machine Learning Based on Signals from Two sEMG Electrodes

Classification-based myoelectric control has attracted significant interest in recent years, leading to prosthetic hands with advanced functionality, such as multi-grip hands. Thus far, high classification accuracies have been achieved by increasing the number of surface electromyography (sEMG) electrodes or adding other sensing mechanisms. While many prescribed myoelectric hands still adopt two-electrode sEMG systems, detailed studies on signal processing and classification performance are still lacking. In this study, nine able-bodied participants were recruited to perform six typical hand actions, from which sEMG signals from two electrodes were acquired using a Delsys Trigno Research+ acquisition system. Signal processing and machine learning algorithms, specifically, linear discriminant analysis (LDA), k-nearest neighbors (KNN), and support vector machines (SVM), were used to study classification accuracies. Overall classification accuracy of 93 ± 2%, action-specific accuracy of 97 ± 2%, and F1-score of 87 ± 7% were achieved, which are comparable with those reported from multi-electrode systems. The highest accuracies were achieved using SVM algorithm compared to LDA and KNN algorithms. A logarithmic relationship between classification accuracy and number of features was revealed, which plateaued at five features. These comprehensive findings may potentially contribute to signal processing and machine learning strategies for commonly prescribed myoelectric hand systems with two sEMG electrodes to further improve functionality.

Mechanomyography signals pattern recognition in hand movements using swarm intelligence algorithm optimized support vector machine based on acceleration sensors

On the basis of extracting mechanomyography (MMG) signal features, the classification of hand movements has certain application values in human-machine interaction systems and wearable devices. In this paper, pattern recognition of hand movements based on MMG signal is studied with swarm intelligence algorithms introduced to optimize support vector machine (SVM). Time domain (TD) features, wavelet packet node energy (WPNE) features, frequency domain (FD) features, convolution neural network (CNN) features were extracted from each channel to constitute different feature sets. Three novel swarm intelligence algorithms (i.e., bald eagle search (BES), sparrow search algorithm (SSA), grey wolf optimization (GWO)) optimized SVM is proposed to train the models and recognition of hand movements are tested for each MMG feature extraction method. Using GWO as the optimization algorithm, time consumption is less than using the other two swarm algorithms. Using GWO with TD+FD features can obtain the classification accuracy of 93.55 %, which is higher than other methods while using CNN to extract features can be independent of domain knowledge. The results confirm GWO-SVM with TD + FD features is superior to some other methods in the classification problem for tiny samples based on MMG.

TraHGR: Transformer for Hand Gesture Recognition via Electromyography

Deep learning-based Hand Gesture Recognition (HGR) via surface Electromyogram (sEMG) signals have recently shown considerable potential for development of advanced myoelectric-controlled prosthesis. Although deep learning techniques can improve HGR accuracy compared to their classical counterparts, classifying hand movements based on sparse multichannel sEMG signals is still a challenging task. Furthermore, existing deep learning approaches, typically, include only one model as such can hardly extract representative features. In this paper, we aim to address this challenge by capitalizing on the recent advances in hybrid models and transformers. In other words, we propose a hybrid framework based on the transformer architecture, which is a relatively new and revolutionizing deep learning model. The proposed hybrid architecture, referred to as the Transformer for Hand Gesture Recognition (TraHGR), consists of two parallel paths followed by a linear layer that acts as a fusion center to integrate the advantage of each module. We evaluated the proposed architecture TraHGR based on the commonly used second Ninapro dataset, referred to as the DB2. The sEMG signals in the DB2 dataset are measured in real-life conditions from 40 healthy users, each performing 49 gestures. We have conducted an extensive set of experiments to test and validate the proposed TraHGR architecture, and compare its achievable accuracy with several recently proposed HGR classification algorithms over the same dataset. We have also compared the results of the proposed TraHGR architecture with each individual path and demonstrated the distinguishing power of the proposed hybrid architecture. The recognition accuracies of the proposed TraHGR architecture for the window of size 200ms and step size of 100ms are 86.00%, 88.72%, 81.27%, and 93.74%, which are 2.30%, 4.93%, 8.65%, and 4.20% higher than the state-of-the-art performance for DB2 (49 gestures), DB2-B (17 gestures), DB2-C (23 gestures), and DB2-D (9 gestures), respectively.

Forearm sEMG data from young healthy humans during the execution of hand movements

This work provides a complete dataset containing surface electromyography (sEMG) signals acquired from the forearm with a sampling frequency of 1000 Hz. The dataset is named WyoFlex sEMG Hand Gesture and recorded the data of 28 participants between 18 and 37 years old without neuromuscular diseases or cardiovascular problems. The test protocol consisted of sEMG signals acquisition corresponding to ten wrist and grasping movements (extension, flexion, ulnar deviation, radial deviation, hook grip, power grip, spherical grip, precision grip, lateral grip, and pinch grip), considering three repetitions for each gesture. Also, the dataset contains general information such as anthropometric measures of the upper limb, gender, age, laterally of the person, and physical condition. Likewise, the implemented acquisition system consists of a portable armband with four sEMG channels distributed equidistantly for each forearm. The database could be used for the recognition of hand gestures, evaluation of the evolution of patients in rehabilitation processes, control of upper limb orthoses or prostheses, and biomechanical analysis of the forearm.

sEMG-Based Hand Gesture Recognition Using Binarized Neural Network

Recently, human-machine interfaces (HMI) that make life convenient have been studied in many fields. In particular, a hand gesture recognition (HGR) system, which can be implemented as a wearable system, has the advantage that users can easily and intuitively control the device. Among the various sensors used in the HGR system, the surface electromyography (sEMG) sensor is independent of the acquisition environment, easy to wear, and requires a small amount of data. Focusing on these advantages, previous sEMG-based HGR systems used several sensors or complex deep-learning algorithms to achieve high classification accuracy. However, systems that use multiple sensors are bulky, and embedded platforms with complex deep-learning algorithms are difficult to implement. To overcome these limitations, we propose an HGR system using a binarized neural network (BNN), a lightweight convolutional neural network (CNN), with one dry-type sEMG sensor, which is implemented on a field-programmable gate array (FPGA). The proposed HGR system classifies nine dynamic gestures that can be useful in real life rather than static gestures that can be classified relatively easily. Raw sEMG data collected from a dynamic gesture are converted into a spectrogram with information in the time-frequency domain and transferred to the classifier. As a result, the proposed HGR system achieved 95.4% classification accuracy, with a computation time of 14.1 ms and a power consumption of 91.81 mW.

Hand Gestures Recognition for Human-Machine Interfaces: A Low-Power Bio-Inspired Armband

Hand gesture recognition has recently increased its popularity as Human-Machine Interface (HMI) in the biomedical field. Indeed, it can be performed involving many different non-invasive techniques, e.g., surface ElectroMyoGraphy (sEMG) or PhotoPlethysmoGraphy (PPG). In the last few years, the interest demonstrated by both academia and industry brought to a continuous spawning of commercial and custom wearable devices, which tried to address different challenges in many application fields, from tele-rehabilitation to sign language recognition. In this work, we propose a novel 7-channel sEMG armband, which can be employed as HMI for both serious gaming control and rehabilitation support. In particular, we designed the prototype focusing on the capability of our device to compute the Average Threshold Crossing (ATC) parameter, which is evaluated by counting how many times the sEMG signal crosses a threshold during a fixed time duration (i.e., 130 ms), directly on the wearable device. Exploiting the event-driven characteristic of the ATC, our armband is able to accomplish the on-board prediction of common hand gestures requiring less power w.r.t. state of the art devices. At the end of an acquisition campaign that involved the participation of 26 people, we obtained an average classifier accuracy of 91.9% when aiming to recognize in real time 8 active hand gestures plus the idle state. Furthermore, with 2.92 mA of current absorption during active functioning and 1.34 ms prediction latency, this prototype confirmed our expectations and can be an appealing solution for long-term (up to 60 h) medical and consumer applications.

Multi-day dataset of forearm and wrist electromyogram for hand gesture recognition and biometrics

Surface electromyography (sEMG) signals have been used for advanced prosthetics control, hand-gesture recognition (HGR), and more recently as a novel biometric trait. For these sEMG-based applications, the translation from laboratory research setting to real-life scenarios suffers from two major limitations: (1) a small subject pool, and (2) single-session data recordings, both of which prevents acceptable generalization ability. In this longitudinal database, forearm and wrist sEMG data were collected from 43 participants over three different days with long separation (Days 1, 8, and 29) while they performed static hand/wrist gestures. The objective of this dataset is to provide a comprehensive dataset for the development of robust machine learning algorithms of sEMG, for both HGR and biometric applications. We demonstrated the high quality of the current dataset by comparing with the Ninapro dataset. And we presented its usability for both HGR and biometric applications. Among other applications, the dataset can also be used for developing electrode-shift invariant generalized models, which can further bolster the development of wristband and forearm-bracelet sensors.

Extreme Gradient Boosting for Surface Electromyography Classification on Time-Domain Features

Surface electromyography (sEMG) signals play an essential role in disease diagnosis and rehabilitation. This study applied a powerful machine learning algorithm called extreme gradient boosting (XGBoost) to classify sEMG signals acquired from muscles around the knee for distinguishing patients with knee osteoarthritis (KOA) from healthy subjects. First, to improve data quality, we preprocessed the data via interpolation and normalization. Next, to ensure the description integrity of model input, we extracted nine time-domain features based on the statistical characteristics of sEMG signals over time. Finally, we classified the samples using XGBoost and cross-validation (CV) and compared the results to those produced by the support vector machine (SVM) and the deep neural network (DNN). Experimental results illustrate that the presented method effectively improves classification performance. Moreover, compared with the SVM and the DNN, XGBoost has higher accuracy and better classification performance, which indicates its advantages in the classification of patients with KOA based on sEMG signals.

A Focused Review on the Flexible Wearable Sensors for Sports: From Kinematics to Physiologies

As an important branch of wearable electronics, highly flexible and wearable sensors are gaining huge attention due to their emerging applications. In recent years, the participation of wearable devices in sports has revolutionized the way to capture the kinematical and physiological status of athletes. This review focuses on the rapid development of flexible and wearable sensor technologies for sports. We identify and discuss the indicators that reveal the performance and physical condition of players. The kinematical indicators are mentioned according to the relevant body parts, and the physiological indicators are classified into vital signs and metabolisms. Additionally, the available wearable devices and their significant applications in monitoring these kinematical and physiological parameters are described with emphasis. The potential challenges and prospects for the future developments of wearable sensors in sports are discussed comprehensively. This review paper will assist both athletic individuals and researchers to have a comprehensive glimpse of the wearable techniques applied in different sports.

Low-Cost Wearable Band Sensors of Surface Electromyography for Detecting Hand Movements

Surface electromyography (sEMG) is a non-invasive measure of electrical activity generated due to muscle contraction. In recent years, sEMG signals have been increasingly used in diverse applications such as rehabilitation, pattern recognition, and control of orthotic and prosthetic systems. This study presents the development of a versatile multi-channel sEMG low-cost wearable band system to acquire 4 signals. In this case, the signals acquired with the proposed device have been used to detect hand movements. However, the WyoFlex band could be used in some sections of the arm or the leg if the section's diameter matches the diameter of the WyoFlex band. The designed WyoFlex band was fabricated using three-dimensional (3D) printing techniques employing thermoplastic polyurethane and polylactic acid as manufacturing materials. Then, the proposed wearable electromyographic system (WES) consists of 2 WyoFlex bands, which simultaneously allow the wireless acquisition of 4 sEMG channels of each forearm. The collected sEMG can be visualized and stored for future post-processing stages using a graphical user interface designed in Node-RED. Several experimental tests were conducted to verify the performance of the WES. A dataset with sEMG collected from 15 healthy humans has been obtained as part of the presented results. In addition, a classification algorithm based on artificial neural networks has been implemented to validate the usability of the collected sEMG signals.

A Structured and Methodological Review on Vision-Based Hand Gesture Recognition System

Researchers have recently focused their attention on vision-based hand gesture recognition. However, due to several constraints, achieving an effective vision-driven hand gesture recognition system in real time has remained a challenge. This paper aims to uncover the limitations faced in image acquisition through the use of cameras, image segmentation and tracking, feature extraction, and gesture classification stages of vision-driven hand gesture recognition in various camera orientations. This paper looked at research on vision-based hand gesture recognition systems from 2012 to 2022. Its goal is to find areas that are getting better and those that need more work. We used specific keywords to find 108 articles in well-known online databases. In this article, we put together a collection of the most notable research works related to gesture recognition. We suggest different categories for gesture recognition-related research with subcategories to create a valuable resource in this domain. We summarize and analyze the methodologies in tabular form. After comparing similar types of methodologies in the gesture recognition field, we have drawn conclusions based on our findings. Our research also looked at how well the vision-based system recognized hand gestures in terms of recognition accuracy. There is a wide variation in identification accuracy, from 68% to 97%, with the average being 86.6 percent. The limitations considered comprise multiple text and interpretations of gestures and complex non-rigid hand characteristics. In comparison to current research, this paper is unique in that it discusses all types of gesture recognition techniques.

Dynamic corticomuscular multi-regional modulations during finger movement revealed by time-varying network analysis

OBJECTIVE

A body movement involves the complicated information exchange between the central and peripheral systems, which is characterized by the dynamical coupling patterns between the multiple brain areas and multiple muscle units. How the central and peripheral nerves coordinate multiple internal brain regions and muscle groups is very important when accomplishing the action.

APPROACH

In this study, we extend the adaptive directed transfer function to construct the time-varying networks between multiple corticomuscular regions and divide the movement duration into different stages by the time-varying corticomuscular network patterns.

MAIN RESULTS

The inter dynamical corticomuscular network demonstrated the different interaction patterns between the central and peripheral systems during the different hand movement stages. The muscles transmit bottom-up movement information in the preparation stage, but the brain issues top-down control commands and dominates in the execution stage, and finally, the brain's dominant advantage gradually weakens in the relaxation stage. When classifying the different movement stages based on time-varying corticomuscular network indicators, an average accuracy above 74% could be reliably achieved.

SIGNIFICANCE

The findings of this study help deepen our knowledge of central-peripheral nerve pathways and coordination mechanisms, and also provide opportunities for monitoring and regulating movement disorders.

User-Independent EMG Gesture Recognition Method Based on Adaptive Learning

In a gesture recognition system based on surface electromyogram (sEMG) signals, the recognition model established by existing users cannot directly generalize to the across-user scenarios due to the individual variability of sEMG signals. In this article, we propose an adaptive learning method to handle the problem. The muscle synergy is chosen as the feature vector because it can well-characterize the neural origin of movement. The initial train set is composed of representative samples extracted from the synergy matrix of the existing user. When the new users use the system, the label is obtained by the adaptive K nearest neighbor algorithm (KNN). The recognition process does not require the pre-experiment for new users due to the introduction of adaptive learning strategy, namely, the qualified data and the label of new user data evaluated by a risk evaluator are used to update the train set and KNN weights, so as to adapt to the new users. We have tested the algorithm in DB1 and DB5 of Ninapro databases. The average recognition accuracy is 68.04, 73.35, and 83.05% for different types of gestures, respectively, achieving the effects of the user-dependent method. Our study can avoid the re-training steps and the recognition performance will improve with the increased frequency of uses, which will further facilitate the widespread implementation of sEMG control systems using pattern recognition techniques.

Frequency-Domain sEMG Classification Using a Single Sensor

Working towards the development of robust motion recognition systems for assistive technology control, the widespread approach has been to use a plethora of, often times, multi-modal sensors. In this paper, we develop single-sensor motion recognition systems. Utilising the peripheral nature of surface electromyography (sEMG) data acquisition, we optimise the information extracted from sEMG sensors. This allows the reduction in sEMG sensors or provision of contingencies in a system with redundancies. In particular, we process the sEMG readings captured at the trapezius descendens and platysma muscles. We demonstrate that sEMG readings captured at one muscle contain distinct information on movements or contractions of other agonists. We used the trapezius and platysma muscle sEMG data captured in able-bodied participants and participants with tetraplegia to classify shoulder movements and platysma contractions using white-box supervised learning algorithms. Using the trapezius sensor, shoulder raise is classified with an accuracy of 99%. Implementing subject-specific multi-class classification, shoulder raise, shoulder forward and shoulder backward are classified with a 94% accuracy amongst object raise and shoulder raise-and-hold data in able bodied adults. A three-way classification of the platysma sensor data captured with participants with tetraplegia achieves a 95% accuracy on platysma contraction and shoulder raise detection.

A Way of Bionic Control Based on EI, EMG, and FMG Signals

Creating highly functional prosthetic, orthotic, and rehabilitation devices is a socially relevant scientific and engineering task. Currently, certain constraints hamper the development of such devices. The primary constraint is the lack of an intuitive and reliable control interface working between the organism and the actuator. The critical point in developing these devices and systems is determining the type and parameters of movements based on control signals recorded on an extremity. In the study, we investigate the simultaneous acquisition of electric impedance (EI), electromyography (EMG), and force myography (FMG) signals during basic wrist movements: grasping, flexion/extension, and rotation. For investigation, a laboratory instrumentation and software test setup were made for registering signals and collecting data. The analysis of the acquired signals revealed that the EI signals in conjunction with the analysis of EMG and FMG signals could potentially be highly informative in anthropomorphic control systems. The study results confirm that the comprehensive real-time analysis of EI, EMG, and FMG signals potentially allows implementing the method of anthropomorphic and proportional control with an acceptable delay.

Optimization of Data Quality Related EMG Feature Extraction Parameters to Increase Hand Movement Classification Accuracy

Many biomedical robotic interfaces (e.g., prostheses, exoskeletons) classify or estimate user movement intent based on features extracted from measured electromyograms (EMG). In most cases, the parameters of feature extraction are determined heuristically or assigned arbitrary values. We propose a more rigorous method, numerical optimization, to systematically identify parameters that maximize classification accuracy based on EMG signal characteristics. In this study, we used simulated annealing, a common global numerical optimization method, to find the optimal values of three feature extraction parameters based on the root mean square (rms) magnitude of the EMG signal. The EMG data, obtained from a public database, had been measured from 2 muscles (one hand flexor and one hand extensor) of 5 able-bodied participants performing 6 different movement tasks. Using optimization, we increased the offline movement classification accuracy by 3-5% for each participant and from 79.91% to 92.25% overall. The value of one optimized parameter (threshold of Wilson amplitude) was strongly correlated with the rms magnitude of the EMG signal (R²=0.81). Other parameters were suspected to be related to signal noise, since no strong correlation with rms magnitude was observed. Future studies will refine the optimization approach and test its practicality and effectiveness for improving online classification accuracy with robotic interfaces.

Verification-Based Design of a Robust EMG Wake Word

Surface electromyography (sEMG) signals are now commonly used in continuous myoelectric control of prostheses. More recently, researchers have considered EMG-based gesture recognition systems for human computer interaction research. These systems instead focus on recognizing discrete gestures (like a finger snap). The majority of works, however, have focused on improving multi-class performance, with little consideration for false activations from "other" classes. Consequently, they lack the robustness needed for real-world applications which generally require a single motion class such as a mouse click or a wake word. Furthermore, many works have borrowed the windowed classification schemes from continuous control, and thus fail to leverage the temporal structure of the gesture. In this paper, we propose a verification-based approach to creating a robust EMG wake word using one-class classifiers (Support Vector Data Description, One Class-Support Vector Machine, Dynamic Time Warping (DTW) & Hidden Markov Models). The area under the ROC curve (AUC) is used as a feature optimization objective as it provides a better representation of the verification performance. Equal error rate (EER) and AUC are then used as evaluation metrics. The results are computed using both window-based and temporal classifiers on a dataset consisting of five different gestures, with a best EER of 0.04 and AUC of 0.98, recorded using a DTW scheme. These results demonstrate a design framework that may benefit the development of more robust solutions for EMG-based wake words or input commands for a variety of interactive applications.

Inter-classifier comparison for upper extremity EMG signal at different hand postures and arm positions using pattern recognition

The utilization of surface EMG and intramuscular EMG signals has been observed to create significant improvement in pattern recognition approaches and myoelectric control. However, there is less data of different arm positions and hand postures available. Hand postures and arm positions tend to affect the combination of surface and intramuscular EMG signal acquisition in terms of classifier accuracy. Hence, this study aimed to find a robust classifier for two scenarios: (1) at fixed arm position (FAP) where classifiers classify different hand postures and (2) at fixed hand posture (FHP) where classifiers classify different arm positions. A total of 20 healthy male participants (30.62 ± 3.87 years old) were recruited for this study. They were asked to perform five motion classes including hand grasp, hand open, rest, hand extension, and hand flexion at four different arm positions at 0°, 45°, 90°, and 135°. SVM, KNN, and LDA classifier were deployed. Statistical analysis in the form of pairwise comparisons was carried out using SPSS. It is concluded that there is no significant difference among the three classifiers. SVM gave highest accuracy of 75.35% and 58.32% at FAP and FHP respectively for each motion classification. KNN yielded the highest accuracies of 69.11% and 79.04% when data was pooled and was classified at different arm positions and at different hand postures respectively. The results exhibited that there is no significant effect of changing arm position and hand posture on the classifier accuracy.

FS-HGR: Few-Shot Learning for Hand Gesture Recognition via Electromyography

This work is motivated by the recent advances in Deep Neural Networks (DNNs) and their widespread applications in human-machine interfaces. DNNs have been recently used for detecting the intended hand gesture through the processing of surface electromyogram (sEMG) signals. Objective: Although DNNs have shown superior accuracy compared to conventional methods when large amounts of data are available for training, their performance substantially decreases when data are limited. Collecting large datasets for training may be feasible in research laboratories, but it is not a practical approach for real-life applications. The main objective of this work is to design a modern DNN-based gesture detection model that relies on minimal training data while providing high accuracy. Methods: We propose the novel Few-Shot learning- Hand Gesture Recognition (FS-HGR) architecture. Few-shot learning is a variant of domain adaptation with the goal of inferring the required output based on just one or a few training observations. The proposed FS-HGR generalizes after seeing very few observations from each class by combining temporal convolutions with attention mechanisms. This allows the meta-learner to aggregate contextual information from experience and to pinpoint specific pieces of information within its available set of inputs. Data Source & Summary of Results: The performance of FS-HGR was tested on the second and fifth Ninapro databases, referred to as the DB2 and DB5, respectively. The DB2 consists of 50 gestures (rest included) from 40 healthy subjects. The Ninapro DB5 contains data from 10 healthy participants performing a total of 53 different gestures (rest included). The proposed approach for the Ninapro DB2 led to 85.94% classification accuracy on new repetitions with few-shot observation (5-way 5-shot), 81.29% accuracy on new subjects with few-shot observation (5-way 5-shot), and 73.36% accuracy on new gestures with few-shot observation (5-way 5-shot). Moreover, the proposed approach for the Ninapro DB5 led to 64.65% classification accuracy on new subjects with few-shot observation (5-way 5-shot).

Smart healthcare solutions using the internet of medical things for hand gesture recognition system

Patient gesture recognition is a promising method to gain knowledge and assist patients. Healthcare monitoring systems integrated with the Internet of Things (IoT) paradigm to perform the remote solutions for the acquiring inputs. In recent years, wearable sensors, and information and communication technologies are assisting for remote monitoring and recommendations in smart healthcare. In this paper, the dependable gesture recognition (DGR) using a series learning method for identifying the action of patient monitoring through remote access is presented. The gesture recognition systems connect to the end-user (remote) and the patient for instantaneous gesture identification. The gesture is recognized by the analysis of the intermediate and structuring features using series learning. The proposed gesture recognition system is capable of monitoring patient activities and differentiating the gestures from the regular actions to improve the convergence. Gesture recognition through remote monitoring is indistinguishable due to the preliminary errors. Further, it is convertible using series learning. Therefore, the misdetections and classifications are promptly identified using the DGR and verified by comparative analysis and experimental study. From the analysis, the proposed DGR approach attains 94.92% high precision for the varying gestures and 89.85% high accuracy for varying mess factor. The proposed DGR reduces recognition time to 4.97 s and 4.93 s for the varying gestures and mess factor, respectively.

A PILOT STUDY OF MECHANOMYOGRAPHY-BASED HAND MOVEMENTS RECOGNITION EMPHASIZING ON THE INFLUENCE OF FABRICS BETWEEN SENSOR AND SKIN

Multi-channel mechanomyography (MMG) signals were acquired from the forearm when the subjects were performing eight classes of hand movements related to rehabilitation training. Ten time domain (TD) features and wavelet packet node energy (WPNE) features were extracted from each channel of MMG, and the hand movements were classified by support vector machine (SVM), extreme learning machine (ELM), linear discriminant analysis (LDA) and [Formula: see text]-nearest neighborhood (KNN) and the classifying results of three methods of collecting MMG (sensors directly on skin, sensors on cotton fabric and sensors on acrylic fiber) were compared. When all TD features were selected and SVM was adopted as the classifier, the total recognition rates of hand movements were 94.0%, 93.9% and 93.6%, respectively, of three collection methods. Using ELM can obtain similar results as SVM, with the recognition rates of 94.3%, 94.3% and 94.1%, respectively, better than using LDA (88.5%, 88.6% and 88.0%) or KNN (88.9%, 89.4% and 89.0%). For each algorithm, using TD features can acquire the highest recognition rates. Once the feature set and the classifier were selected, the total recognition rates were almost equally among three collection methods (especially for some feature sets, the differences are smaller than 1%). The results confirmed that satisfactory effects could be acquired even when the MMG was collected from sensors on fabrics with specific material, thus indicating that MMG has a unique potential value for developing wearable devices.

HGM-4: A new multi-cameras dataset for hand gesture recognition

Gesture recognition technology is rapidly growing in the recent years due to the demands of many application such as computer game and sport, human robot interaction, assistant systems, sign language interpretation and e-commerce. One of the most important of gesture recognition is hand-gesture recognition. For example, it can be used to control all devices (television, radio, air-condition, and doors) by just hand gestures for smart home application. The HGM-4 dataset is built for hand gesture recognition (the full dataset is available from: https://data.mendeley.com/datasets/jzy8zngkbg/4) which contains total 4,160 color images (1280 × 700 pixels) of 26 hand gestures captured by four cameras at different position. The training and testing set are defined to create a benchmark framework for comparing the experimental results.

Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review

Today, daily life is composed of many computing systems, therefore interacting with them in a natural way makes the communication process more comfortable. Human-Computer Interaction (HCI) has been developed to overcome the communication barriers between humans and computers. One form of HCI is Hand Gesture Recognition (HGR), which predicts the class and the instant of execution of a given movement of the hand. One possible input for these models is surface electromyography (EMG), which records the electrical activity of skeletal muscles. EMG signals contain information about the intention of movement generated by the human brain. This systematic literature review analyses the state-of-the-art of real-time hand gesture recognition models using EMG data and machine learning. We selected and assessed 65 primary studies following the Kitchenham methodology. Based on a common structure of machine learning-based systems, we analyzed the structure of the proposed models and standardized concepts in regard to the types of models, data acquisition, segmentation, preprocessing, feature extraction, classification, postprocessing, real-time processing, types of gestures, and evaluation metrics. Finally, we also identified trends and gaps that could open new directions of work for future research in the area of gesture recognition using EMG.

A Smart Terrain Identification Technique Based on Electromyography, Ground Reaction Force, and Machine Learning for Lower Limb Rehabilitation

Automatic terrain classification in lower limb rehabilitation systems has gained worldwide attention. In this field, a simple system architecture and high classification accuracy are two desired attributes. In this article, a smart neuromuscular–mechanical fusion and machine learning-based terrain classification technique utilizing only two electromyography (EMG) sensors and two ground reaction force (GRF) sensors is reported for classifying three different terrains (downhill, level, and uphill). The EMG and GRF signals from ten healthy subjects were collected, preprocessed and segmented to obtain the EMG and GRF profiles in each stride, based on which twenty-one statistical features, including 9 GRF features and 12 EMG features, were extracted. A support vector machine (SVM) machine learning model is established and trained by the extracted EMG features, GRF features and the fusion of them, respectively. Several methods or statistical metrics were used to evaluate the goodness of the proposed technique, including a paired-t-test and Kruskal–Wallis test for correlation analysis of the selected features and ten-fold cross-validation accuracy, confusion matrix, sensitivity and specificity for the performance of the SVM model. The results show that the extracted features are highly correlated with the terrain changes and the fusion of the EMG and GRF features produces the highest accuracy of 96.8%. The presented technique allows simple system construction to achieve the precise detection of outcomes, potentially advancing the development of terrain classification techniques for rehabilitation.

Fully Untethered Battery-free Biomonitoring Electronic Tattoo with Wireless Energy Harvesting

Bioelectronics stickers that interface the human epidermis and collect electrophysiological data will constitute important tools in the future of healthcare. Rapid progress is enabled by novel fabrication methods for adhesive electronics patches that are soft, stretchable and conform to the human skin. Yet, the ultimate functionality of such systems still depends on rigid components such as silicon chips and the largest rigid component on these systems is usually the battery. In this work, we demonstrate a quickly deployable, untethered, battery-free, ultrathin (~5 μm) passive "electronic tattoo" that interfaces with the human skin for acquisition and transmission of physiological data. We show that the ultrathin film adapts well with the human skin, and allows an excellent signal to noise ratio, better than the gold-standard Ag/AgCl electrodes. To supply the required energy, we rely on a wireless power transfer (WPT) system, using a printed stretchable Ag-In-Ga coil, as well as printed biopotential acquisition electrodes. The tag is interfaced with data acquisition and communication electronics. This constitutes a "data-by-request" system. By approaching the scanning device to the applied tattoo, the patient's electrophysiological data is read and stored to the caregiver device. The WPT device can provide more than 300 mW of measured power if it is transferred over the skin or 100 mW if it is implanted under the skin. As a case study, we transferred this temporary tattoo to the human skin and interfaced it with an electrocardiogram (ECG) device, which could send the volunteer's heartbeat rate in real-time via Bluetooth.

Energy per Operation Optimization for Energy-Harvesting Wearable IoT Devices

Wearable internet of things (IoT) devices can enable a variety of biomedical applications,such as gesture recognition, health monitoring, and human activity tracking. Size and weightconstraints limit the battery capacity, which leads to frequent charging requirements and userdissatisfaction. Minimizing the energy consumption not only alleviates this problem, but alsopaves the way for self-powered devices that operate on harvested energy. This paper considers anenergy-optimal gesture recognition application that runs on energy-harvesting devices. We firstformulate an optimization problem for maximizing the number of recognized gestures when energybudget and accuracy constraints are given. Next, we derive an analytical energy model from thepower consumption measurements using a wearable IoT device prototype. Then, we prove thatmaximizing the number of recognized gestures is equivalent to minimizing the duration of gesturerecognition. Finally, we utilize this result to construct an optimization technique that maximizes thenumber of gestures recognized under the energy budget constraints while satisfying the recognitionaccuracy requirements. Our extensive evaluations demonstrate that the proposed analytical modelis valid for wearable IoT applications, and the optimization approach increases the number ofrecognized gestures by up to 2.4× compared to a manual optimization.

Support Vector Machine-Based EMG Signal Classification Techniques: A Review

This paper gives an overview of the different research works related to electromyographic signals (EMG) classification based on Support Vector Machines (SVM). The article summarizes the techniques used to make the classification in each reference. Furthermore, it includes the obtained accuracy, the number of signals or channels used, the way the authors made the feature vector, and the type of kernels used. Hence, this article also includes a compilation about the bands used to filter signals, the number of signals recommended, the most commonly used sampling frequencies, and certain features that can create the characteristics of the vector. This research gathers articles related to different kinds of SVM-based classification and other tools for signal processing in the field.

Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network

In recent years, surface electromyography (sEMG) signals have been increasingly used in pattern recognition and rehabilitation. In this paper, a real-time hand gesture recognition model using sEMG is proposed. We use an armband to acquire sEMG signals and apply a sliding window approach to segment the data in extracting features. A feedforward artificial neural network (ANN) is founded and trained by the training dataset. A test method is used in which the gesture will be recognized when recognized label times reach the threshold of activation times by the ANN classifier. In the experiment, we collected real sEMG data from twelve subjects and used a set of five gestures from each subject to evaluate our model, with an average recognition rate of 98.7% and an average response time of 227.76 ms, which is only one-third of the gesture time. Therefore, the pattern recognition system might be able to recognize a gesture before the gesture is completed.

Gyroscope-Based Continuous Human Hand Gesture Recognition for Multi-Modal Wearable Input Device for Human Machine Interaction

Human hand gestures are a widely accepted form of real-time input for devices providing a human-machine interface. However, hand gestures have limitations in terms of effectively conveying the complexity and diversity of human intentions. This study attempted to address these limitations by proposing a multi-modal input device, based on the observation that each application program requires different user intentions (and demanding functions) and the machine already acknowledges the running application. When the running application changes, the same gesture now offers a new function required in the new application, and thus, we can greatly reduce the number and complexity of required hand gestures. As a simple wearable sensor, we employ one miniature wireless three-axis gyroscope, the data of which are processed by correlation analysis with normalized covariance for continuous gesture recognition. Recognition accuracy is improved by considering both gesture patterns and signal strength and by incorporating a learning mode. In our system, six unit hand gestures successfully provide most functions offered by multiple input devices. The characteristics of our approach are automatically adjusted by acknowledging the application programs or learning user preferences. In three application programs, the approach shows good accuracy (90–96%), which is very promising in terms of designing a unified solution. Furthermore, the accuracy reaches 100% as the users become more familiar with the system.

Wearable Hardware Design for the Internet of Medical Things (IoMT)

As the life expectancy of individuals increases with recent advancements in medicine and quality of living, it is important to monitor the health of patients and healthy individuals on a daily basis. This is not possible with the current health care system in North America, and thus there is a need for wireless devices that can be used from home. These devices are called biomedical wearables, and they have become popular in the last decade. There are several reasons for that, but the main ones are: expensive health care, longer wait times, and an increase in public awareness about improving quality of life. With this, it is vital for anyone working on wearables to have an overall understanding of how they function, how they were designed, their significance, and what factors were considered when the hardware was designed. Therefore, this study attempts to investigate the hardware components that are required to design wearable devices that are used in the emerging context of the Internet of Medical Things (IoMT). This means that they can be used, to an extent, for disease monitoring through biosignal capture. In particular, this review study covers the basic components that are required for the front-end of any biomedical wearable, and the limitations that these wearable devices have. Furthermore, there is a discussion of the opportunities that they create, and the direction that the wearable industry is heading in.