Repeatability of grasp recognition for robotic hand prosthesis control based on sEMG data

Advancements in Textile-Based sEMG Sensors for Muscle Fatigue Detection: A Journey from Material Evolution to Technological Integration

Textile-based surface electromyography (sEMG) electrodes have emerged as a prominent tool in muscle fatigue assessment, marking a significant shift toward innovative, noninvasive methods. This review examines the transition from metallic fibers to novel conductive polymers, elastomers, and advanced material-based electrodes, reflecting on the rapid evolution of materials in sEMG sensor technology. It highlights the pivotal role of materials science in enhancing sensor adaptability, signal accuracy, and longevity, crucial for practical applications in health monitoring, while examining the balance of clinical precision with user comfort. Additionally, it maps the global sEMG research landscape of diverse regional contributors and their impact on technological progress, focusing on the integration of Eastern manufacturing prowess with Western technological innovations and exploring both the opportunities and challenges in this global synergy. The integration of such textile-based sEMG innovations with artificial intelligence, nanotechnology, energy harvesting, and IoT connectivity is also anticipated as future prospects. Such advancements are poised to revolutionize personalized preventive healthcare. As the exploration of textile-based sEMG electrodes continues, the transformative potential not only promises to revolutionize integrated wellness and preventive healthcare but also signifies a seamless transition from laboratory innovations to real-world applications in sports medicine, envisioning the future of truly wearable material technologies.

A Multimodal Bracelet to Acquire Muscular Activity and Gyroscopic Data to Study Sensor Fusion for Intent Detection

Researchers have attempted to control robotic hands and prostheses through biosignals but could not match the human hand. Surface electromyography records electrical muscle activity using non-invasive electrodes and has been the primary method in most studies. While surface electromyography-based hand motion decoding shows promise, it has not yet met the requirements for reliable use. Combining different sensing modalities has been shown to improve hand gesture classification accuracy. This work introduces a multimodal bracelet that integrates a 24-channel force myography system with six commercial surface electromyography sensors, each containing a six-axis inertial measurement unit. The device's functionality was tested by acquiring muscular activity with the proposed device from five participants performing five different gestures in a random order. A random forest model was then used to classify the performed gestures from the acquired signal. The results confirmed the device's functionality, making it suitable to study sensor fusion for intent detection in future studies. The results showed that combining all modalities yielded the highest classification accuracies across all participants, reaching 92.3±2.6% on average, effectively reducing misclassifications by 37% and 22% compared to using surface electromyography and force myography individually as input signals, respectively. This demonstrates the potential benefits of sensor fusion for more robust and accurate hand gesture classification and paves the way for advanced control of robotic and prosthetic hands.

Effects of Training and Calibration Data on Surface Electromyogram-Based Recognition for Upper Limb Amputees

Surface electromyogram (sEMG)-based gesture recognition has emerged as a promising avenue for developing intelligent prostheses for upper limb amputees. However, the temporal variations in sEMG have rendered recognition models less efficient than anticipated. By using cross-session calibration and increasing the amount of training data, it is possible to reduce these variations. The impact of varying the amount of calibration and training data on gesture recognition performance for amputees is still unknown. To assess these effects, we present four datasets for the evaluation of calibration data and examine the impact of the amount of training data on benchmark performance. Two amputees who had undergone amputations years prior were recruited, and seven sessions of data were collected for analysis from each of them. Ninapro DB6, a publicly available database containing data from ten healthy subjects across ten sessions, was also included in this study. The experimental results show that the calibration data improved the average accuracy by 3.03%, 6.16%, and 9.73% for the two subjects and Ninapro DB6, respectively, compared to the baseline results. Moreover, it was discovered that increasing the number of training sessions was more effective in improving accuracy than increasing the number of trials. Three potential strategies are proposed in light of these findings to enhance cross-session models further. We consider these findings to be of the utmost importance for the commercialization of intelligent prostheses, as they demonstrate the criticality of gathering calibration and cross-session training data, while also offering effective strategies to maximize the utilization of the entire dataset.

Mid-Activity and At-Home Wearable Bioimpedance Elucidates an Interpretable Digital Biomarker of Muscle Fatigue

OBJECTIVE

Muscle health and decreased muscle performance (fatigue) quantification has proven to be an invaluable tool for both athletic performance assessment and injury prevention. However, existing methods estimating muscle fatigue are infeasible for everyday use. Wearable technologies are feasible for everyday use and can enable discovery of digital biomarkers of muscle fatigue. Unfortunately, the current state-of-the-art wearable systems for muscle fatigue tracking suffer from either low specificity or poor usability.

METHODS

We propose using dual-frequency bioimpedance analysis (DFBIA) to non-invasively assess intramuscular fluid dynamics and thereby muscle fatigue. A wearable DFBIA system was developed to measure leg muscle fatigue of 11 individuals during a 13-day protocol consisting of exercise and unsupervised at-home portions.

RESULTS

We derived a digital biomarker of muscle fatigue, fatigue score, from the DFBIA signals that was able to estimate the percent reduction in muscle force during exercise with repeated-measures Pearson's r = 0.90 and mean absolute error (MAE) of 3.6%. This fatigue score also estimated delayed onset muscle soreness with repeated-measures Pearson's r = 0.83 and MAE = 0.83. Using at-home data, DFBIA was strongly associated with absolute muscle force of participants (n = 198, p < 0.001).

CONCLUSION

These results demonstrate the utility of wearable DFBIA for non-invasively estimating muscle force and pain through the changes in intramuscular fluid dynamics.

SIGNIFICANCE

The presented approach may inform development of future wearable systems for quantifying muscle health and provide a novel framework for athletic performance optimization and injury prevention.

Alignment-Enhanced Interactive Fusion Model for Complete and Incomplete Multimodal Hand Gesture Recognition

Hand gesture recognition (HGR) based on surface electromyogram (sEMG) and Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for performance and remain challenging. Currently, neural network-based fusion methods have gained superior performance. Nevertheless, these methods typically fuse sEMG and ACC either in the early or late stages, overlooking the integration of entire cross-modal hierarchical information within each individual hidden layer, thus inducing inefficient inter-modal fusion. To this end, we propose a novel Alignment-Enhanced Interactive Fusion (AiFusion) model, which achieves effective fusion via a progressive hierarchical fusion strategy. Notably, AiFusion can flexibly perform both complete and incomplete multimodal HGR. Specifically, AiFusion contains two unimodal branches and a cascaded transformer-based multimodal fusion branch. The fusion branch is first designed to adequately characterize modality-interactive knowledge by adaptively capturing inter-modal similarity and fusing hierarchical features from all branches layer by layer. Then, the modality-interactive knowledge is aligned with that of unimodality using cross-modal supervised contrastive learning and online distillation from embedding and probability spaces respectively. These alignments further promote fusion quality and refine modality-specific representations. Finally, the recognition outcomes are set to be determined by available modalities, thus contributing to handling the incomplete multimodal HGR problem, which is frequently encountered in real-world scenarios. Experimental results on five public datasets demonstrate that AiFusion outperforms most state-of-the-art benchmarks in complete multimodal HGR. Impressively, it also surpasses the unimodal baselines in the challenging incomplete multimodal HGR. The proposed AiFusion provides a promising solution to realize effective and robust multimodal HGR-based interfaces.

Improved Network and Training Scheme for Cross-Trial Surface Electromyography (sEMG)-Based Gesture Recognition

To enhance the performance of surface electromyography (sEMG)-based gesture recognition, we propose a novel network-agnostic two-stage training scheme, called sEMGPoseMIM, that produces trial-invariant representations to be aligned with corresponding hand movements via cross-modal knowledge distillation. In the first stage, an sEMG encoder is trained via cross-trial mutual information maximization using the sEMG sequences sampled from the same time step but different trials in a contrastive learning manner. In the second stage, the learned sEMG encoder is fine-tuned with the supervision of gesture and hand movements in a knowledge-distillation manner. In addition, we propose a novel network called sEMGXCM as the sEMG encoder. Comprehensive experiments on seven sparse multichannel sEMG databases are conducted to demonstrate the effectiveness of the training scheme sEMGPoseMIM and the network sEMGXCM, which achieves an average improvement of +1.3% on the sparse multichannel sEMG databases compared to the existing methods. Furthermore, the comparison between training sEMGXCM and other existing networks from scratch shows that sEMGXCM outperforms the others by an average of +1.5%.

Reducing the Energy Consumption of sEMG-Based Gesture Recognition at the Edge Using Transformers and Dynamic Inference

Hand gesture recognition applications based on surface electromiographic (sEMG) signals can benefit from on-device execution to achieve faster and more predictable response times and higher energy efficiency. However, deploying state-of-the-art deep learning (DL) models for this task on memory-constrained and battery-operated edge devices, such as wearables, requires a careful optimization process, both at design time, with an appropriate tuning of the DL models' architectures, and at execution time, where the execution of large and computationally complex models should be avoided unless strictly needed. In this work, we pursue both optimization targets, proposing a novel gesture recognition system that improves upon the state-of-the-art models both in terms of accuracy and efficiency. At the level of DL model architecture, we apply for the first time tiny transformer models (which we call bioformers) to sEMG-based gesture recognition. Through an extensive architecture exploration, we show that our most accurate bioformer achieves a higher classification accuracy on the popular Non-Invasive Adaptive hand Prosthetics Database 6 (Ninapro DB6) dataset compared to the state-of-the-art convolutional neural network (CNN) TEMPONet (+3.1%). When deployed on the RISC-V-based low-power system-on-chip (SoC) GAP8, bioformers that outperform TEMPONet in accuracy consume 7.8×-44.5× less energy per inference. At runtime, we propose a three-level dynamic inference approach that combines a shallow classifier, i.e., a random forest (RF) implementing a simple "rest detector" with two bioformers of different accuracy and complexity, which are sequentially applied to each new input, stopping the classification early for "easy" data. With this mechanism, we obtain a flexible inference system, capable of working in many different operating points in terms of accuracy and average energy consumption. On GAP8, we obtain a further 1.03×-1.35× energy reduction compared to static bioformers at iso-accuracy.

Deep transfer learning compared to subject-specific models for sEMG decoders

Objective. Accurate decoding of surface electromyography (sEMG) is pivotal for muscle-to-machine-interfaces and their application e.g. rehabilitation therapy. sEMG signals have high inter-subject variability, due to various factors, including skin thickness, body fat percentage, and electrode placement. Deep learning algorithms require long training time and tend to overfit if only few samples are available. In this study, we aim to investigate methods to calibrate deep learning models to a new user when only a limited amount of training data is available.Approach. Two methods are commonly used in the literature, subject-specific modeling and transfer learning. In this study, we investigate the effectiveness of transfer learning using weight initialization for recalibration of two different pretrained deep learning models on new subjects data and compare their performance to subject-specific models. We evaluate two models on three publicly available databases (non invasive adaptive prosthetics database 2-4) and compare the performance of both calibration schemes in terms of accuracy, required training data, and calibration time.Main results. On average over all settings, our transfer learning approach improves 5%-points on the pretrained models without fine-tuning, and 12%-points on the subject-specific models, while being trained for 22% fewer epochs on average. Our results indicate that transfer learning enables faster learning on fewer training samples than user-specific models.Significance. To the best of our knowledge, this is the first comparison of subject-specific modeling and transfer learning. These approaches are ubiquitously used in the field of sEMG decoding. But the lack of comparative studies until now made it difficult for scientists to assess appropriate calibration schemes. Our results guide engineers evaluating similar use cases.

Can You Do That Again? Time Series Consolidation as a Robust Method of Tailoring Gesture Recognition to Individual Users

Robust inter-session modeling of gestures is still an open learning challenge. A sleeve equipped with capacitive strap sensors was used to capture two gesture data sets from a convenience sample of eight subjects. Two pipelines were explored. In FILT a novel two-stage algorithm was introduced which uses an unsupervised learning algorithm to find samples representing gesture transitions and discards them prior to training and validating conventional models. In TSC a confusion matrix was used to automatically consolidate commonly confused class labels, resulting in a set of gestures tailored to an individual subject's abilities. The inter-session testing accuracy using the Time Series Consolidation (TSC) method increased from a baseline inter-session average of 42.47 ± 3.83% to 93.02% ± 4.97% while retaining an average of 5.29 ± 0.46 out of the 11 possible gesture categories. These pipelines used classic machine learning algorithms which require relatively small amounts of data and computational power compared to deep learning solutions. These methods may also offer more flexibility in interface design for users suffering from handicaps limiting their manual dexterity or ability to reliably make gestures, and be possible to implement on edge devices with low computational power.

Adaptive Lower Limb Pattern Recognition for Multi-Day Control

Pattern recognition in EMG-based control systems suffer from increase in error rate over time, which could lead to unwanted behavior. This so-called concept drift in myoelectric control systems could be caused by fatigue, sensor replacement and varying skin conditions. To circumvent concept drift, adaptation strategies could be used to retrain a pattern recognition system, which could lead to comparable error rates over multiple days. In this study, we investigated the error rate development over one week and compared three adaptation strategies to reduce the error rate increase. The three adaptation strategies were based on entropy, on backward prediction and a combination of backward prediction and entropy. Ten able-bodied subjects were measured on four measurement days while performing gait-related activities. During the measurement electromyography and kinematics were recorded. The three adaptation strategies were implemented and compared against the baseline error rate and against adaptation using the ground truth labels. It can be concluded that without adaptation the baseline error rate increases significantly from day 1 to 2, but plateaus on day 2, 3 and 7. Of the three tested adaptation strategies, entropy based adaptation showed the smallest increase in error rate over time. It can be concluded that entropy based adaptation is simple to implement and can be considered a feasible adaptation strategy for lower limb pattern recognition.

Age-dependent Upper Limb Myoelectric Control Capability in Typically Developing Children

Research in EMG-based control of prostheses has mainly utilized adult subjects who have fully developed neuromuscular control. Little is known about children's ability to generate consistent EMG signals necessary to control artificial limbs with multiple degrees of freedom. As a first step to address this gap, experiments were designed to validate and benchmark two experimental protocols that quantify the ability to coordinate forearm muscle contractions in typically developing children. Able-bodied, healthy adults and children participated in our experiments that aimed to measure an individual's ability to use myoelectric control interfaces. In the first experiment, participants performed 8 repetitions of 16 different hand/wrist movements. Using offline classification analysis based on Support Vector Machine, we quantified their ability to consistently produce distinguishable muscle contraction patterns. We demonstrated that children had a smaller number of highly independent movements (can be classified with >90% accuracy) than adults did. The second experiment measured participants' ability to control the position of a cursor on a 1-DoF virtual slide using proportional EMG control with three different visuomotor gain levels. We found that children had higher failure rates and slower average target acquisitions than adults did, primarily due to longer correction times that did not improve over repetitive practice. We also found that the performance in both experiments was age-dependent in children. The results of this study provide novel insights into the technical and empirical basis to better understand neuromuscular development in children with upper-limb loss.

Smoking Cessation System for Preemptive Smoking Detection

Smoking cessation is a significant challenge for many people addicted to cigarettes and tobacco. Mobile health-related research into smoking cessation is primarily focused on mobile phone data collection either using self-reporting or sensor monitoring techniques. In the past 5 years with the increased popularity of smartwatch devices, research has been conducted to predict smoking movements associated with smoking behaviors based on accelerometer data analyzed from the internal sensors in a user's smartwatch. Previous smoking detection methods focused on classifying current user smoking behavior. For many users who are trying to quit smoking, this form of detection may be insufficient as the user has already relapsed. In this paper, we present a smoking cessation system utilizing a smartwatch and finger sensor that is capable of detecting pre-smoking activities to discourage users from future smoking behavior. Pre-smoking activities include grabbing a pack of cigarettes or lighting a cigarette and these activities are often immediately succeeded by smoking. Therefore, through accurate detection of pre-smoking activities, we can alert the user before they have relapsed. Our smoking cessation system combines data from a smartwatch for gross accelerometer and gyroscope information and a wearable finger sensor for detailed finger bend-angle information. We compare the results of a smartwatch-only system with a combined smartwatch and finger sensor system to illustrate the accuracy of each system. The combined smartwatch and finger sensor system performed at an 80.6% accuracy for the classification of pre-smoking activities compared to 47.0% accuracy of the smartwatch-only system.

A Hybrid 3D Printed Hand Prosthesis Prototype Based on sEMG and a Fully Embedded Computer Vision System

This study presents a new approach for an sEMG hand prosthesis based on a 3D printed model with a fully embedded computer vision (CV) system in a hybrid version. A modified 5-layer Smaller Visual Geometry Group (VGG) convolutional neural network (CNN), running on a Raspberry Pi 3 microcomputer connected to a webcam, recognizes the shape of daily use objects, and defines the pattern of the prosthetic grasp/gesture among five classes: Palmar Neutral, Palmar Pronated, Tripod Pinch, Key Grasp, and Index Finger Extension. Using the Myoware board and a finite state machine, the user's intention, depicted by a myoelectric signal, starts the process, photographing the object, proceeding to the grasp/gesture classification, and commands the prosthetic motors to execute the movements. Keras software was used as an application programming interface and TensorFlow as numerical computing software. The proposed system obtained 99% accuracy, 97% sensitivity, and 99% specificity, showing that the CV system is a promising technology to assist the definition of the grasp pattern in prosthetic devices.

Questioning Domain Adaptation in Myoelectric Hand Prostheses Control: An Inter- and Intra-Subject Study

One major challenge limiting the use of dexterous robotic hand prostheses controlled via electromyography and pattern recognition relates to the important efforts required to train complex models from scratch. To overcome this problem, several studies in recent years proposed to use transfer learning, combining pre-trained models (obtained from prior subjects) with training sessions performed on a specific user. Although a few promising results were reported in the past, it was recently shown that the use of conventional transfer learning algorithms does not increase performance if proper hyperparameter optimization is performed on the standard approach that does not exploit transfer learning. The objective of this paper is to introduce novel analyses on this topic by using a random forest classifier without hyperparameter optimization and to extend them with experiments performed on data recorded from the same patient, but in different data acquisition sessions. Two domain adaptation techniques were tested on the random forest classifier, allowing us to conduct experiments on healthy subjects and amputees. Differently from several previous papers, our results show that there are no appreciable improvements in terms of accuracy, regardless of the transfer learning techniques tested. The lack of adaptive learning is also demonstrated for the first time in an intra-subject experimental setting when using as a source ten data acquisitions recorded from the same subject but on five different days.

Action Recognition of Lower Limbs Based on Surface Electromyography Weighted Feature Method

To improve the recognition rate of lower limb actions based on surface electromyography (sEMG), an effective weighted feature method is proposed, and an improved genetic algorithm support vector machine (IGA-SVM) is designed in this paper. First, for the problem of high feature redundancy and low discrimination in the surface electromyography feature extraction process, the weighted feature method is proposed based on the correlation between muscles and actions. Second, to solve the problem of the genetic algorithm selection operator easily falling into a local optimum solution, the improved genetic algorithm-support vector machine is designed by championship with sorting method. Finally, the proposed method is used to recognize six types of lower limb actions designed, and the average recognition rate reaches 94.75%. Experimental results indicate that the proposed method has definite potentiality in lower limb action recognition.

Inkjet-printed fully customizable and low-cost electrodes matrix for gesture recognition

The use of surface electromyography (sEMG) is rapidly spreading, from robotic prostheses and muscle computer interfaces to rehabilitation devices controlled by residual muscular activities. In this context, sEMG-based gesture recognition plays an enabling role in controlling prosthetics and devices in real-life settings. Our work aimed at developing a low-cost, print-and-play platform to acquire and analyse sEMG signals that can be arranged in a fully customized way, depending on the application and the users' needs. We produced 8-channel sEMG matrices to measure the muscular activity of the forearm using innovative nanoparticle-based inks to print the sensors embedded into each matrix using a commercial inkjet printer. Then, we acquired the multi-channel sEMG data from 12 participants while repeatedly performing twelve standard finger movements (six extensions and six flexions). Our results showed that inkjet printing-based sEMG signals ensured significant similarity values across repetitions in every participant, a large enough difference between movements (dissimilarity index above 0.2), and an overall classification accuracy of 93-95% for flexion and extension, respectively.

A novel framework for designing a multi-DoF prosthetic wrist control using machine learning

Prosthetic arms can significantly increase the upper limb function of individuals with upper limb loss, however despite the development of various multi-DoF prosthetic arms the rate of prosthesis abandonment is still high. One of the major challenges is to design a multi-DoF controller that has high precision, robustness, and intuitiveness for daily use. The present study demonstrates a novel framework for developing a controller leveraging machine learning algorithms and movement synergies to implement natural control of a 2-DoF prosthetic wrist for activities of daily living (ADL). The data was collected during ADL tasks of ten individuals with a wrist brace emulating the absence of wrist function. Using this data, the neural network classifies the movement and then random forest regression computes the desired velocity of the prosthetic wrist. The models were trained/tested with ADLs where their robustness was tested using cross-validation and holdout data sets. The proposed framework demonstrated high accuracy (F-1 score of 99% for the classifier and Pearson's correlation of 0.98 for the regression). Additionally, the interpretable nature of random forest regression was used to verify the targeted movement synergies. The present work provides a novel and effective framework to develop an intuitive control for multi-DoF prosthetic devices.

Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Automatic Fracture Characterization Using Tactile and Proximity Optical Sensing

This paper demonstrates how tactile and proximity sensing can be used to perform automatic mechanical fractures detection (surface cracks). For this purpose, a custom-designed integrated tactile and proximity sensor has been implemented. With the help of fiber optics, the sensor measures the deformation of its body, when interacting with the physical environment, and the distance to the environment's objects. This sensor slides across different surfaces and records data which are then analyzed to detect and classify fractures and other mechanical features. The proposed method implements machine learning techniques (handcrafted features, and state of the art classification algorithms). An average crack detection accuracy of ~94% and width classification accuracy of ~80% is achieved. Kruskal-Wallis results (p < 0.001) indicate statistically significant differences among results obtained when analysing only integrated deformation measurements, only proximity measurements and both deformation and proximity data. A real-time classification method has been implemented for online classification of explored surfaces. In contrast to previous techniques, which mainly rely on visual modality, the proposed approach based on optical fibers might be more suitable for operation in extreme environments (such as nuclear facilities) where radiation may damage electronic components of commonly employed sensing devices, such as standard force sensors based on strain gauges and video cameras.

Dimensionality analysis of forearm muscle activation for myoelectric control in transradial amputees

Establishing a natural communication interface between the user and the terminal device is one of the central challenges of hand neuroprosthetics research. Surface electromyography (EMG) is the most common source of neural signals for interpreting a user’s intent in these interfaces. However, how the capacity of EMG generation is affected by various clinical parameters remains largely unknown. In this study, we examined the EMG activity of forearm muscles recorded from 11 transradially amputated subjects who performed a wide range of movements. EMG recordings from 40 able-bodied subjects were also analyzed to provide comparative benchmarks. By using non-negative matrix factorization, we extracted the synergistic EMG patterns for each subject to estimate the dimensionality of muscle control, under the framework of motor synergies. We found that amputees exhibited less than four synergies (with substantial variability related to the length of remaining limb and age), whereas able-bodied subjects commonly demonstrate five or more synergies. The results of this study provide novel insight into the muscle synergy framework and the design of natural myoelectric control interfaces.

Variability of Muscle Synergies in Hand Grasps: Analysis of Intra- and Inter-Session Data

Background. Muscle synergy analysis is an approach to understand the neurophysiological mechanisms behind the hypothesized ability of the Central Nervous System (CNS) to reduce the dimensionality of muscle control. The muscle synergy approach is also used to evaluate motor recovery and the evolution of the patients’ motor performance both in single-session and longitudinal studies. Synergy-based assessments are subject to various sources of variability: natural trial-by-trial variability of performed movements, intrinsic characteristics of subjects that change over time (e.g., recovery, adaptation, exercise, etc.), as well as experimental factors such as different electrode positioning. These sources of variability need to be quantified in order to resolve challenges for the application of muscle synergies in clinical environments. The objective of this study is to analyze the stability and similarity of extracted muscle synergies under the effect of factors that may induce variability, including inter- and intra-session variability within subjects and inter-subject variability differentiation. The analysis was performed using the comprehensive, publicly available hand grasp NinaPro Database, featuring surface electromyography (EMG) measures from two EMG electrode bracelets. Methods. Intra-session, inter-session, and inter-subject synergy stability was analyzed using the following measures: variance accounted for (VAF) and number of synergies (NoS) as measures of reconstruction stability quality and cosine similarity for comparison of spatial composition of extracted synergies. Moreover, an approach based on virtual electrode repositioning was applied to shed light on the influence of electrode position on inter-session synergy similarity. Results. Inter-session synergy similarity was significantly lower with respect to intra-session similarity, both considering coefficient of variation of VAF (approximately 0.2–15% for inter vs. approximately 0.1% to 2.5% for intra, depending on NoS) and coefficient of variation of NoS (approximately 6.5–14.5% for inter vs. approximately 3–3.5% for intra, depending on VAF) as well as synergy similarity (approximately 74–77% for inter vs. approximately 88–94% for intra, depending on the selected VAF). Virtual electrode repositioning revealed that a slightly different electrode position can lower similarity of synergies from the same session and can increase similarity between sessions. Finally, the similarity of inter-subject synergies has no significant difference from the similarity of inter-session synergies (both on average approximately 84–90% depending on selected VAF). Conclusion. Synergy similarity was lower in inter-session conditions with respect to intra-session. This finding should be considered when interpreting results from multi-session assessments. Lastly, electrode positioning might play an important role in the lower similarity of synergies over different sessions.

Homecare Robotic Systems for Healthcare 4.0: Visions and Enabling Technologies

Powered by the technologies that have originated from manufacturing, the fourth revolution of healthcare technologies is happening (Healthcare 4.0). As an example of such revolution, new generation homecare robotic systems (HRS) based on the cyber-physical systems (CPS) with higher speed and more intelligent execution are emerging. In this article, the new visions and features of the CPS-based HRS are proposed. The latest progress in related enabling technologies is reviewed, including artificial intelligence, sensing fundamentals, materials and machines, cloud computing and communication, as well as motion capture and mapping. Finally, the future perspectives of the CPS-based HRS and the technical challenges faced in each technical area are discussed.

Robust Real-Time Embedded EMG Recognition Framework Using Temporal Convolutional Networks on a Multicore IoT Processor

Hand movement classification via surface electromyographic (sEMG) signal is a well-established approach for advanced Human-Computer Interaction. However, sEMG movement recognition has to deal with the long-term reliability of sEMG-based control, limited by the variability affecting the sEMG signal. Embedded solutions are affected by a recognition accuracy drop over time that makes them unsuitable for reliable gesture controller design. In this paper, we present a complete wearable-class embedded system for robust sEMG-based gesture recognition, based on Temporal Convolutional Networks (TCNs). Firstly, we developed a novel TCN topology (TEMPONet), and we tested our solution on a benchmark dataset (Ninapro), achieving 49.6% average accuracy, 7.8%, better than current State-Of-the-Art (SoA). Moreover, we designed an energy-efficient embedded platform based on GAP8, a novel 8-core IoT processor. Using our embedded platform, we collected a second 20-sessions dataset to validate the system on a setup which is representative of the final deployment. We obtain 93.7% average accuracy with the TCN, comparable with a SoA SVM approach (91.1%). Finally, we profiled the performance of the network implemented on GAP8 by using an 8-bit quantization strategy to fit the memory constraint of the processor. We reach a 4× lower memory footprint (460 kB) with a performance degradation of only 3% accuracy. We detailed the execution on the GAP8 platform, showing that the quantized network executes a single classification in 12.84 ms with a power envelope of 0.9 mJ, making it suitable for a long-lifetime wearable deployment.

Pattern recognition and direct control home use of a multi-articulating hand prosthesis

Although more multi-articulating hand prostheses have become commercially available, replacing a missing hand remains challenging from a control perspective. This study investigated myoelectric direct control and pattern recognition home use of a multi-articulating hand prosthesis for individuals with a transradial amputation. Four participants were fitted with an i-limb Ultra Revolution hand and a Coapt COMPLETE CONTROL system. An occupational therapist provided training for each control style and how to use the various grips. The number of grips available to each individual was determined by clinician and user feedback to optimize both the number of grips available and the reliability of grip selection. Home trial data corresponding to individual usage were recorded. No significant differences were found between direct and pattern recognition control home trials in regards to trial length (p=0.96), days powered on (p=0.21), or total time powered on (p=0.91). There was a higher average number of configured grips for direct control at 4.8 [0.5] compared to 3.8 [0.5] for pattern recognition control, but this difference did not reach significance (p=0.092). Across all hand close movements, users spent a majority of time $(\gt80$%) in one grip when using direct control. For pattern recognition usage was spread across more grips $(\gt45$% time in one grip, 25% time in a 2^nd grip, and 20% time in a 3^rd grip). Pattern recognition control may provide users with a more intuitive way to select and use the various grips available to them.

Wearable technology in stroke rehabilitation: towards improved diagnosis and treatment of upper-limb motor impairment

Stroke is one of the main causes of long-term disability worldwide, placing a large burden on individuals and society. Rehabilitation after stroke consists of an iterative process involving assessments and specialized training, aspects often constrained by limited resources of healthcare centers. Wearable technology has the potential to objectively assess and monitor patients inside and outside clinical environments, enabling a more detailed evaluation of the impairment and allowing the individualization of rehabilitation therapies. The present review aims to provide an overview of wearable sensors used in stroke rehabilitation research, with a particular focus on the upper extremity. We summarize results obtained by current research using a variety of wearable sensors and use them to critically discuss challenges and opportunities in the ongoing effort towards reliable and accessible tools for stroke rehabilitation. Finally, suggestions concerning data acquisition and processing to guide future studies performed by clinicians and engineers alike are provided.

PaWFE: Fast Signal Feature Extraction Using Parallel Time Windows

Motivation: Hand amputations can dramatically affect the quality of life of a person. Researchers are developing surface electromyography and machine learning solutions to control dexterous and robotic prosthetic hands, however long computational times can slow down this process.

Objective: This paper aims at creating a fast signal feature extraction algorithm that can extract widely used features and allow researchers to easily add new ones.

Methods: PaWFE (Parallel Window Feature Extractor) extracts the signal features from several time windows in parallel. The MATLAB code is publicly available and supports several time domain and frequency features. The code was tested and benchmarked using 1,2,4,8,16,32, and 48 threads on a server with four Xeon E7- 4820 and 128 GB RAM using the first 5 datasets of the Ninapro database, that are recorded with different acquisition setups.

Results: The parallel time window analysis approach allows to reduce the computational time up to 20 times when using 32 cores, showing a very good scalability. Signal features can be extracted in few seconds from an entire data acquisition and in <100 ms from a single time window, easily reducing of up to over 15 times the feature extraction procedure in comparison to traditional approaches. The code allows users to easily add new signal feature extraction scripts, that can be added to the code and on the Ninapro website upon request.

Significance: The code allows researchers in machine learning and biosignals data analysis to easily and quickly test modern machine learning approaches on big datasets and it can be used as a resource for real time data analysis too.

putEMG-A Surface Electromyography Hand Gesture Recognition Dataset

In this paper, we present a putEMG dataset intended for the evaluation of hand gesture recognition methods based on sEMG signal. The dataset was acquired for 44 able-bodied subjects and include 8 gestures (3 full hand gestures, 4 pinches and idle). It consists of uninterrupted recordings of 24 sEMG channels from the subject's forearm, RGB video stream and depth camera images used for hand motion tracking. Moreover, exemplary processing scripts are also published. The putEMG dataset is available under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). The dataset was validated regarding sEMG amplitudes and gesture recognition performance. The classification was performed using state-of-the-art classifiers and feature sets. An accuracy of 90% was achieved for SVM classifier utilising RMS feature and for LDA classifier using Hudgin's and Du's feature sets. Analysis of performance for particular gestures showed that LDA/Du combination has significantly higher accuracy for full hand gestures, while SVM/RMS performs better for pinch gestures. The presented dataset can be used as a benchmark for various classification methods, the evaluation of electrode localisation concepts, or the development of classification methods invariant to user-specific features or electrode displacement.

Open Database for Accurate Upper-Limb Intent Detection Using Electromyography and Reliable Extreme Learning Machines

Surface Electromyography (sEMG) signal processing has a disruptive technology potential to enable a natural human interface with artificial limbs and assistive devices. However, this biosignal real-time control interface still presents several restrictions such as control limitations due to a lack of reliable signal prediction and standards for signal processing among research groups. Our paper aims to present and validate our sEMG database through the signal classification performed by the reliable forms of our Extreme Learning Machines (ELM) classifiers, used to maintain a more consistent signal classification. To perform the signal processing, we explore the use of a stochastic filter based on the Antonyan Vardan Transform (AVT) in combination with two variations of our Reliable classifiers (denoted R-ELM and R-Regularized ELM (RELM), respectively), to derive a reliability metric from the system, which autonomously selects the most reliable samples for the signal classification. To validate and compare our database and classifiers with related papers, we performed the classification of the whole of Databases 1, 2, and 6 (DB1, DB2, and DB6) of the NINAProdatabase. Our database presented consistent results, while the reliable forms of ELM classifiers matched or outperformed related papers, reaching average accuracies higher than 99% for the IEEdatabase, while average accuracies of 75.1%, 79.77%, and 69.83% were achieved for NINAPro DB1, DB2, and DB6, respectively.

Surface-Electromyography-Based Gesture Recognition by Multi-View Deep Learning

Gesture recognition using sparse multichannel surface electromyography (sEMG) is a challenging problem, and the solutions are far from optimal from the point of view of muscle-computer interface. In this paper, we address this problem from the context of multi-view deep learning. A novel multi-view convolutional neural network (CNN) framework is proposed by combining classical sEMG feature sets with a CNN-based deep learning model. The framework consists of two parts. In the first part, multi-view representations of sEMG are modeled in parallel by a multistream CNN, and a performance-based view construction strategy is proposed to choose the most discriminative views from classical feature sets for sEMG-based gesture recognition. In the second part, the learned multi-view deep features are fused through a view aggregation network composed of early and late fusion subnetworks, taking advantage of both early and late fusion of learned multi-view deep features. Evaluations on 11 sparse multichannel sEMG databases as well as five databases with both sEMG and inertial measurement unit data demonstrate that our multi-view framework outperforms single-view methods on both unimodal and multimodal sEMG data streams.

EMG Pattern Recognition in the Era of Big Data and Deep Learning

The increasing amount of data in electromyographic (EMG) signal research has greatly increased the importance of developing advanced data analysis and machine learning techniques which are better able to handle “big data”. Consequently, more advanced applications of EMG pattern recognition have been developed. This paper begins with a brief introduction to the main factors that expand EMG data resources into the era of big data, followed by the recent progress of existing shared EMG data sets. Next, we provide a review of recent research and development in EMG pattern recognition methods that can be applied to big data analytics. These modern EMG signal analysis methods can be divided into two main categories: (1) methods based on feature engineering involving a promising big data exploration tool called topological data analysis; and (2) methods based on feature learning with a special emphasis on “deep learning”. Finally, directions for future research in EMG pattern recognition are outlined and discussed.