A preliminary investigation on the utility of temporal features of Force Myography in the two-class problem of grasp vs. no-grasp in the presence of upper-extremity movements

k-Tournament Grasshopper Extreme Learner for FMG-Based Gesture Recognition

The recognition of hand signs is essential for several applications. Due to the variation of possible signals and the complexity of sensor-based systems for hand gesture recognition, a new artificial neural network algorithm providing high accuracy with a reduced architecture and automatic feature selection is needed. In this paper, a novel classification method based on an extreme learning machine (ELM), supported by an improved grasshopper optimization algorithm (GOA) as a core for a weight-pruning process, is proposed. The k-tournament grasshopper optimization algorithm was implemented to select and prune the ELM weights resulting in the proposed k-tournament grasshopper extreme learner (KTGEL) classifier. Myographic methods, such as force myography (FMG), deliver interesting signals that can build the basis for hand sign recognition. FMG was investigated to limit the number of sensors at suitable positions and provide adequate signal processing algorithms for perspective implementation in wearable embedded systems. Based on the proposed KTGEL, the number of sensors and the effect of the number of subjects was investigated in the first stage. It was shown that by increasing the number of subjects participating in the data collection, eight was the minimal number of sensors needed to result in acceptable sign recognition performance. Moreover, implemented with 3000 hidden nodes, after the feature selection wrapper, the ELM had both a microaverage precision and a microaverage sensitivity of 97% for the recognition of a set of gestures, including a middle ambiguity level. The KTGEL reduced the hidden nodes to only 1000, reaching the same total sensitivity with a reduced total precision of only 1% without needing an additional feature selection method.

Proposal of a Wearable Multimodal Sensing-Based Serious Games Approach for Hand Movement Training After Stroke

Stroke often leads to hand motor dysfunction, and effective rehabilitation requires keeping patients engaged and motivated. Among the existing automated rehabilitation approaches, data glove-based systems are not easy to wear for patients due to spasticity, and single sensor-based approaches generally provided prohibitively limited information. We thus propose a wearable multimodal serious games approach for hand movement training after stroke. A force myography (FMG), electromyography (EMG), and inertial measurement unit (IMU)-based multi-sensor fusion model was proposed for hand movement classification, which was worn on the user’s affected arm. Two movement recognition-based serious games were developed for hand movement and cognition training. Ten stroke patients with mild to moderate motor impairments (Brunnstrom Stage for Hand II-VI) performed experiments while playing interactive serious games requiring 12 activities-of-daily-living (ADLs) hand movements taken from the Fugl Meyer Assessment. Feasibility was evaluated by movement classification accuracy and qualitative patient questionnaires. The offline classification accuracy using combined FMG-EMG-IMU was 81.0% for the 12 movements, which was significantly higher than any single sensing modality; only EMG, only FMG, and only IMU were 69.6, 63.2, and 47.8%, respectively. Patients reported that they were more enthusiastic about hand movement training while playing the serious games as compared to conventional methods and strongly agreed that they subjectively felt that the proposed training could be beneficial for improving upper limb motor function. These results showed that multimodal-sensor fusion improved hand gesture classification accuracy for stroke patients and demonstrated the potential of this proposed approach to be used as upper limb movement training after stroke.

Real-World Functional Grasping Activity in Individuals With Stroke and Healthy Controls Using a Novel Wearable Wrist Sensor

Background. While wrist-worn accelerometers have been used to measure upper extremity use in the past, they primarily measure arm motion and lack the ability to capture functional hand opening and grasping activities which are essential for activities of daily living. Objectives. To characterize real-world functional hand opening and grasping activities captured over multiple days in adults with stroke and in matched controls using a novel wrist-worn device. Methods. Twenty-eight individuals (fourteen individuals with stroke and 14 healthy controls) wore the devices on both wrists for 3 days. Functional hand activity was characterized by daily hand counts, hourly hand counts, and asymmetry between hands. The Mann-Whitney U test was used to evaluate differences in functional hand activities between the two groups. Results. The stroke group had 1480 and 4691 daily hand counts in their affected and nonaffected hands, respectively. The control group had 3559 and 5021 daily hand counts in their nondominant and dominant hands, respectively. Significantly fewer daily hand counts (P = .019), fewer hourly hand counts (P = .024), and a larger asymmetry index (P = .01) of the affected hand in the stroke group were found compared to that of the nondominant hand in the control group. Conclusions. Real-world functional upper extremity activity can be measured using this novel wrist-worn device. Unlike wrist-worn accelerometers, this wrist-worn device can provide a measurement of functional grasping activity. The findings have implications for clinicians and researchers to monitor and assess real-world hand activity, as well as to apply specific doses of repetitions to improve neural recovery after stroke.

FMG- and RNN-Based Estimation of Motor Intention of Upper-Limb Motion in Human-Robot Collaboration

Research on human-robot interactions has been driven by the increasing employment of robotic manipulators in manufacturing and production. Toward developing more effective human-robot collaboration during shared tasks, this paper proposes an interaction scheme by employing machine learning algorithms to interpret biosignals acquired from the human user and accordingly planning the robot reaction. More specifically, a force myography (FMG) band was wrapped around the user's forearm and was used to collect information about muscle contractions during a set of collaborative tasks between the user and an industrial robot. A recurrent neural network model was trained to estimate the user's hand movement pattern based on the collected FMG data to determine whether the performed motion was random or intended as part of the predefined collaborative tasks. Experimental evaluation during two practical collaboration scenarios demonstrated that the trained model could successfully estimate the category of hand motion, i.e., intended or random, such that the robot either assisted with performing the task or changed its course of action to avoid collision. Furthermore, proximity sensors were mounted on the robotic arm to investigate if monitoring the distance between the user and the robot had an effect on the outcome of the collaborative effort. While further investigation is required to rigorously establish the safety of the human worker, this study demonstrates the potential of FMG-based wearable technologies to enhance human-robot collaboration in industrial settings.

A comparative study of motion detection with FMG and sEMG methods for assistive applications

Introduction

While surface-electromyography (sEMG) has been widely used in limb motion detection for the control of exoskeleton, there is an increasing interest to use forcemyography (FMG) method to detect motion. In this paper, we review the applications of two types of motion detection methods. Their performances were experimentally compared in day-to-day classification of forearm motions. The objective is to select a detection method suitable for motion assistance on a daily basis.

Methods

Comparisons of motion detection with FMG and sEMG were carried out considering classification accuracy (CA), repeatability and training scheme. For both methods, classification of motions was achieved through feed-forward neural network. Repeatability was evaluated on the basis of change in CA between days and also training schemes.

Results

The experiments shows that day-to-day CA with FMG can reach 84.9%, compared with a CA of 77.8% with sEMG, when the classifiers were trained only on the first day. Moreover, the CA with FMG can reach to 86.5%, comparable to CA of 84.1% with sEMG, if classifiers were trained daily.

Conclusions

Results suggest that data recorded from FMG is more repeatable in day-to-day testing and therefore FMG-based methods can be more useful than sEMG-based methods for motion detection in applications where exoskeletons are used as needed on a daily basis.

Estimating Vertical Ground Reaction Force during Walking Using a Single Inertial Sensor

The vertical ground reaction force (vGRF) and its passive and active peaks are important gait parameters and of great relevance for musculoskeletal injury analysis and prevention, the detection of gait abnormities, and the evaluation of lower-extremity prostheses. Most currently available methods to estimate the vGRF require a force plate. However, in real-world scenarios, gait monitoring would not be limited to a laboratory setting. This paper reports a novel solution using machine learning algorithms to estimate the vGRF and the timing and magnitude of its peaks from data collected by a single inertial measurement unit (IMU) on one of the lower limb locations. Nine volunteers participated in this study, walking on a force plate-instrumented treadmill at various speeds. Four IMUs were worn on the foot, shank, distal thigh, and proximal thigh, respectively. A random forest model was employed to estimate the vGRF from data collected by each of the IMUs. We evaluated the performance of the models against the gold standard measurement of the vGRF generated by the treadmill. The developed model achieved a high accuracy with a correlation coefficient, root mean square error, and normalized root mean square error of 1.00, 0.02 body weight (BW), and 1.7% in intra-participant testing, and 0.97, 0.10 BW, and 7.15% in inter-participant testing, respectively, for the shank location. The difference between the reference and estimated passive force peak values was 0.02 BW and 0.14 BW with a delay of −0.14% and 0.57% of stance duration for the intra- and inter-participant testing, respectively; the difference between the reference and estimated active force peak values was 0.02 BW and 0.08 BW with a delay of 0.45% and 1.66% of stance duration for the intra- and inter-participant evaluation, respectively. We concluded that vertical ground reaction force can be estimated using only a single IMU via machine learning algorithms. This research sheds light on the development of a portable wearable gait monitoring system reporting the real-time vGRF in real-life scenarios.

Wrist-worn wearables based on force myography: on the significance of user anthropometry

Background

Force myography (FMG) is a non-invasive technology used to track functional movements and hand gestures by sensing volumetric changes in the limbs caused by muscle contraction. Force transmission through tissue implies that differences in tissue mechanics and/or architecture might impact FMG signal acquisition and the accuracy of gesture classifier models. The aim of this study is to identify if and how user anthropometry affects the quality of FMG signal acquisition and the performance of machine learning models trained to classify different hand and wrist gestures based on that data.

Methods

Wrist and forearm anthropometric measures were collected from a total of 21 volunteers aged between 22 and 82 years old. Participants performed a set of tasks while wearing a custom-designed FMG band. Primary outcome measure was the Spearman’s correlation coefficient (R) between the anthropometric measures and FMG signal quality/ML model performance.

Results

Results demonstrated moderate (0.3 ≤|R| < 0.67) and strong (0.67 ≤ |R|) relationships for ratio of skinfold thickness to forearm circumference, grip strength and ratio of wrist to forearm circumference. These anthropometric features contributed to 23–30% of the variability in FMG signal acquisition and as much as 50% of the variability in classification accuracy for single gestures.

Conclusions

Increased grip strength, larger forearm girth, and smaller skinfold-to-forearm circumference ratio improve signal quality and gesture classification accuracy.

Estimation of Ankle Joint Power during Walking Using Two Inertial Sensors

(1) Background: Ankle joint power, as an indicator of the ability to control lower limbs, is of great relevance for clinical diagnosis of gait impairment and control of lower limb prosthesis. However, the majority of available techniques for estimating joint power are based on inverse dynamics methods, which require performing a biomechanical analysis of the foot and using a highly instrumented environment to tune the parameters of the resulting biomechanical model. Such techniques are not generally applicable to real-world scenarios in which gait monitoring outside of the clinical setting is desired. This paper proposes a viable alternative to such techniques by using machine learning algorithms to estimate ankle joint power from data collected by two miniature inertial measurement units (IMUs) on the foot and shank, (2) Methods: Nine participants walked on a force-plate-instrumented treadmill wearing two IMUs. The data from the IMUs were processed to train and test a random forest model to estimate ankle joint power. The performance of the model was then evaluated by comparing the estimated power values to the reference values provided by the motion tracking system and the force-plate-instrumented treadmill. (3) Results: The proposed method achieved a high accuracy with the correlation coefficient, root mean square error, and normalized root mean square error of 0.98, 0.06 w/kg, and 1.05% in the intra-subject test, and 0.92, 0.13 w/kg, and 2.37% in inter-subject test, respectively. The difference between the predicted and true peak power values was 0.01 w/kg and 0.14 w/kg with a delay of 0.4% and 0.4% of gait cycle duration for the intra- and inter-subject testing, respectively. (4) Conclusions: The results of this study demonstrate the feasibility of using only two IMUs to estimate ankle joint power. The proposed technique provides a basis for developing a portable and compact gait monitoring system that can potentially offer monitoring and reporting on ankle joint power in real-time during activities of daily living.

FMG Versus EMG: A Comparison of Usability for Real-Time Pattern Recognition Based Control

OBJECTIVE

Force myography (FMG), which measures the surface pressure profile exerted by contracting muscles, has been proposed as an alternative to electromyography (EMG) for human-machine interfaces. Although FMG pattern recognition-based control systems have yielded higher offline classification accuracy, comparatively few works have examined the usability of FMG for real-time control. In this work, we conduct a comprehensive comparison of EMG- and FMG-based schemes using both classification and regression controllers.

METHODS

A total of 20 participants performed a two-degree-of-freedom Fitts' Law-style virtual target acquisition task using both FMG- and EMG-based classification and regression control schemes. Performance was evaluated based on the standard Fitts' law testing metrics throughput, path efficiency, average speed, number of timeouts, overshoot, stopping distance, and simultaneity.

RESULTS

The FMG-based classification system significantly outperformed the EMG-based classification system in both throughput (0.902 ± 0.270) versus (0.751 ± 0.309), (ρ < 0.001) and path efficiency (87.2 ± 8.7) versus (83.2 ± 7.8), (ρ < 0.001). Similarly, FMG-based regression significantly outperformed EMG-based regression in throughput (0.871 ± 0.2) versus (0.69 ± 0.3), (ρ < 0.001) and path efficiency (64.8 ± 5.3) versus (58.8 ± 7.1), (ρ < 0.001).

CONCLUSIONS

The FMG-based schemes outperformed the EMG-based schemes regardless of which controller was used. This provides further evidence for FMG as a viable alternative to EMG for human-machine interfaces.

SIGNIFICANCE

This work describes a comprehensive evaluation of the online usability of FMG- and EMG-based control using both sequential classification and simultaneous regression control.

A Wearable Gait Phase Detection System Based on Force Myography Techniques

(1) Background: Quantitative evaluation of gait parameters can provide useful information for constructing individuals’ gait profile, diagnosing gait abnormalities, and better planning of rehabilitation schemes to restore normal gait pattern. Objective determination of gait phases in a gait cycle is a key requirement in gait analysis applications; (2) Methods: In this study, the feasibility of using a force myography-based technique for a wearable gait phase detection system is explored. In this regard, a force myography band is developed and tested with nine participants walking on a treadmill. The collected force myography data are first examined sample-by-sample and classified into four phases using Linear Discriminant Analysis. The gait phase events are then detected from these classified samples using a set of supervisory rules; (3) Results: The results show that the force myography band can correctly detect more than 99.9% of gait phases with zero insertions and only four deletions over 12,965 gait phase segments. The average temporal error of gait phase detection is 55.2 ms, which translates into 2.1% error with respect to the corresponding labelled stride duration; (4) Conclusions: This proof-of-concept study demonstrates the feasibility of force myography techniques as viable solutions in developing wearable gait phase detection systems.