Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses

Does Exerting Grasps Involve a Finite Set of Muscle Patterns? A Study of Intra- and Intersubject Variability of Forearm sEMG Signals in Seven Grasp Types

Surface Electromyography (sEMG) signals are widely used as input to control robotic devices, prosthetic limbs, exoskeletons, among other devices, and provide information about someone's intention to perform a particular movement. However, the redundant action of 32 muscles in the forearm and hand means that the neuromotor system can select different combinations of muscular activities to perform the same grasp, and these combinations could differ among subjects, and even among the trials done by the same subject. In this work, 22 healthy subjects performed seven representative grasp types (the most commonly used). sEMG signals were recorded from seven representative forearm spots identified in a previous work. Intra- and intersubject variability are presented by using four sEMG characteristics: muscle activity, zero crossing, enhanced wavelength and enhanced mean absolute value. The results confirmed the presence of both intra- and intersubject variability, which evidences the existence of distinct, yet limited, muscle patterns while executing the same grasp. This work underscores the importance of utilizing diverse combinations of sEMG features or characteristics of various natures, such as time-domain or frequency-domain, and it is the first work to observe the effect of considering different muscular patterns during grasps execution. This approach is applicable for fine-tuning the control settings of current sEMG devices.

Advances in active knee brace technology: A review of gait analysis, actuation, and control applications

This article discusses the significance of knee joint mechanics and the consequences of knee dysfunctions on an individual's quality of life. The utilization of active knee braces, which incorporate concepts of mechatronics systems, is investigated here as a potential treatment option. The complexity of the construction of the knee joint, which has six degrees of motion and is more prone to injury since it bears weight, is emphasized in this article. By wearing braces and using other support devices, one's knee can increase stability and mobility. In addition, the paper discusses various technologies that can be used to measure the knee adduction moment and supply spatial information on gait. Actuators for active knee braces must be compact, lightweight, and capable of producing a significant amount of torque; as a result, electric, hydraulic, and pneumatic actuators are the most common types. Creating control mechanisms, such as position control techniques and force/torque control approaches, is essential to knee exoskeleton research and development. These methods might make knee joint rehabilitation and assistive technology safer and more effective.

Continuous Locomotion Mode and Task Identification for an Assistive Exoskeleton Based on Neuromuscular-Mechanical Fusion

Human walking parameters exhibit significant variability depending on the terrain, speed, and load. Assistive exoskeletons currently focus on the recognition of locomotion terrain, ignoring the identification of locomotion tasks, which are also essential for control strategies. The aim of this study was to develop an interface for locomotion mode and task identification based on a neuromuscular-mechanical fusion algorithm. The modes of level and incline and tasks of speed and load were explored, and seven able-bodied participants were recruited. A continuous stream of assistive decisions supporting timely exoskeleton control was achieved according to the classification of locomotion. We investigated the optimal algorithm, feature set, window increment, window length, and robustness for precise identification and synchronization between exoskeleton assistive force and human limb movements (human-machine collaboration). The best recognition results were obtained when using a support vector machine, a root mean square/waveform length/acceleration feature set, a window length of 170, and a window increment of 20. The average identification accuracy reached 98.7% ± 1.3%. These results suggest that the surface electromyography-acceleration can be effectively used for locomotion mode and task identification. This study contributes to the development of locomotion mode and task recognition as well as exoskeleton control for seamless transitions.

Toward the Development of User-Centered Neurointegrated Lower Limb Prostheses

The last few years witnessed radical improvements in lower-limb prostheses. Researchers have presented innovative solutions to overcome the limits of the first generation of prostheses, refining specific aspects which could be implemented in future prostheses designs. Each aspect of lower-limb prostheses has been upgraded, but despite these advances, a number of deficiencies remain and the most capable limb prostheses fall far short of the capabilities of the healthy limb. This article describes the current state of prosthesis technology; identifies a number of deficiencies across the spectrum of lower limb prosthetic components with respect to users' needs; and discusses research opportunities in design and control that would substantially improve functionality concerning each deficiency. In doing so, the authors present a roadmap of patients related issues that should be addressed in order to fulfill the vision of a next-generation, neurally-integrated, highly-functional lower limb prosthesis.

Recent trends and challenges of surface electromyography in prosthetic applications

Surface electromyography (sEMG) meets extensive applications in the field of prosthesis in the current period. The effectiveness of sEMG in prosthesis applications has been verified by numerous revolutionary developments and extensive research attempts. A large volume of research and literature works have explored and validated the vast use of these signals in prostheses as an assistive technology. The objective of this paper is to conduct a systematic review and offer a detailed overview of the work record in the prosthesis and myoelectric interfaces framework. This review utilized a systematic search strategy to identify published articles discussing the state-of-the-art applications of sEMG in prostheses (including upper limb prosthesis and lower limb prostheses). Relevant studies were identified using electronic databases such as PubMed, IEEE Explore, SCOPUS, ScienceDirect, Google Scholar and Web of Science. Out of 3791 studies retrieved from the databases, 188 articles were found to be potentially relevant (after screening of abstracts and application of inclusion-exclusion criteria) and included in this review. This review presents an investigative analysis of sEMG-based prosthetic applications to assist the readers in making further advancements in this field. It also discusses the fundamental advantages and disadvantages of using sEMG in prosthetic applications. It also includes some important guidelines to follow in order to improve the performance of sEMG-based prosthesis. The findings of this study support the widespread use of sEMG in prosthetics. It is concluded that sEMG-based prosthesis technology, still in its sprouting phase, requires significant explorations for further development. Supplementary investigations are necessary in the direction of making a seamless mechanism of biomechatronics for sEMG-based prosthesis by cohesive efforts of robotic researchers and biomedical engineers.

Electromyography-Based Control of Lower Limb Prostheses: A Systematic Review

Most amputations occur in lower limbs and despite improvements in prosthetic technology, no commercially available prosthetic leg uses electromyography (EMG) information as an input for control. Efforts to integrate EMG signals as part of the control strategy have increased in the last decade. In this systematic review, we summarize the research in the field of lower limb prosthetic control using EMG. Four different online databases were searched until June 2022: Web of Science, Scopus, PubMed, and Science Direct. We included articles that reported systems for controlling a prosthetic leg (with an ankle and/or knee actuator) by decoding gait intent using EMG signals alone or in combination with other sensors. A total of 1,331 papers were initially assessed and 121 were finally included in this systematic review. The literature showed that despite the burgeoning interest in research, controlling a leg prosthesis using EMG signals remains challenging. Specifically, regarding EMG signal quality and stability, electrode placement, prosthetic hardware, and control algorithms, all of which need to be more robust for everyday use. In the studies that were investigated, large variations were found between the control methodologies, type of research participant, recording protocols, assessments, and prosthetic hardware.

The use of nonnormalized surface EMG and feature inputs for LSTM-based powered ankle prosthesis control algorithm development

Advancements in instrumentation support improved powered ankle prostheses hardware development. However, control algorithms have limitations regarding number and type of sensors utilized and achieving autonomous adaptation, which is key to a natural ambulation. Surface electromyogram (sEMG) sensors are promising. With a minimized number of sEMG inputs an economic control algorithm can be developed, whereas limiting the use of lower leg muscles will provide a practical algorithm for both ankle disarticulation and transtibial amputation. To determine appropriate sensor combinations, a systematic assessment of the predictive success of variations of multiple sEMG inputs in estimating ankle position and moment has to conducted. More importantly, tackling the use of nonnormalized sEMG data in such algorithm development to overcome processing complexities in real-time is essential, but lacking. We used healthy population level walking data to (1) develop sagittal ankle position and moment predicting algorithms using nonnormalized sEMG, and (2) rank all muscle combinations based on success to determine economic and practical algorithms. Eight lower extremity muscles were studied as sEMG inputs to a long-short-term memory (LSTM) neural network architecture: tibialis anterior (TA), soleus (SO), medial gastrocnemius (MG), peroneus longus (PL), rectus femoris (RF), vastus medialis (VM), biceps femoris (BF) and gluteus maximus (GMax). Five features extracted from nonnormalized sEMG amplitudes were used: integrated EMG (IEMG), mean absolute value (MAV), Willison amplitude (WAMP), root mean square (RMS) and waveform length (WL). Muscle and feature combination variations were ranked using Pearson's correlation coefficient (r > 0.90 indicates successful correlations), the root-mean-square error and one-dimensional statistical parametric mapping between the original data and LSTM response. The results showed that IEMG+WL yields the best feature combination performance. The best performing variation was MG + RF + VM (r_position = 0.9099 and r_moment = 0.9707) whereas, PL (r_position = 0.9001, r_moment = 0.9703) and GMax+VM (r_position = 0.9010, r_moment = 0.9718) were distinguished as the economic and practical variations, respectively. The study established for the first time the use of nonnormalized sEMG in control algorithm development for level walking.

Design, Kinematics and Gait Analysis, of Prosthetic Knee Joints: A Systematic Review

The aim of this review article is to appraise the design and functionality of above-knee prosthetic legs. So far, various transfemoral prosthetic legs are found to offer a stable gait to amputees but are limited to laboratories. The commercially available prosthetic legs are not reliable and comfortable enough to satisfy amputees. There is a dire need for creating a powered prosthetic knee joint that could address amputees' requirements. To pinpoint the gap in transfemoral prosthetic legs, prosthetic knee unit model designs, control frameworks, kinematics, and gait evaluations are concentrated. Ambulation exercises, ground-level walking, running, and slope walking are considered to help identify research gaps and areas where existing prostheses can be ameliorated. The results show that above-knee amputees can more effectively manage their issues with the aid of an active prosthesis, capable of reliable gait. To accomplish the necessary control, closed loop controllers and volitional control are integral parts. Future studies should consider designing a transfemoral electromechanical prosthesis based on electromyographic (EMG) signals to better predict the amputee's intent and control in accordance with that intent.

Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals

Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database "Ninapro DB7" is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.

Ambulation Mode Classification of Individuals with Transfemoral Amputation through A-Mode Sonomyography and Convolutional Neural Networks

Many people struggle with mobility impairments due to lower limb amputations. To participate in society, they need to be able to walk on a wide variety of terrains, such as stairs, ramps, and level ground. Current lower limb powered prostheses require different control strategies for varying ambulation modes, and use data from mechanical sensors within the prosthesis to determine which ambulation mode the user is in. However, it can be challenging to distinguish between ambulation modes. Efforts have been made to improve classification accuracy by adding electromyography information, but this requires a large number of sensors, has a low signal-to-noise ratio, and cannot distinguish between superficial and deep muscle activations. An alternative sensing modality, A-mode ultrasound, can detect and distinguish between changes in superficial and deep muscles. It has also shown promising results in upper limb gesture classification. Despite these advantages, A-mode ultrasound has yet to be employed for lower limb activity classification. Here we show that A- mode ultrasound can classify ambulation mode with comparable, and in some cases, superior accuracy to mechanical sensing. In this study, seven transfemoral amputee subjects walked on an ambulation circuit while wearing A-mode ultrasound transducers, IMU sensors, and their passive prosthesis. The circuit consisted of sitting, standing, level-ground walking, ramp ascent, ramp descent, stair ascent, and stair descent, and a spatial-temporal convolutional network was trained to continuously classify these seven activities. Offline continuous classification with A-mode ultrasound alone was able to achieve an accuracy of 91.8±3.4%, compared with 93.8±3.0%, when using kinematic data alone. Combined kinematic and ultrasound produced 95.8±2.3% accuracy. This suggests that A-mode ultrasound provides additional useful information about the user's gait beyond what is provided by mechanical sensors, and that it may be able to improve ambulation mode classification. By incorporating these sensors into powered prostheses, users may enjoy higher reliability for their prostheses, and more seamless transitions between ambulation modes.

Small-Data-Driven Temporal Convolutional Capsule Network for Locomotion Mode Recognition of Robotic Prostheses

Locomotion mode recognition has been shown to substantially contribute to the precise control of robotic lower-limb prostheses under different walking conditions. In this study, we proposed a temporal convolutional capsule network (TCCN) which integrates the spatial-temporal-based, dilation-convolution-based, dynamic routing and vector-based features for recognizing locomotion mode recognition with small data rather than big-data-based neural networks for robotic prostheses. TCCN proposed in this study has four characteristics, which extracts the (1) spatial-temporal information in the data and then makes (2) dilated convolution to deal with small data, and uses (3) dynamic routing, which produces some similarity to the human brain to processes the data as a (4) vector, which is different from other scalar-based networks, such as convolutional neural network (CNN). By comparison with a traditional machine learning, e.g., support vector machine(SVM) and big-data-driven neural networks, e.g., CNN, recurrent neural network(RNN), temporal convolutional network(TCN) and capsule network(CN). The accuracy of TCCN is 4.1% higher than CNN under 5-fold cross-validation of three-locomotion-mode and 5.2% higher under the 5-fold cross-validation of five-locomotion modes. The main confusion we found appears in the transition state. The results indicate that TCCN may handle small data balancing global and local information which is closer to the way how the human brain works, and the capsule layer allows for better processing of vector information and retains not only magnitude information, but also direction information.

Agonist-antagonist muscle strain in the residual limb preserves motor control and perception after amputation

Background

Elucidating underlying mechanisms in subject-specific motor control and perception after amputation could guide development of advanced surgical and neuroprosthetic technologies. In this study, relationships between preserved agonist-antagonist muscle strain within the residual limb and preserved motor control and perception capacity are investigated.

Methods

Fourteen persons with unilateral transtibial amputations spanning a range of ages, etiologies, and surgical procedures underwent evaluations involving free-space mirrored motions of their lower limbs. Research has shown that varied motor control in biologically intact limbs is executed by the activation of muscle synergies. Here, we assess the naturalness of phantom joint motor control postamputation based on extracted muscle synergies and their activation profiles. Muscle synergy extraction, degree of agonist-antagonist muscle strain, and perception capacity are estimated from electromyography, ultrasonography, and goniometry, respectively.

Results

Here, we show significant positive correlations (P < 0.005–0.05) between sensorimotor responses and residual limb agonist-antagonist muscle strain. Identified trends indicate that preserving even 20–26% of agonist-antagonist muscle strain within the residuum compared to a biologically intact limb is effective in preserving natural motor control postamputation, though preserving limb perception capacity requires more (61%) agonist-antagonist muscle strain preservation.

Conclusions

The results suggest that agonist-antagonist muscle strain is a characteristic, readily ascertainable residual limb structural feature that can help explain variability in amputation outcome, and agonist-antagonist muscle strain preserving surgical amputation strategies are one way to enable more effective and biomimetic sensorimotor control postamputation.

People who undergo limb amputation can have issues with controlling movement and perception of residual limbs. This, in turn, can impact the success of neuroprosthetic strategies, which use signals from the body to control a prosthetic limb. Here, we wanted to understand how sensory signals within the muscle help to preserve movement and limb perception following amputation. We used ultrasound imaging and other methods to measure muscle activity and limb perception in fourteen people who have undergone lower limb amputations. We show that the level at which the relationship between pairs of related muscles is preserved is associated with more natural control of limb movement after amputation. Developing surgical techniques that preserve this relationship may help people living with amputations to naturally perceive and control their residual limbs, and ultimately may improve controllability of assistive prosthetic devices.

Song et al. study the relationship between agonist-antagonist muscle strain (AMS) and motor control and perception in lower limb amputees, with some receiving a myoneural interface intervention. The authors report that the degree of AMS within the residual limb is associated with preserved motor control and perception.

Effect of Subcutaneous Muscle Displacement of Flexor Carpi Radialis on Surface Electromyography

Changes in joint angle can change the position and orientation of muscle fibers relative to the surface EMG electrode. Our previous study has shown that EMG patterns can identify hand/wrist movements with a greater degree of classification accuracy (CA) when muscle contractions involve a change in the joint angle. The results of this study suggest that changes in the position of the muscle relative to the recording electrode can influence the properties of the recorded EMG signals, however, this was not directly quantified. The present study aims to further investigate the effect of subcutaneous muscle displacement caused by the changes in joint angle on surface EMG signals. Nine able-bodied subjects were tested. The subjects were instructed to perform wrist flexion at five different joint angles (0, 20, 40, 60, and 80) with the same level of muscle contraction. EMG signals and ultrasound images were acquired from the flexor carpi radialis (FCR) simultaneously. Time and frequency domain analysis was adopted to extract features from the EMG signals. The subcutaneous muscle displacement of the FCR relative to the skin surface was measured from the ultrasound images. Spearmans rank correlation coefficient was employed to analyze the correlation between the subcutaneous muscle displacement and the EMG signals. The results showed the subcutaneous muscle displacement of the FCR measured by the ultrasound images was 1 cm when the wrist joint angle changed from 0 to 80. There was a positive relationship between the subcutaneous muscle displacement and the mean absolute value (MAV) (rs = 0.896) and median frequency (MF) (rs = 0.849) extracted from the EMG signals. The results demonstrated that subcutaneous muscle displacement associated with wrist angle change had a significant effect on FCR EMG signals. This property might have a positive effect on the CA of dynamic tasks.

EMG-driven control in lower limb prostheses: a topic-based systematic review

BACKGROUND

The inability of users to directly and intuitively control their state-of-the-art commercial prosthesis contributes to a low device acceptance rate. Since Electromyography (EMG)-based control has the potential to address those inabilities, research has flourished on investigating its incorporation in microprocessor-controlled lower limb prostheses (MLLPs). However, despite the proposed benefits of doing so, there is no clear explanation regarding the absence of a commercial product, in contrast to their upper limb counterparts.

OBJECTIVE AND METHODOLOGIES

This manuscript aims to provide a comparative overview of EMG-driven control methods for MLLPs, to identify their prospects and limitations, and to formulate suggestions on future research and development. This is done by systematically reviewing academical studies on EMG MLLPs. In particular, this review is structured by considering four major topics: (1) type of neuro-control, which discusses methods that allow the nervous system to control prosthetic devices through the muscles; (2) type of EMG-driven controllers, which defines the different classes of EMG controllers proposed in the literature; (3) type of neural input and processing, which describes how EMG-driven controllers are implemented; (4) type of performance assessment, which reports the performance of the current state of the art controllers.

RESULTS AND CONCLUSIONS

The obtained results show that the lack of quantitative and standardized measures hinders the possibility to analytically compare the performances of different EMG-driven controllers. In relation to this issue, the real efficacy of EMG-driven controllers for MLLPs have yet to be validated. Nevertheless, in anticipation of the development of a standardized approach for validating EMG MLLPs, the literature suggests that combining multiple neuro-controller types has the potential to develop a more seamless and reliable EMG-driven control. This solution has the promise to retain the high performance of the currently employed non-EMG-driven controllers for rhythmic activities such as walking, whilst improving the performance of volitional activities such as task switching or non-repetitive movements. Although EMG-driven controllers suffer from many drawbacks, such as high sensitivity to noise, recent progress in invasive neural interfaces for prosthetic control (bionics) will allow to build a more reliable connection between the user and the MLLPs. Therefore, advancements in powered MLLPs with integrated EMG-driven control have the potential to strongly reduce the effects of psychosomatic conditions and musculoskeletal degenerative pathologies that are currently affecting lower limb amputees.

Optimization of immune receptor-related hypersensitive cell death response assay using agrobacterium-mediated transient expression in tobacco plants

BACKGROUND

The study of the regulatory mechanisms of evolutionarily conserved Nucleotide-binding leucine-rich repeat (NLR) resistance (R) proteins in animals and plants is of increasing importance due to understanding basic immunity and the value of various crop engineering applications of NLR immune receptors. The importance of temperature is also emerging when applying NLR to crops responding to global climate change. In particular, studies of pathogen effector recognition and autoimmune activity of NLRs in plants can quickly and easily determine their function in tobacco using agro-mediated transient assay. However, there are conditions that should not be overlooked in these cell death-related assays in tobacco.

RESULTS

Environmental conditions play an important role in the immune response of plants. The system used in this study was to establish conditions for optimal hypertensive response (HR) cell death analysis by using the paired NLR RPS4/RRS1 autoimmune and AvrRps4 effector recognition system. The most suitable greenhouse temperature for growing plants was fixed at 22 °C. In this study, RPS4/RRS1-mediated autoimmune activity, RPS4 TIR domain-dependent cell death, and RPS4/RRS1-mediated HR cell death upon AvrRps4 perception significantly inhibited under conditions of 65% humidity. The HR is strongly activated when the humidity is below 10%. Besides, the leaf position of tobacco is important for HR cell death. Position #4 of the leaf from the top in 4-5 weeks old tobacco plants showed the most effective HR cell death.

CONCLUSIONS

As whole genome sequencing (WGS) or resistance gene enrichment sequencing (RenSeq) of various crops continues, different types of NLRs and their functions will be studied. At this time, if we optimize the conditions for evaluating NLR-mediated HR cell death, it will help to more accurately identify the function of NLRs. In addition, it will be possible to contribute to crop development in response to global climate change through NLR engineering.

Subject-Independent Continuous Locomotion Mode Classification for Robotic Hip Exoskeleton Applications

Autonomous lower-limb exoskeletons must modulate assistance based on locomotion mode (e.g., ramp or stair ascent) to adapt to the corresponding changes in human biological joint dynamics. However, current mode classification strategies for exoskeletons often require user-specific tuning, have a slow update rate, and rely on additional sensors outside of the exoskeleton sensor suite. In this study, we introduce a deep convolutional neural network-based locomotion mode classifier for hip exoskeleton applications using an open-source gait biomechanics dataset with various wearable sensors. Our approach removed the limitations of previous systems as it is 1) subject-independent (i.e., no user-specific data), 2) capable of continuously classifying for smooth and seamless mode transitions, and 3) only utilizes minimal wearable sensors native to a conventional hip exoskeleton. We optimized our model, based on several important factors contributing to overall performance, such as transition label timing, model architecture, and sensor placement, which provides a holistic understanding of mode classifier design. Our optimized DL model showed a 3.13% classification error (steady-state: 0.80 ± 0.38% and transitional: 6.49 ± 1.42%), outperforming other machine learning-based benchmarks commonly practiced in the field (p<0.05). Furthermore, our multi-modal analysis indicated that our model can maintain high performance in different settings such as to unseen slopes on stairs or ramps. Thus, our study presents a novel locomotion mode framework, capable of advancing robotic exoskeleton applications towards assisting community ambulation.

Evaluating Electromyography and Sonomyography Sensor Fusion to Estimate Lower-Limb Kinematics Using Gaussian Process Regression

Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

A Multimodal Sensory Apparatus for Robotic Prosthetic Feet Combining Optoelectronic Pressure Transducers and IMU

Timely and reliable identification of control phases is functional to the control of a powered robotic lower-limb prosthesis. This study presents a commercial energy-store-and-release foot prosthesis instrumented with a multimodal sensory system comprising optoelectronic pressure sensors (PS) and IMU. The performance was verified with eight healthy participants, comparing signals processed by two different algorithms, based on PS and IMU, respectively, for real-time detection of heel strike (HS) and toe-off (TO) events and an estimate of relevant biomechanical variables such as vertical ground reaction force (vGRF) and center of pressure along the sagittal axis (CoPy). The performance of both algorithms was benchmarked against a force platform and a marker-based stereophotogrammetric motion capture system. HS and TO were estimated with a time error lower than 0.100 s for both the algorithms, sufficient for the control of a lower-limb robotic prosthesis. Finally, the CoPy computed from the PS showed a Pearson correlation coefficient of 0.97 (0.02) with the same variable computed through the force platform.

Performance of Sonomyographic and Electromyographic Sensing for Continuous Estimation of Joint Torque During Ambulation on Multiple Terrains

Advances in powered assistive device technology, including the ability to provide net mechanical power to multiple joints within a single device, have the potential to dramatically improve the mobility and restore independence to their users. However, these devices rely on the ability of their users to continuously control multiple powered lower-limb joints simultaneously. Success of such approaches rely on robust sensing of user intent and accurate mapping to device control parameters. Here, we compare two non-invasive sensing modalities: surface electromyography and sonomyography, (i.e., ultrasound imaging of skeletal muscle), as inputs to Gaussian process regression models trained to estimate hip, knee and ankle joint moments during varying forms of ambulation. Experiments were performed with ten non-disabled individuals instrumented with surface electromyography and sonomyography sensors while completing trials of level, incline (10°) and decline (10°) walking. Results suggest sonomyography of muscles on the anterior and posterior thigh can be used to estimate hip, knee and ankle joint moments more accurately than surface electromyography. Furthermore, these results can be achieved by training Gaussian process regression models in a task-independent manner; i.e., incorporating features of level and ramp walking within the same predictive framework. These findings support the integration of sonomyographic and electromyographic sensing within powered assistive devices to continuously control joint torque.

Genetic Algorithm for Feature Selection in Lower Limb Pattern Recognition

Choosing the right features is important to optimize lower limb pattern recognition, such as in prosthetic control. EMG signals are noisy in nature, which makes it more challenging to extract useful information. Many features are used in the literature, which raises the question which features are most suited for use in lower limb myoelectric control. Therefore, it is important to find combinations of best performing features. One way to achieve this is by using a genetic algorithm, a meta-heuristic capable of searching vast feature spaces. The goal of this research is to demonstrate the capabilities of a genetic algorithm and come up with a feature set that has a better performance than the state-of-the-art feature set. In this study, we collected a dataset containing ten able-bodied subjects who performed various gait-related activities while measuring EMG and kinematics. The genetic algorithm selected features based on the performance on the training partition of this dataset. The selected feature sets were evaluated on the remaining test set and on the online benchmark dataset ENABL3S, against a state-of-the-art feature set. The results show that a feature set based on the selected features of a genetic algorithm outperforms the state-of-the-art set. The overall error decreased up to 0.54% and the transitional error by 2.44%, which represent a relative decrease in overall errors up to 11.6% and transitional errors up to 14.1%, although these results were not significant. This study showed that a genetic algorithm is capable of searching a large feature space and that systematic feature selection shows promising results for lower limb myoelectric control.

Locomotion Mode Transition Prediction Based on Gait-Event Identification Using Wearable Sensors and Multilayer Perceptrons

People walk on different types of terrain daily; for instance, level-ground walking, ramp and stair ascent and descent, and stepping over obstacles are common activities in daily life. Movement patterns change as people move from one terrain to another. The prediction of transitions between locomotion modes is important for developing assistive devices, such as exoskeletons, as the optimal assistive strategies may differ for different locomotion modes. The prediction of locomotion mode transitions is often accompanied by gait-event detection that provides important information during locomotion about critical events, such as foot contact (FC) and toe off (TO). In this study, we introduce a method to integrate locomotion mode prediction and gait-event identification into one machine learning framework, comprised of two multilayer perceptrons (MLP). Input features to the framework were from fused data from wearable sensors—specifically, electromyography sensors and inertial measurement units. The first MLP successfully identified FC and TO, FC events were identified accurately, and a small number of misclassifications only occurred near TO events. A small time difference (2.5 ms and −5.3 ms for FC and TO, respectively) was found between predicted and true gait events. The second MLP correctly identified walking, ramp ascent, and ramp descent transitions with the best aggregate accuracy of 96.3%, 90.1%, and 90.6%, respectively, with sufficient prediction time prior to the critical events. The models in this study demonstrate high accuracy in predicting transitions between different locomotion modes in the same side’s mid- to late stance of the stride prior to the step into the new mode using data from EMG and IMU sensors. Our results may help assistive devices achieve smooth and seamless transitions in different locomotion modes for those with motor disorders.

A decade retrospective of medical robotics research from 2010 to 2020

[Figure: see text].

Rejecting Impulse Artifacts from Surface EMG Signals using Real-time Cumulative Histogram Filtering

This paper presents a cumulative histogram filtering (CHF) algorithm to filter impulsive artifacts within surface electromyograhy (sEMG) signal for time-domain signal feature extraction. The proposed CHF algorithm filters sEMG signals by extracting a continuous subset of amplitude-sorted values within a real-time window of measured samples using information about the probabilistic distribution of sEMG amplitude. For real-time deployment of the proposed CHF algorithm on an embedded computing platform, we also present an efficient, iterative implementation of the proposed algorithm. The proposed CHF algorithm was evaluated on synthetic impulse artifacts superimposed upon undisturbed sEMG recorded from a subject with transtibial amputation. Results suggest that the CHF algorithm effectively suppresses the simulated impulse artifacts while preserving a minimum signal-to-noise ratio of 95% and an average Pearson correlation of 0.99 compared to the undisturbed sEMG recordings.

Electromyography and Inertial Motion Sensors Based Wearable Data Acquisition System for Stroke Patients: A Pilot Study

Development of wearable data acquisition systems with applications to human-machine interaction (HMI) is of great interest to assist stroke patients or people with motor disabilities. This paper proposes a hybrid wireless data acquisition system, which combines surface electromyography (sEMG) and inertial measurement unit (IMU) sensors. It is designed to interface wrist extension with external devices, which allows the user to operate devices with hand orientations. A pilot study of the system performed on four healthy subjects has successfully produced two different control signals corresponding to wrist extensions. Preliminary results show a high correlation (0.42-0.75) between sEMG and IMU signals, thus proving the feasibility of such a system. Results also show that the developed system is robust as well as less susceptible to external interferences. The generated control signals can be used to perform real-time control of different devices in daily-life activities, such as turning ON/OFF of lights in a smart home, controlling an electric wheelchair, and other assistive devices. Such a system will help decrease the dependency of disabled people on their caretakers and empower them to perform their daily-life activities independently.

Estimation of knee and ankle angles during walking using thigh and shank angles

Estimation of joints' trajectories is commonly used in human gait analysis, and in the development of motion planners and high-level controllers for prosthetics, orthotics, exoskeletons and humanoids. Human locomotion is the result of the cooperation between leg joints and limbs. This suggests the existence of underlying relationships between them which lead to a harmonic gait. In this study we aimed to estimate knee and ankle trajectories using thigh and shank angles. To do so, an estimation approach was developed that continuously mapped the inputs to the outputs, which did not require switching rules, speed estimation, gait percent identification or look-up tables. The estimation algorithm was based on a nonlinear auto-regressive model with exogenous inputs. The method was then combined with wavelets theory, and then the two were used in a neural network. To evaluate the estimation performance, three scenarios were developed which used only one source of inputs (i.e., only shank angles or only thigh angles). First, knee anglesθ_k(outputs) were estimated using thigh anglesθ_th(inputs). Second, ankle anglesθ_a(outputs) were estimated using thigh anglesθ_sh(inputs), and third, the ankle angles were estimated using shank angles (inputs). The proposed approach was investigated for 22 subjects at different walking speeds and the leave-one-subject-out procedure was used for training and testing the estimation algorithm. Average root mean square errors were 3.9°-5.3° and 2.1°-2.3° for knee and ankle angles, respectively. Average mean absolute errors (MAEs) MAEs were 3.2°-4° and 1.7°-1.8°, and average correlation coefficientsρ_ccwere 0.95-0.98 and 0.94-0.96 for knee and ankle angles, respectively. The limitations and strengths of the proposed approach are discussed in detail and the results are compared with several studies.

Continuous Myoelectric Prediction of Future Ankle Angle and Moment Across Ambulation Conditions and Their Transitions

A hallmark of human locomotion is that it continuously adapts to changes in the environment and predictively adjusts to changes in the terrain, both of which are major challenges to lower limb amputees due to the limitations in prostheses and control algorithms. Here, the ability of a single-network nonlinear autoregressive model to continuously predict future ankle kinematics and kinetics simultaneously across ambulation conditions using lower limb surface electromyography (EMG) signals was examined. Ankle plantarflexor and dorsiflexor EMG from ten healthy young adults were mapped to normal ranges of ankle angle and ankle moment during level overground walking, stair ascent, and stair descent, including transitions between terrains (i.e., transitions to/from staircase). Prediction performance was characterized as a function of the time between current EMG/angle/moment inputs and future angle/moment model predictions (prediction interval), the number of past EMG/angle/moment input values over time (sampling window), and the number of units in the network hidden layer that minimized error between experimentally measured values (targets) and model predictions of ankle angle and moment. Ankle angle and moment predictions were robust across ambulation conditions with root mean squared errors less than 1° and 0.04 Nm/kg, respectively, and cross-correlations (R²) greater than 0.99 for prediction intervals of 58 ms. Model predictions at critical points of trip-related fall risk fell within the variability of the ankle angle and moment targets (Benjamini-Hochberg adjusted p > 0.065). EMG contribution to ankle angle and moment predictions occurred consistently across ambulation conditions and model outputs. EMG signals had the greatest impact on noncyclic regions of gait such as double limb support, transitions between terrains, and around plantarflexion and moment peaks. The use of natural muscle activation patterns to continuously predict variations in normal gait and the model's predictive capabilities to counteract electromechanical inherent delays suggest that this approach could provide robust and intuitive user-driven real-time control of a wide variety of lower limb robotic devices, including active powered ankle-foot prostheses.

Flexible Dry Electrodes for EMG Acquisition within Lower Extremity Prosthetic Sockets

Acquisition of surface electromyography (sEMG) from a person with an amputated lower extremity (LE) during prosthesis-assisted walking remains a significant challenge due to the dynamic nature of the gait cycle. Current solutions to sEMG-based neural control of active LE prostheses involve a combination of customized electrodes, prosthetic sockets, and liners. These technologies are generally: (i) incompatible with a subject's existing prosthetic socket and liners; (ii) uncomfortable to use; and (iii) expensive. This paper presents a flexible dry electrode design for sEMG acquisition within LE prosthetic sockets which seeks to address these issues. Design criteria and corresponding design decisions are explained and a proposed flexible electrode prototype is presented. Performances of the proposed electrode and commercial Ag/AgCl electrodes are compared in seated subjects without amputations. Quantitative analyses suggest comparable signal qualities for the proposed novel electrode and commercial electrodes. The proposed electrode is demonstrated in a subject with a unilateral transtibial amputation wearing her own liner, socket, and the portable sEMG processing platform in a preliminary standing and level ground walking study. Qualitative analyses suggest the feasibility of real-time sEMG data collection from load-bearing, ambulatory subjects.

Magnetomicrometry

We live in an era of wearable sensing, where our movement through the world can be continuously monitored by devices. Yet, we lack a portable sensor that can continuously monitor muscle, tendon, and bone motion, allowing us to monitor performance, deliver targeted rehabilitation, and provide intuitive, reflexive control over prostheses and exoskeletons. Here, we introduce a sensing modality, magnetomicrometry, that uses the relative positions of implanted magnetic beads to enable wireless tracking of tissue length changes. We demonstrate real-time muscle length tracking in an in vivo turkey model via chronically implanted magnetic beads while investigating accuracy, biocompatibility, and long-term implant stability. We anticipate that this tool will lay the groundwork for volitional control over wearable robots via real-time tracking of muscle lengths and speeds. Further, to inform future biomimetic control strategies, magnetomicrometry may also be used in the in vivo tracking of biological tissues to elucidate biomechanical principles of animal and human movement.

Acquisition of Surface EMG Using Flexible and Low-Profile Electrodes for Lower Extremity Neuroprosthetic Control

For persons with lower extremity (LE) amputation, acquisition of surface electromyography (sEMG) from within the prosthetic socket remains a significant challenge due to the dynamic loads experienced during the gait cycle. However, these signals are critical for both understanding the clinical effects of LE amputation and determining the desired control trajectories of active LE prostheses. Current solutions for collecting within-socket sEMG are generally (i) incompatible with a subject's prescribed prosthetic socket and liners, (ii) uncomfortable, and (iii) expensive. This study presents an alternative within-socket sEMG acquisition paradigm using a novel flexible and low-profile electrode. First, the practical performance of this Sub-Liner Interface for Prosthetics (SLIP) electrode is compared to that of commercial Ag/AgCl electrodes within a cohort of subjects without amputation. Then, the corresponding SLIP electrode sEMG acquisition paradigm is implemented in a single subject with unilateral transtibial amputation performing unconstrained movements and walking on level ground. Finally, a quantitative questionnaire characterizes subjective comfort for SLIP electrode and commercial Ag/AgCl electrode instrumentation setups. Quantitative analyses suggest comparable signal qualities between SLIP and Ag/AgCl electrodes while qualitative analyses suggest the feasibility of using the SLIP electrode for real-time sEMG data collection from load-bearing, ambulatory subjects with LE amputation.

Myoelectric control of robotic lower limb prostheses: a review of electromyography interfaces, control paradigms, challenges and future directions

Objective.Advanced robotic lower limb prostheses are mainly controlled autonomously. Although the existing control can assist cyclic movements during locomotion of amputee users, the function of these modern devices is still limited due to the lack of neuromuscular control (i.e. control based on human efferent neural signals from the central nervous system to peripheral muscles for movement production). Neuromuscular control signals can be recorded from muscles, called electromyographic (EMG) or myoelectric signals. In fact, using EMG signals for robotic lower limb prostheses control has been an emerging research topic in the field for the past decade to address novel prosthesis functionality and adaptability to different environments and task contexts. The objective of this paper is to review robotic lower limb Prosthesis control via EMG signals recorded from residual muscles in individuals with lower limb amputations.Approach.We performed a literature review on surgical techniques for enhanced EMG interfaces, EMG sensors, decoding algorithms, and control paradigms for robotic lower limb prostheses.Main results.This review highlights the promise of EMG control for enabling new functionalities in robotic lower limb prostheses, as well as the existing challenges, knowledge gaps, and opportunities on this research topic from human motor control and clinical practice perspectives.Significance.This review may guide the future collaborations among researchers in neuromechanics, neural engineering, assistive technologies, and amputee clinics in order to build and translate true bionic lower limbs to individuals with lower limb amputations for improved motor function.

SA-SVM-Based Locomotion Pattern Recognition for Exoskeleton Robot

An exoskeleton robot is a kind of wearable mechanical instrument designed according to the shape and function of the human body. The main purpose of its design and manufacture is to enhance human strength, assist human walking and to help patients recover. The walking state of the exoskeleton robot should be highly consistent with the state of the human, so the accurate locomotion pattern recognition is the premise of the flexible control of the exoskeleton robot. In this paper, a simulated annealing (SA) algorithm-based support vector machine model is proposed for the recognition of different locomotion patterns. In order to improve the overall performance of the support vector machine (SVM), the simulated annealing algorithm is adopted to obtain the optimal parameters of support vector machine. The pressure signal measured by the force sensing resistors integrated on the sole of the shoe is fused with the position and pose information measured by the inertial measurement units attached to the thigh, shank and foot, which are used as the input information of the support vector machine. The max-relevance and min-redundancy algorithm was selected for feature extraction based on the window size of 300 ms and the sampling frequency of 100 Hz. Since the signals come from different types of sensors, normalization is required to scale the input signals to the interval (0,1). In order to prevent the classifier from overfitting, five layers of cross validation are used to train the support vector machine classifier. The support vector machine model was obtained offline in MATLAB. The finite state machine is used to limit the state transition and improve the recognition accuracy. Experiments on different locomotion patterns show that the accuracy of the algorithm is 97.47% ± 1.16%. The SA-SVM method can be extended to industrial robots and rehabilitation robots.

A Muscle Synergy-Inspired Method of Detecting Human Movement Intentions Based on Wearable Sensor Fusion

Detecting human movement intentions is fundamental to neural control of robotic exoskeletons, as it is essential for achieving seamless transitions between different locomotion modes. In this study, we enhanced a muscle synergy-inspired method of locomotion mode identification by fusing the electromyography data with two types of data from wearable sensors (inertial measurement units), namely linear acceleration and angular velocity. From the finite state machine perspective, the enhanced method was used to systematically identify 2 static modes, 7 dynamic modes, and 27 transitions among them. In addition to the five broadly studied modes (level ground walking, ramps ascent/descent, stairs ascent/descent), we identified the transition between different walking speeds and modes of ramp walking at different inclination angles. Seven combinations of sensor fusion were conducted, on experimental data from 8 able-bodied adult subjects, and their classification accuracy and prediction time were compared. Prediction based on a fusion of electromyography and gyroscope (angular velocity) data predicted transitions earlier and with higher accuracy. All transitions and modes were identified with a total average classification accuracy of 94.5% with fused sensor data. For nearly all transitions, we were able to predict the next locomotion mode 300-500ms prior to the step into that mode.

Noninvasive Human-Prosthesis Interfaces for Locomotion Intent Recognition: A Review

The lower-limb robotic prostheses can provide assistance for amputees' daily activities by restoring the biomechanical functions of missing limb(s). To set proper control strategies and develop the corresponding controller for robotic prosthesis, a prosthesis user's intent must be acquired in time, which is still a major challenge and has attracted intensive attentions. This work focuses on the robotic prosthesis user's locomotion intent recognition based on the noninvasive sensing methods from the recognition task perspective (locomotion mode recognition, gait event detection, and continuous gait phase estimation) and reviews the state-of-the-art intent recognition techniques in a lower-limb prosthesis scope. The current research status, including recognition approach, progress, challenges, and future prospects in the human's intent recognition, has been reviewed. In particular for the recognition approach, the paper analyzes the recent studies and discusses the role of each element in locomotion intent recognition. This work summarizes the existing research results and problems and contributes a general framework for the intent recognition based on lower-limb prosthesis.

Locomotor activities of individuals with lower-limb amputation

BACKGROUND

Ambulatory individuals with lower-limb amputation perform a variety of locomotor activities, but the step count distribution of these activities is unknown.

OBJECTIVE

To describe a novel method for activity monitoring and to use it to count steps taken while walking straight ahead on level ground, turning right and left, up and down stairs, and up and down ramps.

STUDY DESIGN

This is an observational study.

METHODS

A portable instrument to record leg motion was placed on or inside the prosthetic pylon of 10 individuals with unilateral transtibial amputations. Participants first walked a defined course in a hospital environment to train and validate a machine learning algorithm for classifying locomotor activity. Participants were then free to pursue their usual activities while data were continuously collected over 1-2 d.

RESULTS

Overall classification accuracy was 97.5% ± 1.5%. When participants were free to walk about their home, work, and community environments, 82.8% of all steps were in a straight line, 9.0% were turning steps, 4.8% were steps on stairs, and 3.6% were steps on ramps.

CONCLUSION

A novel activity monitoring method accurately classified the locomotion activities of individuals with lower-limb amputation. Nearly 1 in 5 of all steps taken involved turning or walking on stairs and ramps.

Processing of EMG Signals with High Impact of Power Line and Cardiac Interferences

This work deals with electromyography (EMG) signal processing for the diagnosis and therapy of different muscles. Because the correct muscle activity measurement of strongly noised EMG signals is the major hurdle in medical applications, a raw measured EMG signal should be cleaned of different factors like power network interference and ECG heartbeat. Unfortunately, there are no completed studies showing full multistage signal processing of EMG recordings. In this article, the authors propose an original algorithm to perform muscle activity measurements based on raw measurements. The effectiveness of the proposed algorithm for EMG signal measurement was validated by a portable EMG system developed as a part of the EU research project and EMG raw measurement sets. Examples of removing the parasitic interferences are presented for each stage of signal processing. Finally, it is shown that the proposed processing of EMG signals enables cleaning of the EMG signal with minimal loss of the diagnostic content.

Continuous Classification of Locomotion in Response to Task Complexity and Anticipatory State

Objective

Intent recognition in lower-extremity assistive devices (e.g., prostheses and exoskeletons) is typically limited to either recognition of steady-state locomotion or changes of terrain (e.g., level ground to stair) occurring in a straight-line path and under anticipated condition. Stability is highly affected during non-steady changes of direction such as cuts especially when they are unanticipated, posing high risk of fall-related injuries. Here, we studied the influence of changes of direction and user anticipation on task recognition, and accordingly introduced classification schemes accommodating such effects.

Methods

A linear discriminant analysis (LDA) classifier continuously classified straight-line walking, sidestep/crossover cuts (single transitions), and cuts-to-stair locomotion (mixed transitions) performed under varied task anticipatory conditions. Training paradigms with varying levels of anticipated/unanticipated exposures and analysis windows of size 100–600 ms were examined.

Results

More accurate classification of anticipated relative to unanticipated tasks was observed. Including bouts of target task in the training data was necessary to improve generalization to unanticipated locomotion. Only up to two bouts of target task were sufficient to reduce errors to <20% in unanticipated mixed transitions, whereas, in single transitions and straight walking, substantial unanticipated information (i.e., five bouts) was necessary to achieve similar outcomes. Window size modifications did not have a significant influence on classification performance.

Conclusion

Adjusting the training paradigm helps to achieve classification schemes capable of adapting to changes of direction and task anticipatory state.

Significance

The findings could provide insight into developing classification schemes that can adapt to changes of direction and user anticipation. They could inform intent recognition strategies for controlling lower-limb assistive to robustly handle “unknown” circumstances, and thus deliver increased level of reliability and safety.

Online Adaptive Prediction of Human Motion Intention Based on sEMG

Accurate and reliable motion intention perception and prediction are keys to the exoskeleton control system. In this paper, a motion intention prediction algorithm based on sEMG signal is proposed to predict joint angle and heel strike time in advance. To ensure the accuracy and reliability of the prediction algorithm, the proposed method designs the sEMG feature extraction network and the online adaptation network. The feature extraction utilizes the convolution autoencoder network combined with muscle synergy characteristics to get the high-compression sEMG feature to aid motion prediction. The adaptation network ensures the proposed prediction method can still maintain a certain prediction accuracy even the sEMG signals distribution changes by adjusting some parameters of the feature extraction network and the prediction network online. Ten subjects were recruited to collect surface EMG data from nine muscles on the treadmill. The proposed prediction algorithm can predict the knee angle 101.25 ms in advance with 2.36 degrees accuracy. The proposed prediction algorithm also can predict the occurrence time of initial contact 236±9 ms in advance. Meanwhile, the proposed feature extraction method can achieve 90.71±3.42% accuracy of sEMG reconstruction and can guarantee 73.70±5.01% accuracy even when the distribution of sEMG is changed without any adjustment. The online adaptation network enhances the accuracy of sEMG reconstruction of CAE to 87.65±3.83% and decreases the angle prediction error from 4.03∘ to 2.36∘. The proposed method achieves effective motion prediction in advance and alleviates the influence caused by the non-stationary of sEMG.

Effective recognition of human lower limb jump locomotion phases based on multi-sensor information fusion and machine learning

Jump locomotion is the basic movement of human. However, no thorough research on the recognition of jump sub-phases has been carried so far. This paper aims to use multi-sensor information fusion and machine learning to recognize the human jump phase, which is crucial to the development of exoskeleton that assists jumping. The method of information fusion for sensors including sEMG, IMU, and footswitch sensor is studied. The footswitch signals are filtered by median filter. A processing method of synthesizing Euler angles into phase angle is proposed, which is beneficial to data integration. The jump locomotion is creatively segmented into five phases. The onset and offset of active segment are detected by sample entropy of sEMG and standard deviation of acceleration signal. The features are extracted from analysis windows using multi-sensor information fusion, and the dimension of feature matrix is selected. By comparing the performances of state-of-the-art machine learning classifiers, feature subsets of sEMG, IMU, and footswitch signals are selected from time domain features in a series of analysis window parameters. The average recognition accuracy of sEMG and IMU is 91.76% and 97.68%, respectively. When using the combination of sEMG, IMU, and footswitch signals, the average accuracy is 98.70%, which outperforms the combination of sEMG and IMU (97.97%, p < 0.01). Graphical Abstract The sub-phases of human locomotion are recognized based on multi-sensor information fusion and machine learning method. The feature data of the sub-phases is visualized in 3-dimensional space. The predicted states and the true states in a complete jump are compared along the time axis.

A Machine Learning Strategy for Locomotion Classification and Parameter Estimation Using Fusion of Wearable Sensors

The accurate classification of ambulation modes and estimation of walking parameters is a challenging problem that is key to many applications. Knowledge of the user's state can enable rehabilitative devices to adapt to changing conditions, while in a clinical setting it can provide physicians with more detailed patient activity information. This study describes the development and optimization process of a combined locomotion mode classifier and environmental parameter estimator using machine learning and wearable sensors. A detailed analysis of the best sensor types and placements for each problem is also presented to provide device designers with information on which sensors to prioritize for their application. For this study, 15 able-bodied subjects were unilaterally instrumented with inertial measurement unit, goniometer, and electromyography sensors and data were collected for extensive ranges of level-ground, ramp, and stair walking conditions. The proposed system classifies steady state ambulation modes with 99% accuracy and ambulation mode transitions with 96% accuracy, along with estimating ramp incline within 1.25 degrees, stair height within 1.29 centimeters, and walking speed within 0.04 meters per second. Mechanical sensors (inertial measurement units, goniometers) are found to be most important for classification, while goniometers dominate ramp incline and stair height estimation, and speed estimation is performed largely with a single inertial measurement unit. The feature tables and Matlab code to replicate the study are published as supplemental materials.

Machine Learning Approaches for Activity Recognition and/or Activity Prediction in Locomotion Assistive Devices-A Systematic Review

Locomotion assistive devices equipped with a microprocessor can potentially automatically adapt their behavior when the user is transitioning from one locomotion mode to another. Many developments in the field have come from machine learning driven controllers on locomotion assistive devices that recognize/predict the current locomotion mode or the upcoming one. This review synthesizes the machine learning algorithms designed to recognize or to predict a locomotion mode in order to automatically adapt the behavior of a locomotion assistive device. A systematic review was conducted on the Web of Science and MEDLINE databases (as well as in the retrieved papers) to identify articles published between 1 January 2000 to 31 July 2020. This systematic review is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines and is registered on Prospero (CRD42020149352). Study characteristics, sensors and algorithms used, accuracy and robustness were also summarized. In total, 1343 records were identified and 58 studies were included in this review. The experimental condition which was most often investigated was level ground walking along with stair and ramp ascent/descent activities. The machine learning algorithms implemented in the included studies reached global mean accuracies of around 90%. However, the robustness of those algorithms seems to be more broadly evaluated, notably, in everyday life. We also propose some guidelines for homogenizing future reports.

Advanced techniques in amputation surgery and prosthetic technology in the lower extremity

Bone-anchored implants give patients with unmanageable stump problems hope for drastic improvements in function and quality of life and are therefore increasingly considered a viable solution for lower-limb amputees and their orthopaedic surgeons, despite high infection rates.Regarding diversity and increasing numbers of implants worldwide, efforts are to be supported to arrange an international bone-anchored implant register to transparently overview pros and cons.Due to few, but high-quality, articles about the beneficial effects of targeted muscle innervation (TMR) and regenerative peripheral nerve interface (RPNI), these surgical techniques ought to be directly transferred into clinical protocols, observations and routines.Bionics of the lower extremity is an emerging cutting-edge technology. The main goal lies in the reduction of recognition and classification errors in changes of ambulant modes. Agonist-antagonist myoneuronal interfaces may be a most promising start in controlling of actively powered ankle joints.As advanced amputation surgical techniques are becoming part of clinical routine, the development of financing strategies besides medical strategies ought to be boosted, leading to cutting-edge technology at an affordable price.Microprocessor-controlled components are broadly available, and amputees do see benefits. Devices from different manufacturers differ in gait kinematics with huge inter-individual varieties between amputees that cannot be explained by age. Active microprocessor-controlled knees/ankles (A-MPK/As) might succeed in uneven ground-walking. Patients ought to be supported to receive appropriate prosthetic components to reach their everyday goals in a desirable way.Increased funding of research in the field of prosthetic technology could enhance more high-quality research in order to generate a high level of evidence and to identify individuals who can profit most from microprocessor-controlled prosthetic components. Cite this article: EFORT Open Rev 2020;5:724-741. DOI: 10.1302/2058-5241.5.190070.

Ultrasound Sensing Can Improve Continuous Classification of Discrete Ambulation Modes Compared to Surface Electromyography

Clinical translation of "intelligent" lower-limb assistive technologies relies on robust control interfaces capable of accurately detecting user intent. To date, mechanical sensors and surface electromyography (EMG) have been the primary sensing modalities used to classify ambulation. Ultrasound (US) imaging can be used to detect user-intent by characterizing structural changes of muscle. Our study evaluates wearable US imaging as a new sensing modality for continuous classification of five discrete ambulation modes: level, incline, decline, stair ascent, and stair descent ambulation, and benchmarks performance relative to EMG sensing. Ten able-bodied subjects were equipped with a wearable US scanner and eight unilateral EMG sensors. Time-intensity features were recorded from US images of three thigh muscles. Features from sliding windows of EMG signals were analyzed in two configurations: one including 5 EMG sensors on muscles around the thigh, and another with 3 additional sensors placed on the shank. Linear discriminate analysis was implemented to continuously classify these phase-dependent features of each sensing modality as one of five ambulation modes. US-based sensing statistically improved mean classification accuracy to 99.8% (99.5-100% CI) compared to 8-EMG sensors (85.8%; 84.0-87.6% CI) and 5-EMG sensors (75.3%; 74.5-76.1% CI). Further, separability analyses show the importance of superficial and deep US information for stair classification relative to other modes. These results are the first to demonstrate the ability of US-based sensing to classify discrete ambulation modes, highlighting the potential for improved assistive device control using less widespread, less superficial and higher resolution sensing of skeletal muscle.

On-board Training Strategy for IMU-Based Real-Time Locomotion Recognition of Transtibial Amputees With Robotic Prostheses

The paper puts forward an on-board strategy for a training model and develops a real-time human locomotion mode recognition study based on a trained model utilizing two inertial measurement units (IMUs) of robotic transtibial prosthesis. Three transtibial amputees were recruited as subjects in this study to finish five locomotion modes (level ground walking, stair ascending, stair descending, ramp ascending, and ramp descending) with robotic prostheses. An interaction interface was designed to collect sensors' data and instruct to train model and recognition. In this study, the analysis of variance ratio (no more than 0.05) reflects the good repeatability of gait. The on-board training time for SVM (Support Vector Machines), QDA (Quadratic Discriminant Analysis), and LDA (Linear discriminant analysis) are 89, 25, and 10 s based on a 10,000 × 80 training data set, respectively. It costs about 13.4, 5.36, and 0.067 ms for SVM, QDA, and LDA for each recognition process. Taking the recognition accuracy of some previous studies and time consumption into consideration, we choose QDA for real-time recognition study. The real-time recognition accuracies are 97.19 ± 0.36% based on QDA, and we can achieve more than 95% recognition accuracy for each locomotion mode. The receiver operating characteristic also shows the good quality of QDA classifiers. This study provides a preliminary interaction design for human-machine prosthetics in future clinical application. This study just adopts two IMUs not multi-type sensors fusion to improve the integration and wearing convenience, and it maintains comparable recognition accuracy with multi-type sensors fusion at the same time.

A Wearable Bio-signal Processing System with Ultra-low-power SoC and Collaborative Neural Network Classifier for Low Dimensional Data Communication

In this paper, a real time physiological signal classification system with an integrated ultra-low power collaborative neural network classifier is presented. The developed system includes a specially designed system-on-chip (SoC) and a wireless communication module that transmits classification results to a smartphone app as a convenient user interface in real-time training. The customized SoC provides ultra-low-power and low-latency sensing and classification on physiological signals, e.g. EMG and ECG. A special collaborative neural network classifier was implemented to allow multiple chips to collaborate on classification. As a result, only low dimensional data is being transmitted over the network, significantly reducing data communication across multiple modules. A demonstration of EMG based gesture classification shows 1100X less power consumption from the developed SoC compared with conventional embedded solutions. The transmission of only low dimensional data from the collaborative neural network classifier leads to a 50X reduction of data communication and associated energy for multiple sensing cites.

EMG-Centered Multisensory Based Technologies for Pattern Recognition in Rehabilitation: State of the Art and Challenges

In the field of rehabilitation, the electromyography (EMG) signal plays an important role in interpreting patients' intentions and physical conditions. Nevertheless, utilizing merely the EMG signal suffers from difficulty in recognizing slight body movements, and the detection accuracy is strongly influenced by environmental factors. To address the above issues, multisensory integration-based EMG pattern recognition (PR) techniques have been developed in recent years, and fruitful results have been demonstrated in diverse rehabilitation scenarios, such as achieving high locomotion detection and prosthesis control accuracy. Owing to the importance and rapid development of the EMG centered multisensory fusion technologies in rehabilitation, this paper reviews both theories and applications in this emerging field. The principle of EMG signal generation and the current pattern recognition process are explained in detail, including signal preprocessing, feature extraction, classification algorithms, etc. Mechanisms of collaborations between two important multisensory fusion strategies (kinetic and kinematics) and EMG information are thoroughly explained; corresponding applications are studied, and the pros and cons are discussed. Finally, the main challenges in EMG centered multisensory pattern recognition are discussed, and a future research direction of this area is prospected.

Use of Sonomyographic Sensing to Estimate Knee Angular Velocity During Varying Modes of Ambulation

Ultrasound (US) imaging of muscle has been introduced as a promising sensing modality for assistive device control. Ten able-bodied subjects completed level, incline and decline walking on a treadmill in a motion capture laboratory while wearing reflective markers on upper- and lower-body. A wearable US transducer was affixed to subjects' anterior thigh, and time-intensity features were extracted from transverse US images of the knee extensor muscles. These features were used to train and test Gaussian process regression models for continuous estimation of knee flexion/extension angular velocity. Four regression models were evaluated: (1) subject-dependent/task-specific, (2) subject-dependent/pooled-tasks, (3) subject-independent/task-specific, and (4) subject-independent/pooled-tasks. Subject-independent models were "tuned" with up to six strides of the test subject's data to boost performance. A two-factor analysis of variance test was used to assess the effect of each approach on root mean square error (RMSE) of estimated knee angular velocity (α=0.05). Statistical parametric mapping (SPM) was completed to compare actual vs. estimated knee angular velocity as a function of the gait cycle (α=0.05). For incline and level walking, the subject-dependent/pooled-tasks model resulted in the lowest error while the subject-dependent/task-specific model resulted in the lowest error for decline walk. Impressively, the two-factor test revealed no difference between task-specific and pooled-task models. Furthermore, despite capturing many important features of knee velocity across individuals there were, as expected, significant differences between subject-dependent and subject-independent models. Collectively, these results are promising for potential assistive device control with error rates <10% for all regression models that were tested.Clinical Relevance-This work is the first study to demonstrate the feasibility of using ultrasound-based sensing for estimation of knee angular velocity during multiple modes of ambulation.

A real-time walking pattern recognition method for soft knee power assist wear

Real-time recognition of walking-related activities is an important function that lower extremity assistive devices should possess. This article presents a real-time walking pattern recognition method for soft knee power assist wear. The recognition method employs the rotation angles of thighs and shanks as well as the knee joint angles collected by the inertial measurement units as input signals and adopts the rule-based classification algorithm to achieve the real-time recognition of three most common walking patterns, that is, level-ground walking, stair ascent, and stair descent. To evaluate the recognition performance, 18 subjects are recruited in the experiments. During the experiments, subjects wear the knee power assist wear and carry out a series of walking activities in an out-of-lab scenario. The results show that the average recognition accuracy of three walking patterns reaches 98.2%, and the average recognition delay of all transitions is slightly less than one step.

Accurate Heuristic Terrain Prediction in Powered Lower-Limb Prostheses Using Onboard Sensors

OBJECTIVE

This study describes the development and offline validation of a heuristic algorithm for accurate prediction of ground terrain in a lower limb prosthesis. This method is based on inference of the ground terrain geometry using estimation of prosthetic limb kinematics during gait with a single integrated inertial measurement unit.

METHODS

We asked five subjects with below-knee amputations to traverse level ground, stairs, and ramps using a high-range-of-motion powered prosthesis while internal sensor data were remotely logged. We used these data to develop three terrain prediction algorithms. The first two employed state-of-the-art machine learning approaches, while the third was a directly tuned heuristic using thresholds on estimated prosthetic ankle joint translations and ground slope. We compared the performance of these algorithms using resubstitution error for the machine learning algorithms and overall error for the heuristic algorithm.

RESULTS

Our optimal machine learning algorithm attained a resubstitution error of 3.4% using 45 features, while our heuristic method attained an overall prediction error of 2.8% using only 5 features derived from estimation of ground slope and horizontal and vertical ankle joint displacement. Compared with pattern recognition, the heuristic performed better on each individual subject, and across both level and non-level strides.

CONCLUSION AND SIGNIFICANCE

These results demonstrate a method for heuristic prediction of ground terrain in a powered prosthesis. The method is more accurate, more interpretable, and less computationally expensive than machine learning methods considered state-of-the-art for intent recognition, and relies only on integrated prosthesis sensors. Finally, the method provides intuitively tunable thresholds to improve performance for specific walking conditions.