Proprioceptive Sonomyographic Control: A novel method for intuitive and proportional control of multiple degrees-of-freedom for individuals with upper extremity limb loss

Sonomyography for Control of Upper-Limb Prostheses: Current State and Future Directions

Problem Statement

Despite the recent advancements in technology, many individuals with upper-limb loss struggle to achieve stable control over multiple degrees of freedom in a prosthesis. There is an ongoing need to develop noninvasive prosthesis control modalities that could improve functional patient outcomes.

Proposed Solution

Ultrasound-based sensing of muscle deformation, known as sonomyography, is an emerging sensing modality for upper-limb prosthesis control with the potential to significantly improve functionality. Sonomyography enables spatiotemporal characterization of both superficial and deep muscle activity, making it possible to distinguish the contributions of individual muscles during functional movements and derive a large set of independent prosthesis control signals. Using sonomyography to control a prosthesis has shown great promise in the research literature but has not yet been fully adapted for clinical use. This article describes the implementation of sonomyography for upper-limb prosthesis control, ongoing technological development, considerations for deploying this technology in clinical settings, and recommendations for future study.

Clinical Relevance

Sonomyography may soon become a clinically viable modality for upper-limb prosthesis control that could offer prosthetists an additional solution when selecting optimal treatment plans for their patients.

Clinical, Safety, and Engineering Perspectives on Wearable Ultrasound Technology: A Review

Wearable ultrasound has the potential to become a disruptive technology enabling new applications not only in traditional clinical settings, but also in settings where ultrasound is not currently used. Understanding the basic engineering principles and limitations of wearable ultrasound is critical for clinicians, scientists, and engineers to advance potential applications and translate the technology from bench to bedside. Wearable ultrasound devices, especially monitoring devices, have the potential to apply acoustic energy to the body for far longer durations than conventional diagnostic ultrasound systems. Thus, bioeffects associated with prolonged acoustic exposure as well as skin health need to be carefully considered for wearable ultrasound devices. This article reviews emerging clinical applications, safety considerations, and future engineering and clinical research directions for wearable ultrasound technology.

A Sonomyography-Based Muscle Computer Interface for Individuals With Spinal Cord Injury

Impairment of hand functions in individuals with spinal cord injury (SCI) severely disrupts activities of daily living. Recent advances have enabled rehabilitation assisted by robotic devices to augment the residual function of the muscles. Traditionally, electromyography-based muscle activity sensing interfaces have been utilized to sense volitional motor intent to drive robotic assistive devices. However, the dexterity and fidelity of control that can be achieved with electromyography-based control have been limited due to inherent limitations in signal quality. We have developed and tested a muscle-computer interface (MCI) utilizing sonomyography to provide control of a virtual cursor for individuals with motor-incomplete spinal cord injury. We demonstrate that individuals with SCI successfully gained control of a virtual cursor by utilizing contractions of muscles of the wrist joint. The sonomyography-based interface enabled control of the cursor at multiple graded levels demonstrating the ability to achieve accurate and stable endpoint control. Our sonomyography-based muscle-computer interface can enable dexterous control of upper-extremity assistive devices for individuals with motor-incomplete SCI.

High Performance Wearable Ultrasound as a Human-Machine Interface for Wrist and Hand Kinematic Tracking

OBJECTIVE

Non-invasive human machine interfaces (HMIs) have high potential in medical, entertainment, and industrial applications. Traditionally, surface electromyography (sEMG) has been used to track muscular activity and infer motor intention. Ultrasound (US) has received increasing attention as an alternative to sEMG-based HMIs. Here, we developed a portable US armband system with 24 channels and a multiple receiver approach, and compared it with existing sEMG- and US-based HMIs on movement intention decoding.

METHODS

US and motion capture data was recorded while participants performed wrist and hand movements of four degrees of freedom (DoFs) and their combinations. A linear regression model was used to offline predict hand kinematics from the US (or sEMG, for comparison) features. The method was further validated in real-time for a 3-DoF target reaching task.

RESULTS

In the offline analysis, the wearable US system achieved an average [Formula: see text] of 0.94 in the prediction of four DoFs of the wrist and hand while sEMG reached a performance of [Formula: see text]= 0.60. In online control, the participants achieved an average 93% completion rate of the targets.

CONCLUSION

When tailored for HMIs, the proposed US A-mode system and processing pipeline can successfully regress hand kinematics both in offline and online settings with performances comparable or superior to previously published interfaces.

SIGNIFICANCE

Wearable US technology may provide a new generation of HMIs that use muscular deformation to estimate limb movements. The wearable US system allowed for robust proportional and simultaneous control over multiple DoFs in both offline and online settings.

Moving a missing hand: children born with below elbow deficiency can enact hand grasp patterns with their residual muscles

Children with a unilateral congenital below elbow deficiency (UCBED) have one typical upper limb and one that lacks a hand, ending below the elbow at the proximal/mid forearm. UCBED is an isolated condition, and affected children otherwise develop normal sensorimotor control. Unlike adults with upper limb absence, the majority of whom have an acquired loss, children with UCBED never developed a hand, so their residual muscles have never actuated an intact limb. Their ability to purposefully modulate affected muscle activity is often assumed to be limited, and this assumption has influenced prosthetic design and prescription practices for this population as many modern devices derive control signals from affected muscle activity. To better understand the motor capabilities of the affected muscles, we used ultrasound imaging to study 6 children with UCBED. We examined the extent to which subjects activate their affected muscles when performing mirrored movements with their typical and missing hands. We demonstrate that all subjects could intentionally and consistently enact at least five distinct muscle patterns when attempting different missing hand movements (e.g., power grasp) and found similar performance across affected and typically developed limbs. These results suggest that although participants had never actuated the missing hand they could distinctively and consistently activate the residual muscle patterns associated with actions on the unaffected side. These findings indicate that motor control still develops in the absence of the normal effector, and can serve as a guide for developing prostheses that leverage the full extent of these children's motor control capabilities.

Using principles of motor control to analyze performance of human machine interfaces

There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of an end-effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.

A Comparison of Approaches for Segmenting the Reaching and Targeting Motion Primitives in Functional Upper Extremity Reaching Tasks

There is growing interest in the kinematic analysis of human functional upper extremity movement (FUEM) for applications such as health monitoring and rehabilitation. Deconstructing functional movements into activities, actions, and primitives is a necessary procedure for many of these kinematic analyses. Advances in machine learning have led to progress in human activity and action recognition. However, their utility for analyzing the FUEM primitives of reaching and targeting during reach-to-grasp and reach-to-point tasks remains limited. Domain experts use a variety of methods for segmenting the reaching and targeting motion primitives, such as kinematic thresholds, with no consensus on what methods are best to use. Additionally, current studies are small enough that segmentation results can be manually inspected for correctness. As interest in FUEM kinematic analysis expands, such as in the clinic, the amount of data needing segmentation will likely exceed the capacity of existing segmentation workflows used in research laboratories, requiring new methods and workflows for making segmentation less cumbersome. This paper investigates five reaching and targeting motion primitive segmentation methods in two different domains (haptics simulation and real world) and how to evaluate these methods. This work finds that most of the segmentation methods evaluated perform reasonably well given current limitations in our ability to evaluate segmentation results. Furthermore, we propose a method to automatically identify potentially incorrect segmentation results for further review by the human evaluator. Clinical impact: This work supports efforts to automate aspects of processing upper extremity kinematic data used to evaluate reaching and grasping, which will be necessary for more widespread usage in clinical settings.

Spatially repeatable components from ultrafast ultrasound are associated with motor unit activity in human isometric contractions. J Neural Eng 2023;20:046016. [PMID: 37437598 DOI: 10.1088/1741-2552/ace6fc]

Objective.Ultrafast ultrasound (UUS) imaging has been used to detect intramuscular mechanical dynamics associated with single motor units (MUs). Detecting MUs from ultrasound sequences requires decomposing a velocity field into components, each consisting of an image and a signal. These components can be associated with putative MU activity or spurious movements (noise). The differentiation between putative MUs and noise has been accomplished by comparing the signals with MU firings obtained from needle electromyography (EMG). Here, we examined whether the repeatability of the images over brief time intervals can serve as a criterion for distinguishing putative MUs from noise in low-force isometric contractions.Approach.UUS images and high-density surface EMG (HDsEMG) were recorded simultaneously from 99 MUs in the biceps brachii of five healthy subjects. The MUs identified through HDsEMG decomposition were used as a reference to assess the outcomes of the ultrasound-based components. For each contraction, velocity sequences from the same eight-second ultrasound recording were separated into consecutive two-second epochs and decomposed. To evaluate the repeatability of components' images across epochs, we calculated the Jaccard similarity coefficient (JSC). JSC compares the similarity between two images providing values between 0 and 1. Finally, the association between the components and the MUs from HDsEMG was assessed.Main results.All the MU-matched components had JSC > 0.38, indicating they were repeatable and accounted for about one-third of the HDsEMG-detected MUs (1.8 ± 1.6 matches over 4.9 ± 1.8 MUs). The repeatable components (JSC > 0.38) represented 14% of the total components (6.5 ± 3.3 components). These findings align with our hypothesis that intra-sequence repeatability can differentiate putative MUs from noise and can be used for data reduction.Significance.This study provides the foundation for developing stand-alone methods to identify MU in UUS sequences and towards real-time imaging of MUs. These methods are relevant for studying muscle neuromechanics and designing novel neural interfaces.

A Prototype System for High Frame Rate Ultrasound Imaging based Prosthetic Arm Control

The creation of unique control methods for a hand prosthesis is still a problem that has to be addressed. The best choice of a human-machine interface (HMI) that should be used to enable natural control is still a challenge. Surface electromyography (sEMG), the most popular option, has a variety of difficult-to-fix issues (electrode displacement, sweat, fatigue). The ultrasound imaging-based methodology offers a means of recognising complex muscle activity and configuration with a greater SNR and less hardware requirements as compared to sEMG. In this study, a prototype system for high frame rate ultrasound imaging for prosthetic arm control is proposed. Using the proposed framework, a virtual robotic hand simulation is developed that can mimic a human hand as illustrated in the link: https://youtu.be/LBcwQ0xzQK0. The proposed classification model simulating four hand gestures has a classification accuracy of more than 90%.Clinical relevance-The proposed system enables an ultrasound imaging based human machine interface that can be a research and development platform for novel control strategies of a hand prosthesis.

Using Principles of Motor Control to Analyze Performance of Human Machine Interfaces

There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of a end effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.

Controlling Upper Limb Prostheses Using Sonomyography (SMG): A Review

This paper presents a critical review and comparison of the results of recently published studies in the fields of human-machine interface and the use of sonomyography (SMG) for the control of upper limb prothesis. For this review paper, a combination of the keywords "Human Machine Interface", "Sonomyography", "Ultrasound", "Upper Limb Prosthesis", "Artificial Intelligence", and "Non-Invasive Sensors" was used to search for articles on Google Scholar and PubMed. Sixty-one articles were found, of which fifty-nine were used in this review. For a comparison of the different ultrasound modes, feature extraction methods, and machine learning algorithms, 16 articles were used. Various modes of ultrasound devices for prosthetic control, various machine learning algorithms for classifying different hand gestures, and various feature extraction methods for increasing the accuracy of artificial intelligence used in their controlling systems are reviewed in this article. The results of the review article show that ultrasound sensing has the potential to be used as a viable human-machine interface in order to control bionic hands with multiple degrees of freedom. Moreover, different hand gestures can be classified by different machine learning algorithms trained with extracted features from collected data with an accuracy of around 95%.

Multi-Sensing Techniques with Ultrasound for Musculoskeletal Assessment: A Review

The study of muscle contractions generated by the muscle-tendon unit (MTU) plays a critical role in medical diagnoses, monitoring, rehabilitation, and functional assessments, including the potential for movement prediction modeling used for prosthetic control. Over the last decade, the use of combined traditional techniques to quantify information about the muscle condition that is correlated to neuromuscular electrical activation and the generation of muscle force and vibration has grown. The purpose of this review is to guide the reader to relevant works in different applications of ultrasound imaging in combination with other techniques for the characterization of biological signals. Several research groups have been using multi-sensing systems to carry out specific studies in the health area. We can divide these studies into two categories: human-machine interface (HMI), in which sensors are used to capture critical information to control computerized prostheses and/or robotic actuators, and physiological study, where sensors are used to investigate a hypothesis and/or a clinical diagnosis. In addition, the relevance, challenges, and expectations for future work are discussed.

A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control

Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging's region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ( < .001) and increased the prediction coefficient of determination by 20.13% ( < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.

Clinical implementation of a bionic hand controlled with kineticomyographic signals

Sensing the proper signal could be a vital piece of the solution to the much evading attributes of prosthetic hands, such as robustness to noise, ease of connectivity, and intuitive movement. Towards this end, magnetics tags have been recently suggested as an alternative sensing mechanism to the more common EMG signals. Such sensing technology, however, is inherently invasive and hence only in simulation stages of magnet localization to date. Here, for the first time, we report on the clinical implementation of implanted magnetic tags for an amputee's prosthetic hand from both the medical and engineering perspectives. Specifically, the proposed approach introduces a flexor-extensor tendon transfer surgical procedure to implant the tags, artificial neural networks to extract human intention directly from the implanted magnet's magnetic fields -in short KineticoMyoGraphy (KMG) signals- rather than localizing them, and a game strategy to examine the proposed algorithms and rehabilitate the patient with his new prosthetic hand. The bionic hand's ability is then tested following the patient's intended gesture type and grade. The statistical results confirm the possible utility of surgically implanted magnetic tags as an accurate sensing interface for recognizing the intended gesture and degree of movement between an amputee and his bionic hand.

Fused ultrasound and electromyography-driven neuromuscular model to improve plantarflexion moment prediction across walking speeds

Background

Improving the prediction ability of a human-machine interface (HMI) is critical to accomplish a bio-inspired or model-based control strategy for rehabilitation interventions, which are of increased interest to assist limb function post neurological injuries. A fundamental role of the HMI is to accurately predict human intent by mapping signals from a mechanical sensor or surface electromyography (sEMG) sensor. These sensors are limited to measuring the resulting limb force or movement or the neural signal evoking the force. As the intermediate mapping in the HMI also depends on muscle contractility, a motivation exists to include architectural features of the muscle as surrogates of dynamic muscle movement, thus further improving the HMI’s prediction accuracy.

Objective

The purpose of this study is to investigate a non-invasive sEMG and ultrasound (US) imaging-driven Hill-type neuromuscular model (HNM) for net ankle joint plantarflexion moment prediction. We hypothesize that the fusion of signals from sEMG and US imaging results in a more accurate net plantarflexion moment prediction than sole sEMG or US imaging.

Methods

Ten young non-disabled participants walked on a treadmill at speeds of 0.50, 0.75, 1.00, 1.25, and 1.50 m/s. The proposed HNM consists of two muscle-tendon units. The muscle activation for each unit was calculated as a weighted summation of the normalized sEMG signal and normalized muscle thickness signal from US imaging. The HNM calibration was performed under both single-speed mode and inter-speed mode, and then the calibrated HNM was validated across all walking speeds.

Results

On average, the normalized moment prediction root mean square error was reduced by 14.58 % (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.012$$\end{document}p=0.012) and 36.79 % (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.001$$\end{document}p<0.001) with the proposed HNM when compared to sEMG-driven and US imaging-driven HNMs, respectively. Also, the calibrated models with data from the inter-speed mode were more robust than those from single-speed modes for the moment prediction.

Conclusions

The proposed sEMG-US imaging-driven HNM can significantly improve the net plantarflexion moment prediction accuracy across multiple walking speeds. The findings imply that the proposed HNM can be potentially used in bio-inspired control strategies for rehabilitative devices due to its superior prediction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01061-z.

A Multi-User Transradial Functional-Test Socket for Validation of New Myoelectric Prosthetic Control Strategies

The validation of myoelectric prosthetic control strategies for individuals experiencing upper-limb loss is hindered by the time and cost affiliated with traditional custom-fabricated sockets. Consequently, researchers often rely upon virtual reality or robotic arms to validate novel control strategies, which limits end-user involvement. Prosthetists fabricate diagnostic check sockets to assess and refine socket fit, but these clinical techniques are not readily available to researchers and are not intended to assess functionality for control strategies. Here we present a multi-user, low-cost, transradial, functional-test socket for short-term research use that can be custom-fit and donned rapidly, used in conjunction with various electromyography configurations, and adapted for use with various residual limbs and terminal devices. In this study, participants with upper-limb amputation completed functional tasks in physical and virtual environments both with and without the socket, and they reported on their perceived comfort level over time. The functional-test socket was fabricated prior to participants' arrival, iteratively fitted by the researchers within 10 mins, and donned in under 1 min (excluding electrode placement, which will vary for different use cases). It accommodated multiple individuals and terminal devices and had a total cost of materials under $10 USD. Across all participants, the socket did not significantly impede functional task performance or reduce the electromyography signal-to-noise ratio. The socket was rated as comfortable enough for at least 2 h of use, though it was expectedly perceived as less comfortable than a clinically-prescribed daily-use socket. The development of this multi-user, transradial, functional-test socket constitutes an important step toward increased end-user participation in advanced myoelectric prosthetic research. The socket design has been open-sourced and is available for other researchers.

A review of upper limb pediatric prostheses and perspectives on future advancements

Many complex factors affect whether a child with a congenital upper limb deficiency will wear a prosthetic limb. Ultimately, for a child to wear and use their prosthesis, it must facilitate the effective performance of daily tasks and promote healthy social interactions. Although numerous pediatric devices are available, most provide a single open-close grasp (if a grasping function is available at all) and often offer nonanthropomorphic appearances, falling short of meeting these criteria. In this narrative review, we provide a critical assessment of the state of upper limb prostheses for children. We summarize literature using quality of life measures and categorize driving factors affecting prosthesis use into two main groupings: psychosocial and physical functioning. We define psychosocial functioning as factors related to social inclusion/exclusion, emotional function, independence, and school functioning. Physical functioning is defined as factors associated with the physical use of a prosthesis. The reviewed literature suggests that psychosocial domains of quality of life may be influenced by a congenital limb deficiency, and currently available prostheses provide little benefit in the physical functioning domains. Finally, we discuss technological advancements in adult prostheses that have yet to be leveraged for pediatric devices, including describing recently developed adult electric hands that may improve physical functioning through multiple grasping configurations and provide more hand-like cosmesis. We outline actions necessary to translate similar technologies for children and discuss further strategies to begin removing barriers to pediatric device adoption.

Classification Performance and Feature Space Characteristics in Individuals With Upper Limb Loss Using Sonomyography

Objective: Sonomyography, or ultrasound-based sensing of muscle deformation, is an emerging modality for upper limb prosthesis control. Although prior studies have shown that individuals with upper limb loss can achieve successful motion classification with sonomyography, it is important to better understand the time-course over which proficiency develops. In this study, we characterized user performance during their initial and subsequent exposures to sonomyography. Method: Ultrasound images corresponding to a series of hand gestures were collected from individuals with transradial limb loss under three scenarios: during their initial exposure to sonomyography (Experiment 1), during a subsequent exposure to sonomyography where they were provided biofeedback as part of a training protocol (Experiment 2), and during testing sessions held on different days (Experiment 3). User performance was characterized by offline classification accuracy, as well as metrics describing the consistency and separability of the sonomyography signal patterns in feature space. Results: Classification accuracy was high during initial exposure to sonomyography (96.2 ± 5.9%) and did not systematically change with the provision of biofeedback or on different days. Despite this stable classification performance, some of the feature space metrics changed. Conclusions: User performance was strong upon their initial exposure to sonomyography and did not improve with subsequent exposure. Clinical Impact: Prosthetists may be able to quickly assess if a patient will be successful with sonomyography without submitting them to an extensive training protocol, leading to earlier socket fabrication and delivery.

Monitoring at-home prosthesis control improvements through real-time data logging

OBJECTIVE

Validating the ability for advanced prostheses to improve function beyond the laboratory remains a critical step in enabling long-term benefits for prosthetic limb users.

APPROACH

A nine week take-home case study was completed with a single participant with upper limb amputation and osseointegration (OI) to better understand how an advanced prosthesis is used during daily activities. The participant was already an expert prosthesis user and used the Modular Prosthetic Limb (MPL) at home during the study. The MPL was controlled using wireless electromyography (EMG) pattern recognition-based movement decoding. Clinical assessments were performed before and after the take-home portion of the study. Data was recorded using an onboard data log in order to measure daily prosthesis usage, sensor data, and EMG data.

MAIN RESULT

The participant's continuous prosthesis usage steadily increased (p = 0.04, max = 5.5 hr) over time and over 30% of the total time was spent actively controlling the prosthesis. The duration of prosthesis usage after each pattern recognition training session also increased over time (p = 0.04), resulting in up to 5.4 hr of usage before retraining the movement decoding algorithm. Pattern recognition control accuracy improved (1.2% per week, p < 0.001) with a maximum number of 10 classes trained at once and the transitions between different degrees of freedom increased as the study progressed, indicating smooth and efficient control of the advanced prosthesis. Variability of decoding accuracy also decreased with prosthesis usage (p < 0.001) and 30% of the time was spent performing a prosthesis movement. During clinical evaluations, Box and Blocks and the Assessment of the Capacity for Myoelectric Control (ACMC) scores increased by 43% and 6.2%, respectively, demonstrating prosthesis functionality and the NASA Task Load Index (NASA-TLX) scores decreased, on average, by 25% across assessments, indicating reduced cognitive workload while using the MPL, over the nine week study.

SIGNIFICANCE

In this case study, we demonstrate that an onboard system to monitor prosthesis usage enables better understanding of how prostheses are incorporated into daily life. That knowledge can support the long-term goal of completely restoring independence and quality of life to individuals living with upper limb amputation.

First Demonstration of Functional Task Performance Using a Sonomyographic Prosthesis: A Case Study

Ultrasound-based sensing of muscle deformation, known as sonomyography, has shown promise for accurately classifying the intended hand grasps of individuals with upper limb loss in offline settings. Building upon this previous work, we present the first demonstration of real-time prosthetic hand control using sonomyography to perform functional tasks. An individual with congenital bilateral limb absence was fitted with sockets containing a low-profile ultrasound transducer placed over forearm muscle tissue in the residual limbs. A classifier was trained using linear discriminant analysis to recognize ultrasound images of muscle contractions for three discrete hand configurations (rest, tripod grasp, index finger point) under a variety of arm positions designed to cover the reachable workspace. A prosthetic hand mounted to the socket was then controlled using this classifier. Using this real-time sonomyographic control, the participant was able to complete three functional tasks that required selecting different hand grasps in order to grasp and move one-inch wooden blocks over a broad range of arm positions. Additionally, these tests were successfully repeated without retraining the classifier across 3 hours of prosthesis use and following simulated donning and doffing of the socket. This study supports the feasibility of using sonomyography to control upper limb prostheses in real-world applications.

Evaluating Electromyography and Sonomyography Sensor Fusion to Estimate Lower-Limb Kinematics Using Gaussian Process Regression

Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

A Wearable, Multi-Frequency Device to Measure Muscle Activity Combining Simultaneous Electromyography and Electrical Impedance Myography

The detection of muscle contraction and the estimation of muscle force are essential tasks in robot-assisted rehabilitation systems. The most commonly used method to investigate muscle contraction is surface electromyography (EMG), which, however, shows considerable disadvantages in predicting the muscle force, since unpredictable factors may influence the detected force but not necessarily the EMG data. Electrical impedance myography (EIM) investigates the change in electrical impedance during muscle activities and is another promising technique to investigate muscle functions. This paper introduces the design, development, and evaluation of a device that performs EMG and EIM simultaneously for more robust measurement of muscle conditions subject to artifacts. The device is light, wearable, and wireless and has a modular design, in which the EMG, EIM, micro-controller, and communication modules are stacked and interconnected through connectors. As a result, the EIM module measures the bioimpedance between 20 and 200 Ω with an error of less than 5% at 140 SPS. The settling time during the calibration phase of this module is less than 1000 ms. The EMG module captures the spectrum of the EMG signal between 20–150 Hz at 1 kSPS with an SNR of 67 dB. The micro-controller and communication module builds an ARM-Cortex M3 micro-controller which reads and transfers the captured data every 1 ms over RF (868 Mhz) with a baud rate of 500 kbps to a receptor connected to a PC. Preliminary measurements on a volunteer during leg extension, walking, and sit-to-stand showed the potential of the system to investigate muscle function by combining simultaneous EMG and EIM.

Performance of Sonomyographic and Electromyographic Sensing for Continuous Estimation of Joint Torque During Ambulation on Multiple Terrains

Advances in powered assistive device technology, including the ability to provide net mechanical power to multiple joints within a single device, have the potential to dramatically improve the mobility and restore independence to their users. However, these devices rely on the ability of their users to continuously control multiple powered lower-limb joints simultaneously. Success of such approaches rely on robust sensing of user intent and accurate mapping to device control parameters. Here, we compare two non-invasive sensing modalities: surface electromyography and sonomyography, (i.e., ultrasound imaging of skeletal muscle), as inputs to Gaussian process regression models trained to estimate hip, knee and ankle joint moments during varying forms of ambulation. Experiments were performed with ten non-disabled individuals instrumented with surface electromyography and sonomyography sensors while completing trials of level, incline (10°) and decline (10°) walking. Results suggest sonomyography of muscles on the anterior and posterior thigh can be used to estimate hip, knee and ankle joint moments more accurately than surface electromyography. Furthermore, these results can be achieved by training Gaussian process regression models in a task-independent manner; i.e., incorporating features of level and ramp walking within the same predictive framework. These findings support the integration of sonomyographic and electromyographic sensing within powered assistive devices to continuously control joint torque.

Toward Real-Time Muscle Force Inference and Device Control via Optical-Flow-Tracked Muscle Deformation

Despite the utility of musculoskeletal dynamics modeling, there exists no safe, noninvasive method of measuring in vivo muscle output force in real time - limiting both biomechanical insight into dexterous motion and intuitive control of assistive devices. In this paper, we demonstrate that muscle deformation constitutes a promising, yet unexplored signal from which to 1) infer such forces and 2) build novel device control schemes. Through a case study of the elbow joint on a preliminary cohort of 10 subjects, we show that muscle deformation (specifically, thickness change of the brachioradialis, as measured via ultrasound and tracked via optical flow) correlates well with elbow output force to an extent comparable with standard surface electromyography (sEMG) activation during varied isometric elbow contraction. We then show that, given real-time visual feedback, subjects can readily perform a trajectory tracking task using this deformation signal, and that they largely prefer this method to a comparable sEMG-based control scheme and perform the tracking task with similar accuracy. Together, these contributions illustrate muscle deformation's potential utility for both biomechanical study of individual muscle dynamics and device control, in a manner that - thanks to, unlike sEMG, the localized nature of the signal and its tight mechanistic coupling to output force - is readily extensible to multiple muscles and device degrees of freedom. To enable such future extensions, all modeling, tracking, and visualization software described in this paper, as well as all raw and processed data, have been made available on SimTK as part of the Open-Arm project (https://simtk.org/projects/openarm) for general research use.

Ultrasound-Derived Features of Muscle Architecture Provide Unique Temporal Characterization of Volitional Knee Motion

Sonomyography, or dynamic ultrasound imaging of skeletal muscle, has gained significant interest in rehabilitation medicine. Previously, correlations relating sonomyography features of muscle contraction, including muscle thickness, pennation angle, angle between aponeuroses and fascicle length, to muscle force production, strength and joint motion have been established. Additionally, relationships between grayscale image intensity, or echogenicity, with maximum voluntary isometric contraction of muscle have been noted. However, the time relationship between changes in various sonomyography features during volitional motion has yet to be explored, which would highlight if unique information pertaining to muscle contraction and motion can be obtained from this real-time imaging modality. These new insights could inform how we assess muscle function and/or how we use this modality for assistive device control. Thus, our objective was to characterize the time synchronization of changes in five features of rectus femoris contraction extracted from ultrasound images during seated knee extension and flexion. A cross-correlation analysis was performed on data recorded by a handheld ultrasound system as able-bodied subjects completed seated trials of volitional knee extension and flexion. Changes in muscle thickness, angle between aponeuroses, and mean image echogenicity, a change in brightness of the grayscale image, preceded changes in our estimates of pennation angle and fascicle length. The leading nature of these features suggest they could be objective features for early detection of impending joint motion. Finally, multiple sonomyographic features provided unique temporal information associated with this volitional task.Clinical Relevance-This work evaluates the time relationship between five commonly reported features of skeletal muscle architecture during volitional motion, which can be used for targeted clinical assessments and intent detection.

Understanding the Relationship Between Patient-Reported Function and Actual Function in the Upper Limb Prosthesis User Population: A Preliminary Study

Objective

To understand how perceived function relates to actual function at a specific stage in the rehabilitation process for the population using upper limb prostheses.

Design

Quantitative clinical descriptive study.

Setting

Clinical offices.

Participants

A sample of 61 participants (N=61; mean age, 43.0±12.8y; 51 male/10 female) with upper limb amputation who use a prosthetic device and were in the definitive stage of a prosthesis fitting process.

Interventions

Not applicable.

Main Outcome Measures

A patient-reported outcome measure, the Disabilities of the Arm, Shoulder, and Hand questionnaire (DASH), and 2 performance-based outcome measures, Box and Blocks Test (BBT) and Capacity Assessment of Prosthesis Performance for the Upper Limb (CAPPFUL), were used as variables in multiple linear regression models.

Results

The multiple linear regression models, which controlled for prosthesis type and amputation level, did not show evidence that changes in the independent variable (DASH) are significantly associated with changes in the dependent variables (log(BBT) (B=−0.007; 95% confidence interval [CI], −0.015 to 0.001; P=.0937) and CAPPFUL (B=−0.083, 95% CI, −0.374 to 0.208; P=.5623)). In both models, individuals with elbow, transhumeral (above elbow), and shoulder disarticulation showed a significant negative association with the dependent variable (CAPPFUL or logBBT). In the CAPPFUL model, there was a significant negative association with individuals using a hybrid prosthesis (B=−20.252; 95% CI, −36.562 to −3.942; P=.0170). In the logBBT model, there was a significant positive association with individuals using body-powered prostheses (B=0.430; 95% CI, 0.089-0.771; P=.0157).

Conclusions

Although additional data and analyses are needed to more completely assess the association between self-reported measures and performance-based measures of functional abilities, these preliminary results indicate that patient-reported outcomes alone may not provide a complete assessment of an upper limb prosthesis users’ functional ability and should be accompanied by population-specific performance-based measures.

A Dual-Modal Approach Using Electromyography and Sonomyography Improves Prediction of Dynamic Ankle Movement: A Case Study

For decades, surface electromyography (sEMG) has been a popular non-invasive bio-sensing technology for predicting human joint motion. However, cross-talk, interference from adjacent muscles, and its inability to measure deeply located muscles limit its performance in predicting joint motion. Recently, ultrasound (US) imaging has been proposed as an alternative non-invasive technology to predict joint movement due to its high signal-to-noise ratio, direct visualization of targeted tissue, and ability to access deep-seated muscles. This paper proposes a dual-modal approach that combines US imaging and sEMG for predicting volitional dynamic ankle dorsiflexion movement. Three feature sets: 1) a uni-modal set with four sEMG features, 2) a uni-modal set with four US imaging features, and 3) a dual-modal set with four dominant sEMG and US imaging features, together with measured ankle dorsiflexion angles, were used to train multiple machine learning regression models. The experimental results from a seated posture and five walking trials at different speeds, ranging from 0.50 m/s to 1.50 m/s, showed that the dual-modal set significantly reduced the prediction root mean square errors (RMSEs). Compared to the uni-modal sEMG feature set, the dual-modal set reduced RMSEs by up to 47.84% for the seated posture and up to 77.72% for the walking trials. Similarly, when compared to the US imaging feature set, the dual-modal set reduced RMSEs by up to 53.95% for the seated posture and up to 58.39% for the walking trials. The findings show that potentially the dual-modal sensing approach can be used as a superior sensing modality to predict human intent of a continuous motion and implemented for volitional control of clinical rehabilitative and assistive devices.

Recommendations for the Successful Implementation of Upper Limb Prosthetic Technology

Despite the numerous prosthetic hand designs that are commercially available, people with upper limb loss still frequently report dissatisfaction and abandonment. Over the past decade there have been numerous advances in prosthetic design, control, sensation, and device attachment. Each offers the potential to enhance function and satisfaction, but most come at high costs and involve surgical risks. Here, we discuss potential barriers and solutions to promote the widespread use of novel prosthetic technology. With appropriate reimbursement, multidisciplinary care teams, device-specific rehabilitation, and patient and clinician education, such technology has the potential to revolutionize the field and improve patient outcomes.

A novel framework for designing a multi-DoF prosthetic wrist control using machine learning

Prosthetic arms can significantly increase the upper limb function of individuals with upper limb loss, however despite the development of various multi-DoF prosthetic arms the rate of prosthesis abandonment is still high. One of the major challenges is to design a multi-DoF controller that has high precision, robustness, and intuitiveness for daily use. The present study demonstrates a novel framework for developing a controller leveraging machine learning algorithms and movement synergies to implement natural control of a 2-DoF prosthetic wrist for activities of daily living (ADL). The data was collected during ADL tasks of ten individuals with a wrist brace emulating the absence of wrist function. Using this data, the neural network classifies the movement and then random forest regression computes the desired velocity of the prosthetic wrist. The models were trained/tested with ADLs where their robustness was tested using cross-validation and holdout data sets. The proposed framework demonstrated high accuracy (F-1 score of 99% for the classifier and Pearson's correlation of 0.98 for the regression). Additionally, the interpretable nature of random forest regression was used to verify the targeted movement synergies. The present work provides a novel and effective framework to develop an intuitive control for multi-DoF prosthetic devices.

Hybrid Tongue - Myoelectric Control Improves Functional Use of a Robotic Hand Prosthesis

OBJECTIVE

This study aims at investigating the functional performance of a novel prosthesis control scheme integrating an inductive tongue interface and myoelectric control. The tongue interface allowed direct selection of the desired grasp while myoelectric signals were used to open and close the robotic hand.

METHODS

The novel method was compared to a conventional sequential on/off myoelectric control scheme using functional tasks defined by Assistive Hand Assessment protocol. Ten able-bodied participants were fitted with the SmartHand on their left forearm. They used both the conventional myoelectric control and the Tongue and Myoelectric Hybrid interface (TMH) to accomplish two activities of daily living (i.e., preparing a sandwich and gift wrapping). Sessions were video recorded and the outcome measure was the completion time for the subtasks as well as the full tasks.

RESULTS

The sandwich task was completed significantly faster, with 19% decrease in the completion time, using the TMH when compared to the conventional sequential on/off myoelectric control scheme (p < 0.05).

CONCLUSION

The results indicate that the TMH control scheme facilitates the active use of the prosthetic device by simplifying grasp selection, leading thereby to faster completion of challenging and relevant tasks involving bimanual activities.

Ultrasound Sensing Can Improve Continuous Classification of Discrete Ambulation Modes Compared to Surface Electromyography

Clinical translation of "intelligent" lower-limb assistive technologies relies on robust control interfaces capable of accurately detecting user intent. To date, mechanical sensors and surface electromyography (EMG) have been the primary sensing modalities used to classify ambulation. Ultrasound (US) imaging can be used to detect user-intent by characterizing structural changes of muscle. Our study evaluates wearable US imaging as a new sensing modality for continuous classification of five discrete ambulation modes: level, incline, decline, stair ascent, and stair descent ambulation, and benchmarks performance relative to EMG sensing. Ten able-bodied subjects were equipped with a wearable US scanner and eight unilateral EMG sensors. Time-intensity features were recorded from US images of three thigh muscles. Features from sliding windows of EMG signals were analyzed in two configurations: one including 5 EMG sensors on muscles around the thigh, and another with 3 additional sensors placed on the shank. Linear discriminate analysis was implemented to continuously classify these phase-dependent features of each sensing modality as one of five ambulation modes. US-based sensing statistically improved mean classification accuracy to 99.8% (99.5-100% CI) compared to 8-EMG sensors (85.8%; 84.0-87.6% CI) and 5-EMG sensors (75.3%; 74.5-76.1% CI). Further, separability analyses show the importance of superficial and deep US information for stair classification relative to other modes. These results are the first to demonstrate the ability of US-based sensing to classify discrete ambulation modes, highlighting the potential for improved assistive device control using less widespread, less superficial and higher resolution sensing of skeletal muscle.

Towards Sonomyography-Based Real-Time Control of Powered Prosthesis Grasp Synergies

Sonomyography (ultrasound imaging) offers a way of classifying complex muscle activity and configuration, with higher SNR and lower hardware requirements than sEMG, using various supervised learning algorithms. The physiological image obtained from an ultrasound probe can be used to train a classification algorithm which can run on real time ultrasound images. The predicted values can then be mapped onto assistive or teleoperated robots. This paper describes the classification of ultrasound information and its subsequent mapping onto a soft robotic gripper as a step toward direct synergy control. Support Vector Classification algorithm has been used to classify ultrasound information into a set of defined states: open, closed, pinch and hook grasps. Once the model was trained with the ultrasound image data, real time input from the forearm was used to predict these states. The final predicted state output then set joint stiffnesses in the soft actuators, changing their interactions or synergies, to obtain the corresponding soft robotic gripper states. Data collection was carried out on five different test subjects for eight trials each. An average accuracy percentage of 93% was obtained averaged over all data. This real-time ultrasound-based control of a soft robotic gripper constitutes a promising step toward intuitive and robust biosignal-based control methods for robots.

Ultrasound Features of Skeletal Muscle Can Predict Kinematics of Upcoming Lower-Limb Motion

Seamless integration of lower-limb assistive devices with the human body requires an intuitive human-machine interface, which would benefit from predicting the intent of individuals in advance of the upcoming motion. Ultrasound imaging was recently introduced as an intuitive sensing interface. The objective of the present study was to investigate the predictability of joint kinematics using ultrasound features of the rectus femoris muscle during a non-weight-bearing knee extension/flexion. Motion prediction accuracy was evaluated in 67 ms increments, up to 600 ms in time. Statistical analysis was used to evaluate the feasibility of motion prediction, and the linear mixed-effects model was used to determine a prediction time window where the joint angle prediction error is barely perceivable by the sample population, hence clinically reliable. Surprisingly, statistical tests revealed that the prediction accuracy of the joint angle was more sensitive to temporal shifts than the accuracy of the joint angular velocity prediction. Overall, predictability of the upcoming joint kinematics using ultrasound features of skeletal muscle was confirmed, and a time window for a statistically and clinically reliable prediction was found between 133 and 142 ms. A reliable prediction of user intent may provide the time needed for processing, control planning, and actuation of the assistive devices at critical points during ambulation, contributing to the intuitive behavior of lower-limb assistive devices.

Control Methods for Transradial Prostheses Based on Remnant Muscle Activity and Its Relationship with Proprioceptive Feedback

The loss of a hand can significantly affect one's work and social life. For many patients, an artificial limb can improve their mobility and ability to manage everyday activities, as well as provide the means to remain independent. This paper provides an extensive review of available biosensing methods to implement the control system for transradial prostheses based on the measured activity in remnant muscles. Covered techniques include electromyography, magnetomyography, electrical impedance tomography, capacitance sensing, near-infrared spectroscopy, sonomyography, optical myography, force myography, phonomyography, myokinetic control, and modern approaches to cineplasty. The paper also covers combinations of these approaches, which, in many cases, achieve better accuracy while mitigating the weaknesses of individual methods. The work is focused on the practical applicability of the approaches, and analyses present challenges associated with each technique along with their relationship with proprioceptive feedback, which is an important factor for intuitive control over the prosthetic device, especially for high dexterity prosthetic hands.

Evaluation of Non-Invasive Ankle Joint Effort Prediction Methods for Use in Neurorehabilitation Using Electromyography and Ultrasound Imaging

OBJECTIVE

Reliable measurement of voluntary human effort is essential for effective and safe interaction between the wearer and an assistive robot. Existing voluntary effort prediction methods that use surface electromyography (sEMG) are susceptible to prediction inaccuracies due to non-selectivity in measuring muscle responses. This technical challenge motivates an investigation into alternative non-invasive effort prediction methods that directly visualize the muscle response and improve effort prediction accuracy. The paper is a comparative study of ultrasound imaging (US)-derived neuromuscular signals and sEMG signals for their use in predicting isometric ankle dorsiflexion moment. Furthermore, the study evaluates the prediction accuracy of model-based and model-free voluntary effort prediction approaches that use these signals.

METHODS

The study evaluates sEMG signals and three US imaging-derived signals: pennation angle, muscle fascicle length, and echogenicity and three voluntary effort prediction methods: linear regression (LR), feedforward neural network (FFNN), and Hill-type neuromuscular model (HNM).

RESULTS

In all the prediction methods, pennation angle and fascicle length significantly improve the prediction accuracy of dorsiflexion moment, when compared to echogenicity. Also, compared to LR, both FFNN and HNM improve dorsiflexion moment prediction accuracy.

CONCLUSION

The findings indicate FFNN or HNM approach and using pennation angle or fascicle length predict human ankle movement intent with higher accuracy.

SIGNIFICANCE

The accurate ankle effort prediction will pave the path to safe and reliable robotic assistance in patients with drop foot.

Use of Sonomyographic Sensing to Estimate Knee Angular Velocity During Varying Modes of Ambulation

Ultrasound (US) imaging of muscle has been introduced as a promising sensing modality for assistive device control. Ten able-bodied subjects completed level, incline and decline walking on a treadmill in a motion capture laboratory while wearing reflective markers on upper- and lower-body. A wearable US transducer was affixed to subjects' anterior thigh, and time-intensity features were extracted from transverse US images of the knee extensor muscles. These features were used to train and test Gaussian process regression models for continuous estimation of knee flexion/extension angular velocity. Four regression models were evaluated: (1) subject-dependent/task-specific, (2) subject-dependent/pooled-tasks, (3) subject-independent/task-specific, and (4) subject-independent/pooled-tasks. Subject-independent models were "tuned" with up to six strides of the test subject's data to boost performance. A two-factor analysis of variance test was used to assess the effect of each approach on root mean square error (RMSE) of estimated knee angular velocity (α=0.05). Statistical parametric mapping (SPM) was completed to compare actual vs. estimated knee angular velocity as a function of the gait cycle (α=0.05). For incline and level walking, the subject-dependent/pooled-tasks model resulted in the lowest error while the subject-dependent/task-specific model resulted in the lowest error for decline walk. Impressively, the two-factor test revealed no difference between task-specific and pooled-task models. Furthermore, despite capturing many important features of knee velocity across individuals there were, as expected, significant differences between subject-dependent and subject-independent models. Collectively, these results are promising for potential assistive device control with error rates <10% for all regression models that were tested.Clinical Relevance-This work is the first study to demonstrate the feasibility of using ultrasound-based sensing for estimation of knee angular velocity during multiple modes of ambulation.

Advanced technologies for intuitive control and sensation of prosthetics

The Department of Defense, Department of Veterans Affairs and National Institutes of Health have invested significantly in advancing prosthetic technologies over the past 25 years, with the overall intent to improve the function, participation and quality of life of Service Members, Veterans, and all United States Citizens living with limb loss. These investments have contributed to substantial advancements in the control and sensory perception of prosthetic devices over the past decade. While control of motorized prosthetic devices through the use of electromyography has been widely available since the 1980s, this technology is not intuitive. Additionally, these systems do not provide stimulation for sensory perception. Recent research has made significant advancement not only in the intuitive use of electromyography for control but also in the ability to provide relevant meaningful perceptions through various stimulation approaches. While much of this previous work has traditionally focused on those with upper extremity amputation, new developments include advanced bidirectional neuroprostheses that are applicable to both the upper and lower limb amputation. The goal of this review is to examine the state-of-the-science in the areas of intuitive control and sensation of prosthetic devices and to discuss areas of exploration for the future. Current research and development efforts in external systems, implanted systems, surgical approaches, and regenerative approaches will be explored.

Selection of Features and Classifiers for EMG-EEG-Based Upper Limb Assistive Devices-A Review

Bio-signals are distinctive factors in the design of human-machine interface, essentially useful for prosthesis, orthosis, and exoskeletons. Despite the progress in the analysis of pattern recognition based devices; the acceptance of these devices is still questionable. One reason is the lack of information to identify the possible combinations of features and classifiers. Besides; there is also a need for optimal selection of various sensors for sensations such as touch, force, texture, along with EMGs/EEGs. This article reviews the two bio-signal techniques, named as electromyography and electroencephalography. The details of the features and the classifiers used in the data processing for upper limb assist devices are summarised here. Various features and their sets are surveyed and different classifiers for feature sets are discussed on the basis of the classification rate. The review was carried out on the basis of the last 10-12 years of published research in this area. This article also outlines the influence of modality of EMGs and EEGs with other sensors on classifications. Also, other bio-signals used in upper limb devices and future aspects are considered.