Sparsity Analysis of a Sonomyographic Muscle-Computer Interface

Sonomyography for Control of Upper-Limb Prostheses: Current State and Future Directions

Problem Statement

Despite the recent advancements in technology, many individuals with upper-limb loss struggle to achieve stable control over multiple degrees of freedom in a prosthesis. There is an ongoing need to develop noninvasive prosthesis control modalities that could improve functional patient outcomes.

Proposed Solution

Ultrasound-based sensing of muscle deformation, known as sonomyography, is an emerging sensing modality for upper-limb prosthesis control with the potential to significantly improve functionality. Sonomyography enables spatiotemporal characterization of both superficial and deep muscle activity, making it possible to distinguish the contributions of individual muscles during functional movements and derive a large set of independent prosthesis control signals. Using sonomyography to control a prosthesis has shown great promise in the research literature but has not yet been fully adapted for clinical use. This article describes the implementation of sonomyography for upper-limb prosthesis control, ongoing technological development, considerations for deploying this technology in clinical settings, and recommendations for future study.

Clinical Relevance

Sonomyography may soon become a clinically viable modality for upper-limb prosthesis control that could offer prosthetists an additional solution when selecting optimal treatment plans for their patients.

Applications for oral research in microgravity - lessons learned from burning mouth syndrome and ageing studies

The negative consequences of microgravity for the human body are central aspects of space travel that raise health problems. Altered functions of the same systems and treatment options are common points of spaceflight physiology, age-related diseases, and oral medicine. This work emphasizes the convergence of knowledge between pathophysiological changes brought on by aging, physiological reactions to microgravity exposure, and non-pharmacological and non-invasive treatment methods that can be used in spaceflight. Sarcopenia, peripheral nerves alterations, neuromotor plaque in the masticatory muscles, lingual, labial, and buccal weakness, nociplastic pain in oral mucosal diseases, and microgravity, as well as soft tissue changes and pathologies related to chewing and swallowing, corticomotor neuroplasticity of tongue, and swallowing biomechanics, are of particular interest to us. Neurologic disease and other pathologies such as recovery from post-stroke dysphagia, nociplastic pain in glossodynia, sleep bruxism, and obstructive sleep apnea have been studied and, in some cases, successfully treated with non-invasive direct and transcranial magnetic stimulation (TMS) methods in recent decades. An interdisciplinary team from medical specialties, engineering, and biophysics propose an exploratory study based on the parallelism of ageing and space physiology, along with experiment scenarios considering TMS and non-invasive direct methods.

Controlling Upper Limb Prostheses Using Sonomyography (SMG): A Review

This paper presents a critical review and comparison of the results of recently published studies in the fields of human-machine interface and the use of sonomyography (SMG) for the control of upper limb prothesis. For this review paper, a combination of the keywords "Human Machine Interface", "Sonomyography", "Ultrasound", "Upper Limb Prosthesis", "Artificial Intelligence", and "Non-Invasive Sensors" was used to search for articles on Google Scholar and PubMed. Sixty-one articles were found, of which fifty-nine were used in this review. For a comparison of the different ultrasound modes, feature extraction methods, and machine learning algorithms, 16 articles were used. Various modes of ultrasound devices for prosthetic control, various machine learning algorithms for classifying different hand gestures, and various feature extraction methods for increasing the accuracy of artificial intelligence used in their controlling systems are reviewed in this article. The results of the review article show that ultrasound sensing has the potential to be used as a viable human-machine interface in order to control bionic hands with multiple degrees of freedom. Moreover, different hand gestures can be classified by different machine learning algorithms trained with extracted features from collected data with an accuracy of around 95%.

Wearable super-resolution muscle-machine interfacing

Muscles are the actuators of all human actions, from daily work and life to communication and expression of emotions. Myography records the signals from muscle activities as an interface between machine hardware and human wetware, granting direct and natural control of our electronic peripherals. Regardless of the significant progression as of late, the conventional myographic sensors are still incapable of achieving the desired high-resolution and non-invasive recording. This paper presents a critical review of state-of-the-art wearable sensing technologies that measure deeper muscle activity with high spatial resolution, so-called super-resolution. This paper classifies these myographic sensors according to the different signal types (i.e., biomechanical, biochemical, and bioelectrical) they record during measuring muscle activity. By describing the characteristics and current developments with advantages and limitations of each myographic sensor, their capabilities are investigated as a super-resolution myography technique, including: (i) non-invasive and high-density designs of the sensing units and their vulnerability to interferences, (ii) limit-of-detection to register the activity of deep muscles. Finally, this paper concludes with new opportunities in this fast-growing super-resolution myography field and proposes promising future research directions. These advances will enable next-generation muscle-machine interfaces to meet the practical design needs in real-life for healthcare technologies, assistive/rehabilitation robotics, and human augmentation with extended reality.

Classification Performance and Feature Space Characteristics in Individuals With Upper Limb Loss Using Sonomyography

Objective: Sonomyography, or ultrasound-based sensing of muscle deformation, is an emerging modality for upper limb prosthesis control. Although prior studies have shown that individuals with upper limb loss can achieve successful motion classification with sonomyography, it is important to better understand the time-course over which proficiency develops. In this study, we characterized user performance during their initial and subsequent exposures to sonomyography. Method: Ultrasound images corresponding to a series of hand gestures were collected from individuals with transradial limb loss under three scenarios: during their initial exposure to sonomyography (Experiment 1), during a subsequent exposure to sonomyography where they were provided biofeedback as part of a training protocol (Experiment 2), and during testing sessions held on different days (Experiment 3). User performance was characterized by offline classification accuracy, as well as metrics describing the consistency and separability of the sonomyography signal patterns in feature space. Results: Classification accuracy was high during initial exposure to sonomyography (96.2 ± 5.9%) and did not systematically change with the provision of biofeedback or on different days. Despite this stable classification performance, some of the feature space metrics changed. Conclusions: User performance was strong upon their initial exposure to sonomyography and did not improve with subsequent exposure. Clinical Impact: Prosthetists may be able to quickly assess if a patient will be successful with sonomyography without submitting them to an extensive training protocol, leading to earlier socket fabrication and delivery.

First Demonstration of Functional Task Performance Using a Sonomyographic Prosthesis: A Case Study

Ultrasound-based sensing of muscle deformation, known as sonomyography, has shown promise for accurately classifying the intended hand grasps of individuals with upper limb loss in offline settings. Building upon this previous work, we present the first demonstration of real-time prosthetic hand control using sonomyography to perform functional tasks. An individual with congenital bilateral limb absence was fitted with sockets containing a low-profile ultrasound transducer placed over forearm muscle tissue in the residual limbs. A classifier was trained using linear discriminant analysis to recognize ultrasound images of muscle contractions for three discrete hand configurations (rest, tripod grasp, index finger point) under a variety of arm positions designed to cover the reachable workspace. A prosthetic hand mounted to the socket was then controlled using this classifier. Using this real-time sonomyographic control, the participant was able to complete three functional tasks that required selecting different hand grasps in order to grasp and move one-inch wooden blocks over a broad range of arm positions. Additionally, these tests were successfully repeated without retraining the classifier across 3 hours of prosthesis use and following simulated donning and doffing of the socket. This study supports the feasibility of using sonomyography to control upper limb prostheses in real-world applications.

Evaluating Electromyography and Sonomyography Sensor Fusion to Estimate Lower-Limb Kinematics Using Gaussian Process Regression

Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

Performance of Sonomyographic and Electromyographic Sensing for Continuous Estimation of Joint Torque During Ambulation on Multiple Terrains

Advances in powered assistive device technology, including the ability to provide net mechanical power to multiple joints within a single device, have the potential to dramatically improve the mobility and restore independence to their users. However, these devices rely on the ability of their users to continuously control multiple powered lower-limb joints simultaneously. Success of such approaches rely on robust sensing of user intent and accurate mapping to device control parameters. Here, we compare two non-invasive sensing modalities: surface electromyography and sonomyography, (i.e., ultrasound imaging of skeletal muscle), as inputs to Gaussian process regression models trained to estimate hip, knee and ankle joint moments during varying forms of ambulation. Experiments were performed with ten non-disabled individuals instrumented with surface electromyography and sonomyography sensors while completing trials of level, incline (10°) and decline (10°) walking. Results suggest sonomyography of muscles on the anterior and posterior thigh can be used to estimate hip, knee and ankle joint moments more accurately than surface electromyography. Furthermore, these results can be achieved by training Gaussian process regression models in a task-independent manner; i.e., incorporating features of level and ramp walking within the same predictive framework. These findings support the integration of sonomyographic and electromyographic sensing within powered assistive devices to continuously control joint torque.

Flexible Piezoelectric Sensors for Miniaturized Sonomyography

Sonomyography refers to the measurement of muscle activity with an ultrasonic transducer. It is a candidate modality for applications in diagnosis of muscle conditions, rehabilitation engineering and prosthesis control as an alternative to electromyography. We propose a mechanically-flexible piezoelectric sonomyography transducer. Simulating different components of the transducer, using COMSOL Multiphysics^® software, we analyze various electromechanical parameters, such as von Mises stress and charge accumulation. Our findings on modelling of a single-element device, comprised of a PZT-5H layer of thickness 66µm, with a polymer substrate (E = 2.5 GPa), demonstrate optimal flexibility and charge accumulation for sonomyography. The addition of Polyimide and PMMA (Polymethyl methacrylate) as an acoustic matching layer and an acoustic lens, respectively, allowed for adequate energy transfer to the medium, whilst still maintaining good mechanical properties. In addition, preliminary ultrasound transmission simulations (200 kHz to 30 MHz) showed the importance of the aspect ratio of the device and how there is a need for further studies on it. The development of such a technology could be of great use within the healthcare sector, not only due to its ability to provide highly accurate and varied real-time muscle data, but also because of the range of applications that could benefit from its use.

Ultrasound Sensing Can Improve Continuous Classification of Discrete Ambulation Modes Compared to Surface Electromyography

Clinical translation of "intelligent" lower-limb assistive technologies relies on robust control interfaces capable of accurately detecting user intent. To date, mechanical sensors and surface electromyography (EMG) have been the primary sensing modalities used to classify ambulation. Ultrasound (US) imaging can be used to detect user-intent by characterizing structural changes of muscle. Our study evaluates wearable US imaging as a new sensing modality for continuous classification of five discrete ambulation modes: level, incline, decline, stair ascent, and stair descent ambulation, and benchmarks performance relative to EMG sensing. Ten able-bodied subjects were equipped with a wearable US scanner and eight unilateral EMG sensors. Time-intensity features were recorded from US images of three thigh muscles. Features from sliding windows of EMG signals were analyzed in two configurations: one including 5 EMG sensors on muscles around the thigh, and another with 3 additional sensors placed on the shank. Linear discriminate analysis was implemented to continuously classify these phase-dependent features of each sensing modality as one of five ambulation modes. US-based sensing statistically improved mean classification accuracy to 99.8% (99.5-100% CI) compared to 8-EMG sensors (85.8%; 84.0-87.6% CI) and 5-EMG sensors (75.3%; 74.5-76.1% CI). Further, separability analyses show the importance of superficial and deep US information for stair classification relative to other modes. These results are the first to demonstrate the ability of US-based sensing to classify discrete ambulation modes, highlighting the potential for improved assistive device control using less widespread, less superficial and higher resolution sensing of skeletal muscle.