Evaluation of Non-Invasive Ankle Joint Effort Prediction Methods for Use in Neurorehabilitation Using Electromyography and Ultrasound Imaging

A scoping review of applications of artificial intelligence in kinematics and kinetics of ankle sprains - current state-of-the-art and future prospects

BACKGROUND

Despite the existence of evidence-based rehabilitation strategies that address biomechanical deficits, the persistence of recurrent ankle problems in 70% of patients with acute ankle sprains highlights the unresolved nature of this issue. Artificial intelligence (AI) emerges as a promising tool to identify definitive predictors for ankle sprains. This paper aims to summarize the use of AI in investigating the ankle biomechanics of healthy and subjects with ankle sprains.

METHODS

Articles published between 2010 and 2023 were searched from five electronic databases. 59 papers were included for analysis with regards to: i). types of motion tested (functional vs. purposeful ankle movement); ii) types of biomechanical parameters measured (kinetic vs kinematic); iii) types of sensor systems used (lab-based vs field-based); and, iv) AI techniques used.

FINDINGS

Most studies (83.1%) examined biomechanics during functional motion. Single kinematic parameter, specifically ankle range of motion, could obtain accuracy up to 100% in identifying injury status. Wearable sensor exhibited high reliability for use in both laboratory and on-field/clinical settings. AI algorithms primarily utilized electromyography and joint angle information as input data. Support vector machine was the most used supervised learning algorithm (18.64%), while artificial neural network demonstrated the highest accuracy in eight studies.

INTERPRETATIONS

The potential for remote patient monitoring is evident with the adoption of field-based devices. Nevertheless, AI-based sensors are underutilized in detecting ankle motions at risk of sprain. We identify three key challenges: sensor designs, the controllability of AI models, and the integration of AI-sensor models, providing valuable insights for future research.

A clustering-based method for estimating pennation angle from B-mode ultrasound images

B-mode ultrasound (US) is often used to noninvasively measure skeletal muscle architecture, which contains human intent information. Extracted features from B-mode images can help improve closed-loop human-robotic interaction control when using rehabilitation/assistive devices. The traditional manual approach to inferring the muscle structural features from US images is laborious, time-consuming, and subjective among different investigators. This paper proposes a clustering-based detection method that can mimic a well-trained human expert in identifying fascicle and aponeurosis and, therefore, compute the pennation angle. The clustering-based architecture assumes that muscle fibers have tubular characteristics. It is robust for low-frequency image streams. We compared the proposed algorithm to two mature benchmark techniques: UltraTrack and ImageJ. The performance of the proposed approach showed higher accuracy in our dataset (frame frequency is 20 Hz), that is, similar to the human expert. The proposed method shows promising potential in automatic muscle fascicle orientation detection to facilitate implementations in biomechanics modeling, rehabilitation robot control design, and neuromuscular disease diagnosis with low-frequency data stream.

A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Walking Kinematics Via an Artificial Neural Network

Lower-limb powered prostheses can provide users with volitional control of ambulation. To accomplish this goal, they require a sensing modality that reliably interprets user intention to move. Surface electromyography (EMG) has been previously proposed to measure muscle excitation and provide volitional control to upper- and lower-limb powered prosthesis users. Unfortunately, EMG suffers from a low signal to noise ratio and crosstalk between neighboring muscles, often limiting the performance of EMG-based controllers. Ultrasound has been shown to have better resolution and specificity than surface EMG. However, this technology has yet to be integrated into lower-limb prostheses. Here we show that A-mode ultrasound sensing can reliably predict the prosthesis walking kinematics of individuals with a transfemoral amputation. Ultrasound features from the residual limb of 9 transfemoral amputee subjects were recorded with A-mode ultrasound during walking with their passive prosthesis. The ultrasound features were mapped to joint kinematics through a regression neural network. Testing of the trained model against untrained kinematics from an altered walking speed show accurate predictions of knee position, knee velocity, ankle position, and ankle velocity, with a normalized RMSE of 9.0 ± 3.1%, 7.3 ± 1.6%, 8.3 ± 2.3%, and 10.0 ± 2.5% respectively. This ultrasound-based prediction suggests that A-mode ultrasound is a viable sensing technology for recognizing user intent. This study is the first necessary step towards implementation of volitional prosthesis controller based on A-mode ultrasound for individuals with transfemoral amputation.

Development of a Wearable Ultrasound Transducer for Sensing Muscle Activities in Assistive Robotics Applications

Robotic prostheses and powered exoskeletons are novel assistive robotic devices for modern medicine. Muscle activity sensing plays an important role in controlling assistive robotics devices. Most devices measure the surface electromyography (sEMG) signal for myoelectric control. However, sEMG is an integrated signal from muscle activities. It is difficult to sense muscle movements in specific small regions, particularly at different depths. Alternatively, traditional ultrasound imaging has recently been proposed to monitor muscle activity due to its ability to directly visualize superficial and at-depth muscles. Despite their advantages, traditional ultrasound probes lack wearability. In this paper, a wearable ultrasound (US) transducer, based on lead zirconate titanate (PZT) and a polyimide substrate, was developed for a muscle activity sensing demonstration. The fabricated PZT-5A elements were arranged into a 4 × 4 array and then packaged in polydimethylsiloxane (PDMS). In vitro porcine tissue experiments were carried out by generating the muscle activities artificially, and the muscle movements were detected by the proposed wearable US transducer via muscle movement imaging. Experimental results showed that all 16 elements had very similar acoustic behaviors: the averaged central frequency, -6 dB bandwidth, and electrical impedance in water were 10.59 MHz, 37.69%, and 78.41 Ω, respectively. The in vitro study successfully demonstrated the capability of monitoring local muscle activity using the prototyped wearable transducer. The findings indicate that ultrasonic sensing may be an alternative to standardize myoelectric control for assistive robotics applications.

Multi-Sensing Techniques with Ultrasound for Musculoskeletal Assessment: A Review

The study of muscle contractions generated by the muscle-tendon unit (MTU) plays a critical role in medical diagnoses, monitoring, rehabilitation, and functional assessments, including the potential for movement prediction modeling used for prosthetic control. Over the last decade, the use of combined traditional techniques to quantify information about the muscle condition that is correlated to neuromuscular electrical activation and the generation of muscle force and vibration has grown. The purpose of this review is to guide the reader to relevant works in different applications of ultrasound imaging in combination with other techniques for the characterization of biological signals. Several research groups have been using multi-sensing systems to carry out specific studies in the health area. We can divide these studies into two categories: human-machine interface (HMI), in which sensors are used to capture critical information to control computerized prostheses and/or robotic actuators, and physiological study, where sensors are used to investigate a hypothesis and/or a clinical diagnosis. In addition, the relevance, challenges, and expectations for future work are discussed.

A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control

Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging's region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ( < .001) and increased the prediction coefficient of determination by 20.13% ( < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.

Fused ultrasound and electromyography-driven neuromuscular model to improve plantarflexion moment prediction across walking speeds

Background

Improving the prediction ability of a human-machine interface (HMI) is critical to accomplish a bio-inspired or model-based control strategy for rehabilitation interventions, which are of increased interest to assist limb function post neurological injuries. A fundamental role of the HMI is to accurately predict human intent by mapping signals from a mechanical sensor or surface electromyography (sEMG) sensor. These sensors are limited to measuring the resulting limb force or movement or the neural signal evoking the force. As the intermediate mapping in the HMI also depends on muscle contractility, a motivation exists to include architectural features of the muscle as surrogates of dynamic muscle movement, thus further improving the HMI’s prediction accuracy.

Objective

The purpose of this study is to investigate a non-invasive sEMG and ultrasound (US) imaging-driven Hill-type neuromuscular model (HNM) for net ankle joint plantarflexion moment prediction. We hypothesize that the fusion of signals from sEMG and US imaging results in a more accurate net plantarflexion moment prediction than sole sEMG or US imaging.

Methods

Ten young non-disabled participants walked on a treadmill at speeds of 0.50, 0.75, 1.00, 1.25, and 1.50 m/s. The proposed HNM consists of two muscle-tendon units. The muscle activation for each unit was calculated as a weighted summation of the normalized sEMG signal and normalized muscle thickness signal from US imaging. The HNM calibration was performed under both single-speed mode and inter-speed mode, and then the calibrated HNM was validated across all walking speeds.

Results

On average, the normalized moment prediction root mean square error was reduced by 14.58 % (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.012$$\end{document}p=0.012) and 36.79 % (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.001$$\end{document}p<0.001) with the proposed HNM when compared to sEMG-driven and US imaging-driven HNMs, respectively. Also, the calibrated models with data from the inter-speed mode were more robust than those from single-speed modes for the moment prediction.

Conclusions

The proposed sEMG-US imaging-driven HNM can significantly improve the net plantarflexion moment prediction accuracy across multiple walking speeds. The findings imply that the proposed HNM can be potentially used in bio-inspired control strategies for rehabilitative devices due to its superior prediction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12984-022-01061-z.

Evaluating Electromyography and Sonomyography Sensor Fusion to Estimate Lower-Limb Kinematics Using Gaussian Process Regression

Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

Ultrasound Echogenicity as an Indicator of Muscle Fatigue during Functional Electrical Stimulation

Functional electrical stimulation (FES) is a potential neurorehabilitative intervention to enable functional movements in persons with neurological conditions that cause mobility impairments. However, the quick onset of muscle fatigue during FES is a significant challenge for sustaining the desired functional movements for more extended periods. Therefore, a considerable interest still exists in the development of sensing techniques that reliably measure FES-induced muscle fatigue. This study proposes to use ultrasound (US) imaging-derived echogenicity signal as an indicator of FES-induced muscle fatigue. We hypothesized that the US-derived echogenicity signal is sensitive to FES-induced muscle fatigue under isometric and dynamic muscle contraction conditions. Eight non-disabled participants participated in the experiments, where FES electrodes were applied on their tibialis anterior (TA) muscles. During a fatigue protocol under either isometric and dynamic ankle dorsiflexion conditions, we synchronously collected the isometric dorsiflexion torque or dynamic dorsiflexion angle on the ankle joint, US echogenicity signals from TA muscle, and the applied stimulation intensity. The experimental results showed an exponential reduction in the US echogenicity relative change (ERC) as the fatigue progressed under the isometric (R2=0.891±0.081) and dynamic (R2=0.858±0.065) conditions. The experimental results also implied a strong linear relationship between US ERC and TA muscle fatigue benchmark (dorsiflexion torque or angle amplitude), with R2 values of 0.840±0.054 and 0.794±0.065 under isometric and dynamic conditions, respectively. The findings in this study indicate that the US echogenicity signal is a computationally efficient signal that strongly represents FES-induced muscle fatigue. Its potential real-time implementation to detect fatigue can facilitate an FES closed-loop controller design that considers the FES-induced muscle fatigue.

Performance of Sonomyographic and Electromyographic Sensing for Continuous Estimation of Joint Torque During Ambulation on Multiple Terrains

Advances in powered assistive device technology, including the ability to provide net mechanical power to multiple joints within a single device, have the potential to dramatically improve the mobility and restore independence to their users. However, these devices rely on the ability of their users to continuously control multiple powered lower-limb joints simultaneously. Success of such approaches rely on robust sensing of user intent and accurate mapping to device control parameters. Here, we compare two non-invasive sensing modalities: surface electromyography and sonomyography, (i.e., ultrasound imaging of skeletal muscle), as inputs to Gaussian process regression models trained to estimate hip, knee and ankle joint moments during varying forms of ambulation. Experiments were performed with ten non-disabled individuals instrumented with surface electromyography and sonomyography sensors while completing trials of level, incline (10°) and decline (10°) walking. Results suggest sonomyography of muscles on the anterior and posterior thigh can be used to estimate hip, knee and ankle joint moments more accurately than surface electromyography. Furthermore, these results can be achieved by training Gaussian process regression models in a task-independent manner; i.e., incorporating features of level and ramp walking within the same predictive framework. These findings support the integration of sonomyographic and electromyographic sensing within powered assistive devices to continuously control joint torque.

Ultrasound Echogenicity-based Assessment of Muscle Fatigue During Functional Electrical Stimulation

The rapid onset of muscle fatigue during functional electrical stimulation (FES) is a major challenge when attempting to perform long-term periodic tasks such as walking. Surface electromyography (sEMG) is frequently used to detect muscle fatigue for both volitional and FES-evoked muscle contraction. However, sEMG contamination from both FES stimulation artifacts and residual M-wave signals requires sophisticated processing to get clean signals and evaluate the muscle fatigue level. The objective of this paper is to investigate the feasibility of computationally efficient ultrasound (US) echogenicity as a candidate indicator of FES-induced muscle fatigue. We conducted isometric and dynamic ankle dorsiflexion experiments with electrically stimulated tibialis anterior (TA) muscle on three human participants. During a fatigue protocol, we synchronously recorded isometric dorsiflexion force, dynamic dorsiflexion angle, US images, and stimulation intensity. The temporal US echogenicity from US images was calculated based on a gray-scaled analysis to assess the decrease in dorsiflexion force or motion range due to FES-induced TA muscle fatigue. The results showed a monotonic reduction in US echogenicity change along with the fatigue progression for both isometric (R² =0.870±0.026) and dynamic (R² =0.803±0.048) ankle dorsiflexion. These results implied a strong linear relationship between US echogenicity and TA muscle fatigue level. The findings indicate that US echogenicity may be a promising computationally efficient indicator for assessing FES-induced muscle fatigue and may aid in the design of muscle-in-the-loop FES controllers that consider the onset of muscle fatigue.

Plantarflexion Moment Prediction during the Walking Stance Phase with an sEMG-Ultrasound Imaging-Driven Model

Many rehabilitative exoskeletons use non-invasive surface electromyography (sEMG) to measure human volitional intent. However, signals from adjacent muscle groups interfere with sEMG measurements. Further, the inability to measure sEMG signals from deeply located muscles may not accurately measure the volitional intent. In this work, we combined sEMG and ultrasound (US) imaging-derived signals to improve the prediction accuracy of voluntary ankle effort. We used a multivariate linear model (MLM) that combines sEMG and US signals for ankle joint net plantarflexion (PF) moment prediction during the walking stance phase. We hypothesized that the proposed sEMG-US imaging-driven MLM would result in more accurate net PF moment prediction than sEMG-driven and US imaging-driven MLMs. Synchronous measurements including reflective makers coordinates, ground reaction forces, sEMG signals of lateral/medial gastrocnemius (LGS/MGS), and soleus (SOL) muscles, and US imaging of LGS and SOL muscles were collected from five able-bodied participants walking on a treadmill at multiple speeds. The ankle joint net PF moment benchmark was calculated based on inverse dynamics, while the net PF moment prediction was determined by the sEMG-US imaging-driven, sEMG-driven, and US imaging-driven MLMs. The findings show that the sEMG-US imaging-driven MLM can significantly improve the prediction of net PF moment during the walking stance phase at multiple speeds. Potentially, the proposed sEMG-US imaging-driven MLM can be used as a superior joint motion intent model in advanced and intelligent control strategies for rehabilitative exoskeletons.

A Dual-Modal Approach Using Electromyography and Sonomyography Improves Prediction of Dynamic Ankle Movement: A Case Study

For decades, surface electromyography (sEMG) has been a popular non-invasive bio-sensing technology for predicting human joint motion. However, cross-talk, interference from adjacent muscles, and its inability to measure deeply located muscles limit its performance in predicting joint motion. Recently, ultrasound (US) imaging has been proposed as an alternative non-invasive technology to predict joint movement due to its high signal-to-noise ratio, direct visualization of targeted tissue, and ability to access deep-seated muscles. This paper proposes a dual-modal approach that combines US imaging and sEMG for predicting volitional dynamic ankle dorsiflexion movement. Three feature sets: 1) a uni-modal set with four sEMG features, 2) a uni-modal set with four US imaging features, and 3) a dual-modal set with four dominant sEMG and US imaging features, together with measured ankle dorsiflexion angles, were used to train multiple machine learning regression models. The experimental results from a seated posture and five walking trials at different speeds, ranging from 0.50 m/s to 1.50 m/s, showed that the dual-modal set significantly reduced the prediction root mean square errors (RMSEs). Compared to the uni-modal sEMG feature set, the dual-modal set reduced RMSEs by up to 47.84% for the seated posture and up to 77.72% for the walking trials. Similarly, when compared to the US imaging feature set, the dual-modal set reduced RMSEs by up to 53.95% for the seated posture and up to 58.39% for the walking trials. The findings show that potentially the dual-modal sensing approach can be used as a superior sensing modality to predict human intent of a continuous motion and implemented for volitional control of clinical rehabilitative and assistive devices.