Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Unsupervised Domain Adaptation for Inter-Session Re-Calibration of Ultrasound-Based HMIs

Human-Machine Interfaces (HMIs) have gained popularity as they allow for an effortless and natural interaction between the user and the machine by processing information gathered from a single or multiple sensing modalities and transcribing user intentions to the desired actions. Their operability depends on frequent periodic re-calibration using newly acquired data due to their adaptation needs in dynamic environments, where test-time data continuously change in unforeseen ways, a cause that significantly contributes to their abandonment and remains unexplored by the Ultrasound-based (US-based) HMI community. In this work, we conduct a thorough investigation of Unsupervised Domain Adaptation (UDA) algorithms for the re-calibration of US-based HMIs during within-day sessions, which utilize unlabeled data for re-calibration. Our experimentation led us to the proposal of a CNN-based architecture for simultaneous wrist rotation angle and finger gesture prediction that achieves comparable performance with the state-of-the-art while featuring 87.92% less trainable parameters. According to our findings, DANN (a Domain-Adversarial training algorithm), with proper initialization, offers an average 24.99% classification accuracy performance enhancement when compared to no re-calibration setting. However, our results suggest that in cases where the experimental setup and the UDA configuration may differ, observed enhancements would be rather small or even unnoticeable.

Distributed Wearable Ultrasound Sensors Predict Isometric Ground Reaction Force

Rehabilitation from musculoskeletal injuries focuses on reestablishing and monitoring muscle activation patterns to accurately produce force. The aim of this study is to explore the use of a novel low-powered wearable distributed Simultaneous Musculoskeletal Assessment with Real-Time Ultrasound (SMART-US) device to predict force during an isometric squat task. Participants (N = 5) performed maximum isometric squats under two medical imaging techniques; clinical musculoskeletal motion mode (m-mode) ultrasound on the dominant vastus lateralis and SMART-US sensors placed on the rectus femoris, vastus lateralis, medial hamstring, and vastus medialis. Ultrasound features were extracted, and a linear ridge regression model was used to predict ground reaction force. The performance of ultrasound features to predict measured force was tested using either the Clinical M-mode, SMART-US sensors on the vastus lateralis (SMART-US: VL), rectus femoris (SMART-US: RF), medial hamstring (SMART-US: MH), and vastus medialis (SMART-US: VMO) or utilized all four SMART-US sensors (Distributed SMART-US). Model training showed that the Clinical M-mode and the Distributed SMART-US model were both significantly different from the SMART-US: VL, SMART-US: MH, SMART-US: RF, and SMART-US: VMO models (p < 0.05). Model validation showed that the Distributed SMART-US model had an R2 of 0.80 ± 0.04 and was significantly different from SMART-US: VL but not from the Clinical M-mode model. In conclusion, a novel wearable distributed SMART-US system can predict ground reaction force using machine learning, demonstrating the feasibility of wearable ultrasound imaging for ground reaction force estimation.

Wearable ultrasound devices: An emerging era for biomedicine and clinical translation

In recent years, personalized diagnosis and treatment have gained significant recognition and rapid development in the biomedicine and healthcare. Due to the flexibility, portability and excellent compatibility, wearable ultrasound (WUS) devices have become emerging personalized medical devices with great potential for development. Currently, with the development of the ongoing advancements in materials and structural design of the ultrasound transducers, WUS devices have improved performance and are increasingly applied in the medical field. In this review, we provide an overview of the design and structure of WUS devices, focusing on their application for diagnosis and treatment of various diseases from a clinical application perspective, and then explore the issues that need to be addressed before clinical translation. Finally, we summarize the progress made in the development of WUS devices, and discuss the current challenges and the future direction of their development. In conclusion, WUS devices usher an emerging era for biomedicine with great clinical promise.

Sonomyography for Control of Upper-Limb Prostheses: Current State and Future Directions

Problem Statement

Despite the recent advancements in technology, many individuals with upper-limb loss struggle to achieve stable control over multiple degrees of freedom in a prosthesis. There is an ongoing need to develop noninvasive prosthesis control modalities that could improve functional patient outcomes.

Proposed Solution

Ultrasound-based sensing of muscle deformation, known as sonomyography, is an emerging sensing modality for upper-limb prosthesis control with the potential to significantly improve functionality. Sonomyography enables spatiotemporal characterization of both superficial and deep muscle activity, making it possible to distinguish the contributions of individual muscles during functional movements and derive a large set of independent prosthesis control signals. Using sonomyography to control a prosthesis has shown great promise in the research literature but has not yet been fully adapted for clinical use. This article describes the implementation of sonomyography for upper-limb prosthesis control, ongoing technological development, considerations for deploying this technology in clinical settings, and recommendations for future study.

Clinical Relevance

Sonomyography may soon become a clinically viable modality for upper-limb prosthesis control that could offer prosthetists an additional solution when selecting optimal treatment plans for their patients.

Novel near E-Field Topography Sensor for Human-Machine Interfacing in Robotic Applications

This work investigates a new sensing technology for use in robotic human-machine interface (HMI) applications. The proposed method uses near E-field sensing to measure small changes in the limb surface topography due to muscle actuation over time. The sensors introduced in this work provide a non-contact, low-computational-cost, and low-noise method for sensing muscle activity. By evaluating the key sensor characteristics, such as accuracy, hysteresis, and resolution, the performance of this sensor is validated. Then, to understand the potential performance in intention detection, the unmodified digital output of the sensor is analysed against movements of the hand and fingers. This is done to demonstrate the worst-case scenario and to show that the sensor provides highly targeted and relevant data on muscle activation before any further processing. Finally, a convolutional neural network is used to perform joint angle prediction over nine degrees of freedom, achieving high-level regression performance with an RMSE value of less than six degrees for thumb and wrist movements and 11 degrees for finger movements. This work demonstrates the promising performance of this novel approach to sensing for use in human-machine interfaces.

Continuous A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Kinematics Across Different Ambulation Tasks

OBJECTIVE

Volitional control systems for powered prostheses require the detection of user intent to operate in real life scenarios. Ambulation mode classification has been proposed to address this issue. However, these approaches introduce discrete labels to the otherwise continuous task that is ambulation. An alternative approach is to provide users with direct, voluntary control of the powered prosthesis motion. Surface electromyography (EMG) sensors have been proposed for this task, but poor signal-to-noise ratios and crosstalk from neighboring muscles limit performance. B-mode ultrasound can address some of these issues at the cost of reduced clinical viability due to the substantial increase in size, weight, and cost. Thus, there is an unmet need for a lightweight, portable neural system that can effectively detect the movement intention of individuals with lower-limb amputation.

METHODS

In this study, we show that a small and lightweight A-mode ultrasound system can continuously predict prosthesis joint kinematics in seven individuals with transfemoral amputation across different ambulation tasks. Features from the A-mode ultrasound signals were mapped to the user's prosthesis kinematics via an artificial neural network.

RESULTS

Predictions on testing ambulation circuit trials resulted in a mean normalized RMSE across different ambulation modes of 8.7 ± 3.1%, 4.6 ± 2.5%, 7.2 ± 1.8%, and 4.6 ± 2.4% for knee position, knee velocity, ankle position, and ankle velocity, respectively.

CONCLUSION AND SIGNIFICANCE

This study lays the foundation for future applications of A-mode ultrasound for volitional control of powered prostheses during a variety of daily ambulation tasks.

A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Walking Kinematics Via an Artificial Neural Network

Lower-limb powered prostheses can provide users with volitional control of ambulation. To accomplish this goal, they require a sensing modality that reliably interprets user intention to move. Surface electromyography (EMG) has been previously proposed to measure muscle excitation and provide volitional control to upper- and lower-limb powered prosthesis users. Unfortunately, EMG suffers from a low signal to noise ratio and crosstalk between neighboring muscles, often limiting the performance of EMG-based controllers. Ultrasound has been shown to have better resolution and specificity than surface EMG. However, this technology has yet to be integrated into lower-limb prostheses. Here we show that A-mode ultrasound sensing can reliably predict the prosthesis walking kinematics of individuals with a transfemoral amputation. Ultrasound features from the residual limb of 9 transfemoral amputee subjects were recorded with A-mode ultrasound during walking with their passive prosthesis. The ultrasound features were mapped to joint kinematics through a regression neural network. Testing of the trained model against untrained kinematics from an altered walking speed show accurate predictions of knee position, knee velocity, ankle position, and ankle velocity, with a normalized RMSE of 9.0 ± 3.1%, 7.3 ± 1.6%, 8.3 ± 2.3%, and 10.0 ± 2.5% respectively. This ultrasound-based prediction suggests that A-mode ultrasound is a viable sensing technology for recognizing user intent. This study is the first necessary step towards implementation of volitional prosthesis controller based on A-mode ultrasound for individuals with transfemoral amputation.