A Novel Motion Intention Recognition Approach for Soft Exoskeleton via IMU

Conceptualization of Cloud-Based Motion Analysis and Navigation for Wearable Robotic Applications

The behavior of pedestrians in a non-constrained environment is difficult to predict. In wearable robotics, this poses a challenge, since devices like lower-limb exoskeletons and active orthoses need to support different walking activities, including level walking and climbing stairs. While a fixed movement trajectory can be easily supported, switches between these activities are difficult to predict. Moreover, the demand for these devices is expected to rise in the years ahead. In this work, we propose a cloud software system for use in wearable robotics, based on geographical mapping techniques and Human Activity Recognition (HAR). The system aims to give context to the surrounding pedestrians by providing hindsight information. The system was partially implemented and tested. The results indicate a viable concept with great extensibility prospects.

A Novel Gait Event Detection Algorithm Using a Thigh-Worn Inertial Measurement Unit and Joint Angle Information

This paper describes the development and evaluation of a novel, threshold-based gait event detection algorithm utilizing only one thigh inertial measurement unit (IMU) and unilateral, sagittal plane hip and knee joint angles. The algorithm was designed to detect heel strike (HS) and toe off (TO) gait events, with the eventual goal of detection in a real-time exoskeletal control system. The data used in the development and evaluation of the algorithm were obtained from two gait databases, each containing synchronized IMU and ground reaction force (GRF) data. All database subjects were healthy individuals walking in either a level-ground, urban environment or a treadmill lab environment. Inertial measurements used were three-dimensional thigh accelerations and three-dimensional thigh angular velocities. Parameters for the TO algorithm were identified on a per-subject basis. The GRF data were utilized to validate the algorithm's timing accuracy and quantify the fidelity of the algorithm, measured by the F1-Score. Across all participants, the algorithm reported a mean timing error of -41±20 ms with an F1-Score of 0.988 for HS. For TO, the algorithm reported a mean timing error of -1.4±21 ms with an F1-Score of 0.991. The results of this evaluation suggest that this algorithm is a promising solution to inertial based gait event detection; however, further refinement and real-time evaluation are required for use in exoskeletal control.

Whole-Body Teleoperation Control of Dual-Arm Robot Using Sensor Fusion

As human-robot interaction and teleoperation technologies advance, anthropomorphic control of humanoid arms has garnered increasing attention. However, accurately translating sensor-detected arm motions to the multi-degree freedom of a humanoid robotic arm is challenging, primarily due to occlusion issues with single-sensor setups, which reduce recognition accuracy. To overcome this problem, we propose a human-like arm control strategy based on multi-sensor fusion. We defined the finger bending angle to represent finger posture and employed a depth camera to capture arm movement. Consequently, we developed an arm movement tracking system and achieved anthropomorphic control of the imitation human arm. Finally, we verified our proposed method's effectiveness through a series of experiments, evaluating the system's robustness and real-time performance. The experimental results show that this control strategy can control the motion of the humanoid arm stably, and maintain a high recognition accuracy in the face of complex situations such as occlusion.

Human Posture Transition-Time Detection Based upon Inertial Measurement Unit and Long Short-Term Memory Neural Networks

As human-robot interaction becomes more prevalent in industrial and clinical settings, detecting changes in human posture has become increasingly crucial. While recognizing human actions has been extensively studied, the transition between different postures or movements has been largely overlooked. This study explores using two deep-learning methods, the linear Feedforward Neural Network (FNN) and Long Short-Term Memory (LSTM), to detect changes in human posture among three different movements: standing, walking, and sitting. To explore the possibility of rapid posture-change detection upon human intention, the authors introduced transition stages as distinct features for the identification. During the experiment, the subject wore an inertial measurement unit (IMU) on their right leg to measure joint parameters. The measurement data were used to train the two machine learning networks, and their performances were tested. This study also examined the effect of the sampling rates on the LSTM network. The results indicate that both methods achieved high detection accuracies. Still, the LSTM model outperformed the FNN in terms of speed and accuracy, achieving 91% and 95% accuracy for data sampled at 25 Hz and 100 Hz, respectively. Additionally, the network trained for one test subject was able to detect posture changes in other subjects, demonstrating the feasibility of personalized or generalized deep learning models for detecting human intentions. The accuracies for posture transition time and identification at a sampling rate of 100 Hz were 0.17 s and 94.44%, respectively. In summary, this study achieved some good outcomes and laid a crucial foundation for the engineering application of digital twins, exoskeletons, and human intention control.

Multivariate CNN Model for Human Locomotion Activity Recognition with a Wearable Exoskeleton Robot

This study introduces a novel convolutional neural network (CNN) architecture, encompassing both single and multi-head designs, developed to identify a user's locomotion activity while using a wearable lower limb robot. Our research involved 500 healthy adult participants in an activities of daily living (ADL) space, conducted from 1 September to 30 November 2022. We collected prospective data to identify five locomotion activities (level ground walking, stair ascent/descent, and ramp ascent/descent) across three terrains: flat ground, staircase, and ramp. To evaluate the predictive capabilities of the proposed CNN architectures, we compared its performance with three other models: one CNN and two hybrid models (CNN-LSTM and LSTM-CNN). Experiments were conducted using multivariate signals of various types obtained from electromyograms (EMGs) and the wearable robot. Our results reveal that the deeper CNN architecture significantly surpasses the performance of the three competing models. The proposed model, leveraging encoder data such as hip angles and velocities, along with postural signals such as roll, pitch, and yaw from the wearable lower limb robot, achieved superior performance with an inference speed of 1.14 s. Specifically, the F-measure performance of the proposed model reached 96.17%, compared to 90.68% for DDLMI, 94.41% for DeepConvLSTM, and 95.57% for LSTM-CNN, respectively.

Review of adaptive control for stroke lower limb exoskeleton rehabilitation robot based on motion intention recognition

Stroke is a significant cause of disability worldwide, and stroke survivors often experience severe motor impairments. Lower limb rehabilitation exoskeleton robots provide support and balance for stroke survivors and assist them in performing rehabilitation training tasks, which can effectively improve their quality of life during the later stages of stroke recovery. Lower limb rehabilitation exoskeleton robots have become a hot topic in rehabilitation therapy research. This review introduces traditional rehabilitation assessment methods, explores the possibility of lower limb exoskeleton robots combining sensors and electrophysiological signals to assess stroke survivors' rehabilitation objectively, summarizes standard human-robot coupling models of lower limb rehabilitation exoskeleton robots in recent years, and critically introduces adaptive control models based on motion intent recognition for lower limb exoskeleton robots. This provides new design ideas for the future combination of lower limb rehabilitation exoskeleton robots with rehabilitation assessment, motion assistance, rehabilitation treatment, and adaptive control, making the rehabilitation assessment process more objective and addressing the shortage of rehabilitation therapists to some extent. Finally, the article discusses the current limitations of adaptive control of lower limb rehabilitation exoskeleton robots for stroke survivors and proposes new research directions.

Concurrent validity and reliability of new application for 6-min walk test in healthy adults

Background

Evaluation assessments for physical performance, such as walking tests, are important for measuring a person's well-being. As of current, medical technology is primarily used to administer these assessments. However, medical devices are not easily accessible and are intended for research purposes only, and hence inconvenient for clinical use. Therefore, we aimed to develop a prototype physical performance assessor device with a mobile application and explored concurrent validity and reliability between the standard 6-min walk test (6MWT) and wearable sensor 6MWT using 6-min walk distance in healthy adults.

Methods

Sixty healthy males and females, above 18 years of age, were required to attach a sensor to their dominant ankle while the standard protocol for 6MWT was performed. After completing the walking test, the distance from the wearable sensor 6MWT with a mobile application and the standard 6MWT were recorded and compared.

Results

There was no significant difference between the distance between the standard 6MWT (410.12 ± 74.03 m) and the distance obtained with the wearable sensor. Concurrent validity was found to be moderate, and Cronbach's alpha was 0.79, which indicated good internal consistency.

Conclusion

The innovative prototype wearable walking sensor with a mobile application can effectively evaluate physical performance in healthy individuals.Clinical trial registration number: TCTR20220801002.

Design and Optimization of Lower Limb Rehabilitation Exoskeleton with a Multiaxial Knee Joint

To facilitate rehabilitation training for patients, we proposed the implementation of an anthropomorphic exoskeleton structure that incorporates a variable instantaneous center of rotation (ICR). This design considers the variability in knee ICR among individuals, resulting from the irregular form of the human knee joint, and leverages a double-degrees-of-freedom (2DOF) five-bar mechanism to adapt to these differences. The walking gait of the human lower limb and the corresponding knee ICR were measured and calculated using an optical 3D motion capture system. The optimal dimension parameters of the five-bar mechanism were then obtained through the optimization of human movement position inputs and rod length constraints to minimize the error in knee ICR, gait angle, and ankle trajectory between the human and the exoskeleton. Finally, we established an exoskeleton prototype to conduct relevant experimental tests. The experiment results showed that the average errors of knee ICR trajectory, hip angle, knee angle, and ankle trajectory were 5.52 × 10^-4 m, 0.010 rad, 0.014 rad, and 1.57 × 10^-3 m, respectively. The experimental results demonstrated that the exoskeleton's movement trajectory was close to the human's, reducing the human-mechanism interaction force and improving patient comfort during rehabilitation training.

A Review on Locomotion Mode Recognition and Prediction When Using Active Orthoses and Exoskeletons

Understanding how to seamlessly adapt the assistance of lower-limb wearable assistive devices (active orthosis (AOs) and exoskeletons) to human locomotion modes (LMs) is challenging. Several algorithms and sensors have been explored to recognize and predict the users' LMs. Nevertheless, it is not yet clear which are the most used and effective sensor and classifier configurations in AOs/exoskeletons and how these devices' control is adapted according to the decoded LMs. To explore these aspects, we performed a systematic review by electronic search in Scopus and Web of Science databases, including published studies from 1 January 2010 to 31 August 2022. Sixteen studies were included and scored with 84.7 ± 8.7% quality. Decoding focused on level-ground walking along with ascent/descent stairs tasks performed by healthy subjects. Time-domain raw data from inertial measurement unit sensors were the most used data. Different classifiers were employed considering the LMs to decode (accuracy above 90% for all tasks). Five studies have adapted the assistance of AOs/exoskeletons attending to the decoded LM, in which only one study predicted the new LM before its occurrence. Future research is encouraged to develop decoding tools considering data from people with lower-limb impairments walking at self-selected speeds while performing daily LMs with AOs/exoskeletons.

A CNN–SVM MODEL USING IMU FOR LOCOMOTION MODE RECOGNITION IN LOWER EXTREMITY EXOSKELETON

Human activity intention is indispensable for wearable powered lower extremity exoskeleton such that ensuring the compliant control of the robot. Lots of researches have been done on gait phase detection, which served as a sub-module of locomotion mode recognition to support the follow-on task. Therefore, it is self-evident that locomotion mode recognition is of great importance. Many model-based recognition methods are usually applied in manual extraction of cumbersome features, such as the traditional neural network (NN), support vector machine (SVM), etc. In contrast, the feature mapping layer coming with the convolutional neural network (CNN) can effectively solve the above time-consuming problem. Given that the training of NN is prone to overfitting, SVM with optimal characteristics is considered. A hybrid CNN–SVM model is proposed to identify human locomotion modes by collecting multi-channel inertial measurement unit (IMU) signals and is integrated with the error correction function of the finite state machine (FSM). Therefore, the CNN–SVM model has great influence on the generalization performance and recognition accuracy. The recognition rates of five single locomotion modes and eight mixed locomotion modes reach 97.91% and 98.93%, respectively. The system meets the demand of real-time performance, and the recognition time exceeds 370[Formula: see text]ms on heel strike.

Measurement, Evaluation, and Control of Active Intelligent Gait Training Systems—Analysis of the Current State of the Art

Gait recognition and rehabilitation has been a research hotspot in recent years due to its importance to medical care and elderly care. Active intelligent rehabilitation and assistance systems for lower limbs integrates mechanical design, sensing technology, intelligent control, and robotics technology, and is one of the effective ways to resolve the above problems. In this review, crucial technologies and typical prototypes of active intelligent rehabilitation and assistance systems for gait training are introduced. The limitations, challenges, and future directions in terms of gait measurement and intention recognition, gait rehabilitation evaluation, and gait training control strategies are discussed. To address the core problems of the sensing, evaluation and control technology of the active intelligent gait training systems, the possible future research directions are proposed. Firstly, different sensing methods need to be proposed for the decoding of human movement intention. Secondly, the human walking ability evaluation models will be developed by integrating the clinical knowledge and lower limb movement data. Lastly, the personalized gait training strategy for collaborative control of human–machine systems needs to be implemented in the clinical applications.

Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism

A challenging task for the biological neural signal-based human-exoskeleton interface is to achieve accurate lower limb movement prediction of patients with hemiplegia in rehabilitation training scenarios. The human-exoskeleton interface based on single-modal biological signals such as electroencephalogram (EEG) is currently not mature in predicting movements, due to its unreliability. The multimodal human-exoskeleton interface is a very novel solution to this problem. This kind of interface normally combines the EEG signal with surface electromyography (sEMG) signal. However, their use for the lower limb movement prediction is still limited—the connection between sEMG and EEG signals and the deep feature fusion between them are ignored. In this article, a Dense con-attention mechanism-based Multimodal Enhance Fusion Network (DMEFNet) is proposed for predicting lower limb movement of patients with hemiplegia. The DMEFNet introduces the con-attention structure to extract the common attention between sEMG and EEG signal features. To verify the effectiveness of DMEFNet, an sEMG and EEG data acquisition experiment and an incomplete asynchronous data collection paradigm are designed. The experimental results show that DMEFNet has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 82.96 and 88.44%, respectively.

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

In recent decades, although the research on gait recognition of lower limb exoskeleton robot has been widely developed, there are still limitations in rehabilitation training and clinical practice. The emergence of interactive information fusion technology provides a new research idea for the solution of this problem, and it is also the development trend in the future. In order to better explore the issue, this paper summarizes gait recognition based on interactive information fusion of lower limb exoskeleton robots. This review introduces the current research status, methods, and directions for information acquisition, interaction, fusion, and gait recognition of exoskeleton robots. The content involves the research progress of information acquisition methods, sensor placements, target groups, lower limb sports biomechanics, interactive information fusion, and gait recognition model. Finally, the current challenges, possible solutions, and promising prospects are analysed and discussed, which provides a useful reference resource for the study of interactive information fusion and gait recognition of rehabilitation exoskeleton robots.

MCSNet: Channel Synergy-Based Human-Exoskeleton Interface With Surface Electromyogram

The human-robot interface (HRI) based on biological signals can realize the natural interaction between human and robot. It has been widely used in exoskeleton robots recently to help predict the wearer's movement. Surface electromyography (sEMG)-based HRI has mature applications on the exoskeleton. However, the sEMG signals of paraplegic patients' lower limbs are weak, which means that most HRI based on lower limb sEMG signals cannot be applied to the exoskeleton. Few studies have explored the possibility of using upper limb sEMG signals to predict lower limb movement. In addition, most HRIs do not consider the contribution and synergy of sEMG signal channels. This paper proposes a human-exoskeleton interface based on upper limb sEMG signals to predict lower limb movements of paraplegic patients. The interface constructs an channel synergy-based network (MCSNet) to extract the contribution and synergy of different feature channels. An sEMG data acquisition experiment is designed to verify the effectiveness of MCSNet. The experimental results show that our method has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 94.51 and 80.75%, respectively. Furthermore, feature visualization and model ablation analysis show that the features extracted by MCSNet are physiologically interpretable.

Channel Synergy-based Human-Robot Interface for a Lower Limb Walking Assistance Exoskeleton

The human-robot interface (HRI) based on surface electromyography(sEMG) can realize the natural interaction between human and robot. It has been widely used in exoskeleton robots recently to help predict the wearer's movement. The sEMG signal of the paraplegic patients' lower limbs is weak. How to achieve accurate prediction of the lower limb movement of patients with paraplegia has always been the focus of attention in the field of HRI. Few studies have explored the possibility of using upper limb sEMG signals to predict lower limb movement. In addition, most HRIs do not consider the contribution and synergy of sEMG signal channels. This paper proposes a human-exoskeleton interface based on upper limb sEMG signals to predict lower limb movements of paraplegic patients. The interface constructs a channel synergy-based network (MCSNet) to extract the contribution and synergy of different feature channels. An sEMG data acquisition experiment is designed to verify the effectiveness of MCSNet. The experimental results show that our method has a good movement prediction performance in both within-subject and cross-subject situations, reaching an accuracy of 94.51% and 80.75% respectively.

Kinematics, Speed, and Anthropometry-Based Ankle Joint Torque Estimation: A Deep Learning Regression Approach

Powered Assistive Devices (PADs) have been proposed to enable repetitive, user-oriented gait rehabilitation. They may include torque controllers that typically require reference joint torque trajectories to determine the most suitable level of assistance. However, a robust approach able to automatically estimate user-oriented reference joint torque trajectories, namely ankle torque, while considering the effects of varying walking speed, body mass, and height on the gait dynamics, is needed. This study evaluates the accuracy and generalization ability of two Deep Learning (DL) regressors (Long-Short Term Memory and Convolutional Neural Network (CNN)) to generate user-oriented reference ankle torque trajectories by innovatively customizing them according to the walking speed (ranging from 1.0 to 4.0 km/h) and users’ body height and mass (ranging from 1.51 to 1.83 m and 52.0 to 83.7 kg, respectively). Furthermore, this study hypothesizes that DL regressors can estimate joint torque without resourcing electromyography signals. CNN was the most robust algorithm (Normalized Root Mean Square Error: 0.70 ± 0.06; Spearman Correlation: 0.89 ± 0.03; Coefficient of Determination: 0.91 ± 0.03). No statistically significant differences were found in CNN accuracy (p-value > 0.05) whether electromyography signals are included as inputs or not, enabling a less obtrusive and accurate setup for torque estimation.