Spatial and temporal attention embedded spatial temporal graph convolutional networks for skeleton based gait recognition with multiple IMUs

Gait recognition is one of the key technologies for exoskeleton robot control, while the current IMU-based gait recognition methods only use inertial data and do not fully consider the interconnections of human spatial structure and human joints. In this regard, a skeleton-based gait recognition approach with inertial measurement units using spatial temporal graph convolutional networks with spatial and temporal attention is proposed. A human forward kinematics solver module was used for constructing different human skeleton models and a temporal attention module was added for capturing the more important time frames in the gait cycle. Moreover, the two-stream structure was used to construct spatial temporal graph convolutional networks with spatial and temporal attention for gait recognition, and an average accuracy of about 99% was obtained in user experiments, which is the best performance compared to other algorithms, provides certain reference for gait recognition and real-time control of exoskeleton robots.

Lower Limb Exoskeleton for Rehabilitation with Flexible Joints and Movement Routines Commanded by Electromyography and Baropodometry Sensors

This paper presents the development of an instrumented exoskeleton with baropodometry, electromyography, and torque sensors. The six degrees of freedom (Dof) exoskeleton has a human intention detection system based on a classifier of electromyographic signals coming from four sensors placed in the muscles of the lower extremity together with baropodometric signals from four resistive load sensors placed at the front and rear parts of both feet. In addition, the exoskeleton is instrumented with four flexible actuators coupled with torque sensors. The main objective of the paper was the development of a lower limb therapy exoskeleton, articulated at hip and knees to allow the performance of three types of motion depending on the detected user's intention: sitting to standing, standing to sitting, and standing to walking. In addition, the paper presents the development of a dynamical model and the implementation of a feedback control in the exoskeleton.

Classification of deep vein thrombosis stages using convolutional neural network of electromyogram with vibrotactile stimulation toward developing an early diagnostic tool: A preliminary study on a pig model

Deep vein thrombosis (DVT) can lead to life-threatening disorders; however, it can only be recognized after its symptom appear. This study proposed a novel method that can detect the early stage of DVT using electromyography (EMG) signals with vibration stimuli using the convolutional neural networks (CNN) algorithm. The feasibility of the method was tested with eight legs before and after the surgical induction of DVT at nine-time points. Furthermore, perfusion pressure (PP), intracompartmental pressure (IP), and shear elastic modulus (SEM) of the tibialis anterior were also collected. In the proposed method, principal component analysis (PCA) and CNN were used to analyze the EMG data and classify it before and after the DVT stages. The cross-validation was performed in two strategies. One is for each leg and the other is the leave-one-leg-out (LOLO), test without any predicted information, for considering the practical diagnostic tool. The results showed that PCA-CNN can classify before and after DVT stages with an average accuracy of 100% (each leg) and 68.4±20.5% (LOLO). Moreover, all-time points (before induction of DVT and eight-time points after DVT) were classified with an average accuracy of 72.0±11.9% which is substantially higher accuracy than the chance levels (11% for 9-class classification). Based on the experimental results in the pig model, the proposed CNN-based method can classify the before- and after-DVT stages with high accuracy. The experimental results can provide a basis for further developing an early diagnostic tool for DVT using only EMG signals with vibration stimuli.

EEG guided electrical stimulation parameters generation from texture force profiles

Objective.Our aim is to enhance sensory perception and spatial presence in artificial interfaces guided by EEG. This is done by developing a closed-loop electro-tactile system guided by EEG that adaptively update the electrical stimulation parameters to achieve EEG responses similar to the EEG responses generated from touching textured surface.Approach.In this work, we introduce a model that defines the relationship between the contact force profiles and the electrical stimulation parameters. This is done by using the EEG and force data collected from two experiments. The first was conducted by moving a set of textured surfaces against the subjects' fingertip, while collecting both EEG and force data. Whereas the second was carried out by applying a set of different pulse and amplitude modulated electrical stimuli to the subjects' index finger while recording EEG.Main results.We were able to develop a model which could generate electrical stimulation parameters corresponding to different textured surfaces. We showed by offline testing and validation analysis that the average error between the EEG generated from the estimated electrical stimulation parameters and the actual EEG generated from touching textured surfaces is around 7%.Significance.Haptic feedback plays a vital role in our daily life, as it allows us to become aware of our environment. Even though a number of methods have been developed to measure perception of spatial presence and provide sensory feedback in virtual reality environments, there is currently no closed-loop control of sensory stimulation. The proposed model provides an initial step towards developing a closed loop electro-tactile haptic feedback model that delivers more realistic touch sensation through electrical stimulation.

Design and Assist-as-Needed Control of Flexible Elbow Exoskeleton Actuated by Nonlinear Series Elastic Cable Driven Mechanism

Exoskeletons can assist the daily life activities of the elderly with weakened muscle strength, but traditional rigid exoskeletons bring parasitic torque to the human joints and easily disturbs the natural movement of the wearer’s upper limbs. Flexible exoskeletons have more natural human-machine interaction, lower weight and cost, and have great application potential. Applying assist force according to the patient’s needs can give full play to the wearer’s remaining muscle strength, which is more conducive to muscle strength training and motor function recovery. In this paper, a design scheme of an elbow exoskeleton driven by flexible antagonistic cable actuators is proposed. The cable actuator is driven by a nonlinear series elastic mechanism, in which the elastic elements simulate the passive elastic properties of human skeletal muscle. Based on an improved elbow musculoskeletal model, the assist torque of exoskeleton is predicted. An assist-as-needed (AAN) control algorithm is proposed for the exoskeleton and experiments are carried out. The experimental results on the experimental platform show that the root mean square error between the predicted assist torque and the actual assist torque is 0.00226 Nm. The wearing experimental results also show that the AAN control method designed in this paper can reduce the activation of biceps brachii effectively when the exoskeleton assist level increases.

Joint Torque Prediction via Hybrid Neuromusculoskeletal Modelling during Gait Using Statistical Ground Reaction Estimates: An Exploratory Study

Joint torques of lower extremity are important clinical indicators of gait capability. This parameter can be quantified via hybrid neuromusculoskeletal modelling that combines electromyography-driven modelling and static optimisation. The simulations rely on kinematics and external force measurements, for example, ground reaction forces (GRF) and the corresponding centres of pressure (COP), which are conventionally acquired using force plates. This bulky equipment, however, hinders gait analysis in real-world environments. While this portability issue could potentially be solved by estimating the parameters through machine learning, the effect of the estimation errors on joint torque prediction with biomechanical models remains to be investigated. This study first estimated GRF and COP through feedforward artificial neural networks, and then leveraged them to predict lower-limb sagittal joint torques via (i) inverse dynamics and (ii) hybrid modelling. The approach was evaluated on five healthy subjects, individually. The predicted torques were validated with the measured torques, showing that hip was the most sensitive whereas ankle was the most resistive to the GRF/COP estimates for both models, with average metrics values being 0.70 < R2 < 0.97 and 0.069 < RMSE < 0.15 (Nm/kg). This study demonstrated the feasibility of torque prediction based on personalised (neuro)musculoskeletal modelling using statistical ground reaction estimates, thus providing insights into potential real-world mobile joint torque quantification.

Gait Trajectory and Gait Phase Prediction Based on an LSTM Network

Lower body segment trajectory and gait phase prediction is crucial for the control of assistance-as-needed robotic devices, such as exoskeletons. In order for a powered exoskeleton with phase-based control to determine and provide proper assistance to the wearer during gait, we propose an approach to predict segment trajectories up to 200 ms ahead (angular velocity of the thigh, shank and foot segments) and five gait phases (loading response, mid-stance, terminal stance, preswing and swing), based on collected data from inertial measurement units placed on the thighs, shanks, and feet. The approach we propose is a long-short term memory (LSTM)-based network, a modified version of recurrent neural networks, which can learn order dependence in sequence prediction problems. The algorithm proposed has a weighted discount loss function that places more weight in predicting the next three to five time frames but also contributes to an overall prediction performance for up to 10 time frames. The LSTM model was designed to learn lower limb segment trajectories using training samples and was tested for generalization across participants. All predicted trajectories were strongly correlated with the measured trajectories, with correlation coefficients greater than 0.98. The proposed LSTM approach can also accurately predict the five gait phases, particularly swing phase with 95% accuracy in inter-subject implementation. The ability of the LSTM network to predict future gait trajectories and gait phases can be applied in designing exoskeleton controllers that can better compensate for system delays to smooth the transition between gait phases.

Prediction of Passive Torque on Human Shoulder Joint Based on BPANN

In upper limb rehabilitation training by exploiting robotic devices, the qualitative or quantitative assessment of human active effort is conducive to altering the robot control parameters to offer the patients appropriate assistance, which is considered an effective rehabilitation strategy termed as assist-as-needed. Since active effort of a patient is changeable for the conscious or unconscious behavior, it is considered to be more feasible to determine the distributions of the passive resistance of the patient's joints versus the joint angle in advance, which can be adopted to assess the active behavior of patients combined with the measurement of robotic sensors. However, the overintensive measurements can impose a burden on patients. Accordingly, a prediction method of shoulder joint passive torque based on a Backpropagation neural network (BPANN) was proposed in the present study to expand the passive torque distribution of the shoulder joint of a patient with less measurement data. The experiments recruiting three adult male subjects were conducted, and the results revealed that the BPANN exhibits high prediction accurate for each direction shoulder passive torque. The results revealed that the BPANN can learn the nonlinear relationship between the passive torque and the position of the shoulder joint and can make an accurate prediction without the need to build a force distribution function in advance, making it possible to draw up an assist-as-needed strategy with high accuracy while reducing the measurement burden of patients and physiotherapists.

A New Impedance Controller Based on Nonlinear Model Reference Adaptive Control for Exoskeleton Systems

Robotic exoskeletons are expected to show high compliance and low impedance for human–robot interactions (HRIs). Our study introduces a novel method based on nonlinear model reference adaptive control (MRAC) to reduce the inherent impedance and replace the traditional impedance controller in HRIs. The control law and adaptive law are designed according to a candidate Lyapunov function. A simple system identification and initialization method for the nonlinear MRAC is put forward, which provides a set of better initial values for the controller. From the results of simulation and experiment, our controller can reduce the mechanical impedance and achieve high compliance for HRI. The adaptive control and compliance control can be both achieved by the proposed nonlinear MRAC framework.

REHABILITATIVE STRATEGIES OF MULTIPLE LOWER LIMBS TRAINING MODELS

According to the combination of lower limbs rehabilitative robot (LLRR), the effect of multi-point and single point driving form on muscle force and joint torque is explored, and the rehabilitation effect of the training mode of the active–passive rehabilitation training is studied. The musculoskeletal model of lower limbs is established based on the physiological structure of human lower limbs. And considering the position of the attachment points of each muscle, the mechanical properties of muscles and applied moment of joints can be obtained under different rehabilitative training strategies by inverse dynamic analysis. The rehabilitation training strategies of flexion–extension and abduction and adduction movements are put forward according to the movement of lower limbs. And using the wire-driven rehabilitation robot as the driving device of the rehabilitation training, the robot is used to simulate the motor function of patients’ lower limbs by modifying the parameters of muscle which can affect the resistance moment of joint motion, then the effects of driving form and the active–passive training mode are analyzed. The results show that single point driving form is better than multi-point on muscle strength and joint strength training; the rehabilitation training strategies of flexion–extension and abduction–adduction movements show different superiority.

Adaptive Admittance Control for an Ankle Exoskeleton Using an EMG-Driven Musculoskeletal Model

Various rehabilitation robots have been employed to recover the motor function of stroke patients. To improve the effect of rehabilitation, robots should promote patient participation and provide compliant assistance. This paper proposes an adaptive admittance control scheme (AACS) consisting of an admittance filter, inner position controller, and electromyography (EMG)-driven musculoskeletal model (EDMM). The admittance filter generates the subject's intended motion according to the joint torque estimated by the EDMM. The inner position controller tracks the intended motion, and its parameters are adjusted according to the estimated joint stiffness. Eight healthy subjects were instructed to wear the ankle exoskeleton robot, and they completed a series of sinusoidal tracking tasks involving ankle dorsiflexion and plantarflexion. The robot was controlled by the AACS and a non-adaptive admittance control scheme (NAACS) at four fixed parameter levels. The tracking performance was evaluated using the jerk value, position error, interaction torque, and EMG levels of the tibialis anterior (TA) and gastrocnemius (GAS). For the NAACS, the jerk value and position error increased with the parameter levels, and the interaction torque and EMG levels of the TA tended to decrease. In contrast, the AACS could maintain a moderate jerk value, position error, interaction torque, and TA EMG level. These results demonstrate that the AACS achieves a good tradeoff between accurate tracking and compliant assistance because it can produce a real-time response to stiffness changes in the ankle joint. The AACS can alleviate the conflict between accurate tracking and compliant assistance and has potential for application in robot-assisted rehabilitation.

fNIRS-based Neurorobotic Interface for gait rehabilitation

BACKGROUND

In this paper, a novel functional near-infrared spectroscopy (fNIRS)-based brain-computer interface (BCI) framework for control of prosthetic legs and rehabilitation of patients suffering from locomotive disorders is presented.

METHODS

fNIRS signals are used to initiate and stop the gait cycle, while a nonlinear proportional derivative computed torque controller (PD-CTC) with gravity compensation is used to control the torques of hip and knee joints for minimization of position error. In the present study, the brain signals of walking intention and rest tasks were acquired from the left hemisphere's primary motor cortex for nine subjects. Thereafter, for removal of motion artifacts and physiological noises, the performances of six different filters (i.e. Kalman, Wiener, Gaussian, hemodynamic response filter (hrf), Band-pass, finite impulse response) were evaluated. Then, six different features were extracted from oxygenated hemoglobin signals, and their different combinations were used for classification. Also, the classification performances of five different classifiers (i.e. k-Nearest Neighbour, quadratic discriminant analysis, linear discriminant analysis (LDA), Naïve Bayes, support vector machine (SVM)) were tested.

RESULTS

The classification accuracies obtained from SVM using the hrf were significantly higher (p < 0.01) than those of the other classifier/ filter combinations. Those accuracies were 77.5, 72.5, 68.3, 74.2, 73.3, 80.8, 65, 76.7, and 86.7% for the nine subjects, respectively.

CONCLUSION

The control commands generated using the classifiers initiated and stopped the gait cycle of the prosthetic leg, the knee and hip torques of which were controlled using the PD-CTC to minimize the position error. The proposed scheme can be effectively used for neurofeedback training and rehabilitation of lower-limb amputees and paralyzed patients.

Robotics in Lower-Limb Rehabilitation after Stroke

With the increase in the elderly, stroke has become a common disease, often leading to motor dysfunction and even permanent disability. Lower-limb rehabilitation robots can help patients to carry out reasonable and effective training to improve the motor function of paralyzed extremity. In this paper, the developments of lower-limb rehabilitation robots in the past decades are reviewed. Specifically, we provide a classification, a comparison, and a design overview of the driving modes, training paradigm, and control strategy of the lower-limb rehabilitation robots in the reviewed literature. A brief review on the gait detection technology of lower-limb rehabilitation robots is also presented. Finally, we discuss the future directions of the lower-limb rehabilitation robots.