Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning

Artificial intelligence and machine learning approaches in cerebral palsy diagnosis, prognosis, and management: a comprehensive review

Cerebral palsy (CP) is a group of disorders that alters patients' muscle coordination, posture, and movement, resulting in a wide range of deformities. Cerebral palsy can be caused by various factors, both prenatal and postnatal, such as infections or injuries that damage different parts of the brain. As brain plasticity is more prevalent during childhood, early detection can help take the necessary course of management and treatments that would significantly benefit patients by improving their quality of life. Currently, cerebral palsy patients receive regular physiotherapies, occupational therapies, speech therapies, and medications to deal with secondary abnormalities arising due to CP. Advancements in artificial intelligence (AI) and machine learning (ML) over the years have demonstrated the potential to improve the diagnosis, prognosis, and management of CP. This review article synthesizes existing research on AI and ML techniques applied to CP. It provides a comprehensive overview of the role of AI-ML in cerebral palsy, focusing on its applications, benefits, challenges, and future prospects. Through an extensive examination of existing literature, we explore various AI-ML approaches, including but not limited to assessment, diagnosis, treatment planning, and outcome prediction for cerebral palsy. Additionally, we address the ethical considerations, technical limitations, and barriers to the widespread adoption of AI-ML for CP patient care. By synthesizing current knowledge and identifying gaps in research, this review aims to guide future endeavors in harnessing AI-ML for optimizing outcomes and transforming care delivery in cerebral palsy rehabilitation.

Comparison of Empirical and Reinforcement Learning (RL)-Based Control Based on Proximal Policy Optimization (PPO) for Walking Assistance: Does AI Always Win?

The use of wearable assistive devices is growing in both industrial and medical fields. Combining human expertise and artificial intelligence (AI), e.g., in human-in-the-loop-optimization, is gaining popularity for adapting assistance to individuals. Amidst prevailing assertions that AI could surpass human capabilities in customizing every facet of support for human needs, our study serves as an initial step towards such claims within the context of human walking assistance. We investigated the efficacy of the Biarticular Thigh Exosuit, a device designed to aid human locomotion by mimicking the action of the hamstrings and rectus femoris muscles using Serial Elastic Actuators. Two control strategies were tested: an empirical controller based on human gait knowledge and empirical data and a control optimized using Reinforcement Learning (RL) on a neuromuscular model. The performance results of these controllers were assessed by comparing muscle activation in two assisted and two unassisted walking modes. Results showed that both controllers reduced hamstring muscle activation and improved the preferred walking speed, with the empirical controller also decreasing gastrocnemius muscle activity. However, the RL-based controller increased muscle activity in the vastus and rectus femoris, indicating that RL-based enhancements may not always improve assistance without solid empirical support.

Automatic Assist Level Adjustment Function of a Gait Exercise Rehabilitation Robot with Functional Electrical Stimulation for Spinal Cord Injury: Insights from Clinical Trials

This study aimed to identify whether the combined use of functional electrical stimulation (FES) reduces the motor torque of a gait exercise rehabilitation robot in spinal cord injury (SCI) and to verify the effectiveness of the developed automatic assist level adjustment in people with paraplegia. Acute and chronic SCI patients (1 case each) performed 10 min of gait exercises with and without FES using a rehabilitation robot. Reinforcement learning was used to adjust the assist level automatically. The maximum torque values and assist levels for each of the ten walking cycles when walking became steady were averaged and compared with and without FES. The motor's output torque and the assist level were measured as outcomes. The assist level adjustment allowed both the motor torque and assist level to decrease gradually to a steady state. The motor torque and the assist levels were significantly lower with the FES than without the FES under steady conditions in both cases. No adverse events were reported. The combined use of FES attenuated the motor torque of a gait exercise rehabilitation robot for SCI. Automatic assistive level adjustment is also useful for spinal cord injuries.

Human-Exoskeleton Coupling Simulation for Lifting Tasks with Shoulder, Spine, and Knee-Joint Powered Exoskeletons

In this study, we introduce a two-dimensional (2D) human skeletal model coupled with knee, spine, and shoulder exoskeletons. The primary purpose of this model is to predict the optimal lifting motion and provide torque support from the exoskeleton through the utilization of inverse dynamics optimization. The kinematics and dynamics of the human model are expressed using the Denavit-Hartenberg (DH) representation. The lifting optimization formulation integrates the electromechanical dynamics of the DC motors in the exoskeletons of the knee, spine, and shoulder. The design variables for this study include human joint angle profiles and exoskeleton motor current profiles. The optimization objective is to minimize the squared normalized human joint torques, subject to physical and task-specific lifting constraints. We solve this optimization problem using the gradient-based optimizer SNOPT. Our results include a comparison of predicted human joint angle profiles, joint torque profiles, and ground reaction force (GRF) profiles between lifting tasks with and without exoskeleton assistance. We also explore various combinations of exoskeletons for the knee, spine, and shoulder. By resolving the lifting optimization problems, we designed the optimal torques for the exoskeletons located at the knee, spine, and shoulder. It was found that the support from the exoskeletons substantially lowers the torque levels in human joints. Additionally, we conducted experiments only on the knee exoskeleton. Experimental data indicated that using the knee exoskeleton decreases the muscle activation peaks by 35.00%, 10.03%, 22.12%, 30.14%, 16.77%, and 25.71% for muscles of the erector spinae, latissimus dorsi, vastus medialis, vastus lateralis, rectus femoris, and biceps femoris, respectively.

Experiment-free exoskeleton assistance via learning in simulation

Exoskeletons have enormous potential to improve human locomotive performance1-3. However, their development and broad dissemination are limited by the requirement for lengthy human tests and handcrafted control laws2. Here we show an experiment-free method to learn a versatile control policy in simulation. Our learning-in-simulation framework leverages dynamics-aware musculoskeletal and exoskeleton models and data-driven reinforcement learning to bridge the gap between simulation and reality without human experiments. The learned controller is deployed on a custom hip exoskeleton that automatically generates assistance across different activities with reduced metabolic rates by 24.3%, 13.1% and 15.4% for walking, running and stair climbing, respectively. Our framework may offer a generalizable and scalable strategy for the rapid development and widespread adoption of a variety of assistive robots for both able-bodied and mobility-impaired individuals.

Advancements in Sensor Technologies and Control Strategies for Lower-Limb Rehabilitation Exoskeletons: A Comprehensive Review

Lower-limb rehabilitation exoskeletons offer a transformative approach to enhancing recovery in patients with movement disorders affecting the lower extremities. This comprehensive systematic review delves into the literature on sensor technologies and the control strategies integrated into these exoskeletons, evaluating their capacity to address user needs and scrutinizing their structural designs regarding sensor distribution as well as control algorithms. The review examines various sensing modalities, including electromyography (EMG), force, displacement, and other innovative sensor types, employed in these devices to facilitate accurate and responsive motion control. Furthermore, the review explores the strengths and limitations of a diverse array of lower-limb rehabilitation-exoskeleton designs, highlighting areas of improvement and potential avenues for further development. In addition, the review investigates the latest control algorithms and analysis methods that have been utilized in conjunction with these sensor systems to optimize exoskeleton performance and ensure safe and effective user interactions. By building a deeper understanding of the diverse sensor technologies and monitoring systems, this review aims to contribute to the ongoing advancement of lower-limb rehabilitation exoskeletons, ultimately improving the quality of life for patients with mobility impairments.

Physics-Informed Deep Learning for Muscle Force Prediction With Unlabeled sEMG Signals

Computational biomechanical analysis plays a pivotal role in understanding and improving human movements and physical functions. Although physics-based modeling methods can interpret the dynamic interaction between the neural drive to muscle dynamics and joint kinematics, they suffer from high computational latency. In recent years, data-driven methods have emerged as a promising alternative due to their fast execution speed, but label information is still required during training, which is not easy to acquire in practice. To tackle these issues, this paper presents a novel physics-informed deep learning method to predict muscle forces without any label information during model training. In addition, the proposed method could also identify personalized muscle-tendon parameters. To achieve this, the Hill muscle model-based forward dynamics is embedded into the deep neural network as the additional loss to further regulate the behavior of the deep neural network. Experimental validations on the wrist joint from six healthy subjects are performed, and a fully connected neural network (FNN) is selected to implement the proposed method. The predicted results of muscle forces show comparable or even lower root mean square error (RMSE) and higher coefficient of determination compared with baseline methods, which have to use the labeled surface electromyography (sEMG) signals, and it can also identify muscle-tendon parameters accurately, demonstrating the effectiveness of the proposed physics-informed deep learning method.

Motor Interaction Control Based on Muscle Force Model and Depth Reinforcement Strategy

The current motion interaction model has the problems of insufficient motion fidelity and lack of self-adaptation to complex environments. To address this problem, this study proposed to construct a human motion control model based on the muscle force model and stage particle swarm, and based on this, this study utilized the deep deterministic gradient strategy algorithm to construct a motion interaction control model based on the muscle force model and the deep reinforcement strategy. Empirical analysis of the human motion control model proposed in this study revealed that the joint trajectory correlation and muscle activity correlation of the model were higher than those of other comparative models, and its joint trajectory correlation was up to 0.90, and its muscle activity correlation was up to 0.84. In addition, this study validated the effectiveness of the motion interaction control model using the depth reinforcement strategy and found that in the mixed-obstacle environment, the model's desired results were obtained by training 1.1 × 103 times, and the walking distance was 423 m, which was better than other models. In summary, the proposed motor interaction control model using the muscle force model and deep reinforcement strategy has higher motion fidelity and can realize autonomous decision making and adaptive control in the face of complex environments. It can provide a theoretical reference for improving the effect of motion control and realizing intelligent motion interaction.

AI-based methodologies for exoskeleton-assisted rehabilitation of the lower limb: a review

Over the past few years, there has been a noticeable surge in efforts to design novel tools and approaches that incorporate Artificial Intelligence (AI) into rehabilitation of persons with lower-limb impairments, using robotic exoskeletons. The potential benefits include the ability to implement personalized rehabilitation therapies by leveraging AI for robot control and data analysis, facilitating personalized feedback and guidance. Despite this, there is a current lack of literature review specifically focusing on AI applications in lower-limb rehabilitative robotics. To address this gap, our work aims at performing a review of 37 peer-reviewed papers. This review categorizes selected papers based on robotic application scenarios or AI methodologies. Additionally, it uniquely contributes by providing a detailed summary of input features, AI model performance, enrolled populations, exoskeletal systems used in the validation process, and specific tasks for each paper. The innovative aspect lies in offering a clear understanding of the suitability of different algorithms for specific tasks, intending to guide future developments and support informed decision-making in the realm of lower-limb exoskeleton and AI applications.

Design and Analysis of an Upper Limb Rehabilitation Robot Based on Multimodal Control

To address the rehabilitation needs of upper limb hemiplegic patients in various stages of recovery, streamline the workload of rehabilitation professionals, and provide data visualization, our research team designed a six-degree-of-freedom upper limb exoskeleton rehabilitation robot inspired by the human upper limb's structure. We also developed an eight-channel synchronized signal acquisition system for capturing surface electromyography (sEMG) signals and elbow joint angle data. Utilizing Solidworks, we modeled the robot with a focus on modularity, and conducted structural and kinematic analyses. To predict the elbow joint angles, we employed a back propagation neural network (BPNN). We introduced three training modes: a PID control, bilateral control, and active control, each tailored to different phases of the rehabilitation process. Our experimental results demonstrated a strong linear regression relationship between the predicted reference values and the actual elbow joint angles, with an R-squared value of 94.41% and an average error of four degrees. Furthermore, these results validated the increased stability of our model and addressed issues related to the size and single-mode limitations of upper limb rehabilitation robots. This work lays the theoretical foundation for future model enhancements and further research in the field of rehabilitation.

Patent Review of Lower Limb Rehabilitation Robotic Systems by Sensors and Actuation Systems Used

Robotic systems for lower limb rehabilitation are essential for improving patients' physical conditions in lower limb rehabilitation and assisting patients with various locomotor dysfunctions. These robotic systems mainly integrate sensors, actuation, and control systems and combine features from bionics, robotics, control, medicine, and other interdisciplinary fields. Several lower limb robotic systems have been proposed in the patent literature; some are commercially available. This review is an in-depth study of the patents related to robotic rehabilitation systems for lower limbs from the point of view of the sensors and actuation systems used. The patents awarded and published between 2013 and 2023 were investigated, and the temporal distribution of these patents is presented. Our results were obtained by examining the analyzed information from the three public patent databases. The patents were selected so that there were no duplicates after several filters were used in this review. For each patent database, the patents were analyzed according to the category of sensors and the number of sensors used. Additionally, for the main categories of sensors, an analysis was conducted depending on the type of sensors used. Afterwards, the actuation solutions for robotic rehabilitation systems for upper limbs described in the patents were analyzed, highlighting the main trends in their use. The results are presented with a schematic approach so that any user can easily find patents that use a specific type of sensor or a particular type of actuation system, and the sensors or actuation systems recommended to be used in some instances are highlighted.

Generating Human Arm Kinematics Using Reinforcement Learning to Train Active Muscle Behavior in Automotive Research

Computational human body models (HBMs) are important tools for predicting human biomechanical responses under automotive crash environments. In many scenarios, the prediction of the occupant response will be improved by incorporating active muscle control into the HBMs to generate biofidelic kinematics during different vehicle maneuvers. In this study, we have proposed an approach to develop an active muscle controller based on reinforcement learning (RL). The RL muscle activation control (RL-MAC) approach is a shift from using traditional closed-loop feedback controllers, which can mimic accurate active muscle behavior under a limited range of loading conditions for which the controller has been tuned. Conversely, the RL-MAC uses an iterative training approach to generate active muscle forces for desired joint motion and is analogous to how a child develops gross motor skills. In this study, the ability of a deep deterministic policy gradient (DDPG) RL controller to generate accurate human kinematics is demonstrated using a multibody model of the human arm. The arm model was trained to perform goal-directed elbow rotation by activating the responsible muscles and investigated using two recruitment schemes: as independent muscles or as antagonistic muscle groups. Simulations with the trained controller show that the arm can move to the target position in the presence or absence of externally applied loads. The RL-MAC trained under constant external loads was able to maintain the desired elbow joint angle under a simplified automotive impact scenario, implying the robustness of the motor control approach.