NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

Walking speeds are lower for short distance and turning locomotion: Experiments and modeling in low-cost prosthesis users

Preferred walking speed is a widely-used performance measure for people with mobility issues, but is usually measured in straight line walking for fixed distances or durations, and without explicitly accounting for turning. However, daily walking involves walking for bouts of different distances and walking with turning, with prior studies showing that short bouts with at most 10 steps could be 40% of all bouts and turning steps could be 8-50% of all steps. Here, we studied walking in a straight line for short distances (4 m to 23 m) and walking in circles (1 m to 3 m turning radii) in people with transtibial amputation or transfemoral amputation using a passive ankle-foot prosthesis (Jaipur Foot). We found that the study participants' preferred walking speeds are lower for shorter straight-line walking distances and lower for circles of smaller radii, which is analogous to earlier results in subjects without amputation. Using inverse optimization, we estimated the cost of changing speeds and turning such that the observed preferred walking speeds in our experiments minimizes the total cost of walking. The inferred costs of changing speeds and turning were larger for subjects with amputation compared to subjects without amputation in a previous study, specifically, being 4x to 8x larger for the turning cost and being highest for subjects with transfemoral amputation. Such high costs inferred by inverse optimization could potentially include non-energetic costs such as due to joint or interfacial stress or stability concerns, as inverse optimization cannot distinguish such terms from true metabolic cost. These experimental findings and models capturing the experimental trends could inform prosthesis design and rehabilitation therapy to better assist changing speeds and turning tasks. Further, measuring the preferred speed for a range of distances and radii could be a more comprehensive subject-specific measure of walking performance than commonly used straight line walking metrics.

Minimum effort simulations of split-belt treadmill walking exploit asymmetry to reduce metabolic energy expenditure

Walking on a split-belt treadmill elicits an adaptation response that changes baseline step length asymmetry. The underlying causes of this adaptation, however, are difficult to determine. It has been proposed that effort minimization may drive this adaptation, based on the idea that adopting longer steps on the fast belt, or positive step length asymmetry (SLA), can cause the treadmill to exert net-positive mechanical work on a bipedal walker. However, humans walking on split-belt treadmills have not been observed to reproduce this behavior when allowed to freely adapt. To determine if an effort-minimization motor control strategy would result in experimentally observed adaptation patterns, we conducted simulations of walking on different combinations of belt speeds with a human musculoskeletal model that minimized muscle excitations and metabolic rate. The model adopted increasing amounts of positive SLA and decreased its net metabolic rate with increasing belt speed difference, reaching +42.4% SLA and -5.7% metabolic rate relative to tied-belt walking at our maximum belt speed ratio of 3:1. These gains were primarily enabled by an increase in braking work and a reduction in propulsion work on the fast belt. The results suggest that a purely effort minimization driven split-belt walking strategy would involve substantial positive SLA, and that the lack of this characteristic in human behavior points to additional factors influencing the motor control strategy, such as aversion to excessive joint loads, asymmetry, or instability.NEW & NOTEWORTHY Behavioral observations of split-belt treadmill adaptation have been inconclusive toward its underlying causes. To estimate gait patterns when driven exclusively by one of these possible underlying causes, we simulated split-belt treadmill walking with a musculoskeletal model that minimized its summed muscle excitations. Our model took significantly longer steps on the fast belt and reduced its metabolic rate below tied-belt walking, unlike experimental observations. This suggests that asymmetry is energetically optimal, but human adaptation involves additional factors.

Inertia-Constrained Reinforcement Learning to Enhance Human Motor Control Modeling

Locomotor impairment is a highly prevalent and significant source of disability and significantly impacts the quality of life of a large portion of the population. Despite decades of research on human locomotion, challenges remain in simulating human movement to study the features of musculoskeletal drivers and clinical conditions. Most recent efforts to utilize reinforcement learning (RL) techniques are promising in the simulation of human locomotion and reveal musculoskeletal drives. However, these simulations often fail to mimic natural human locomotion because most reinforcement strategies have yet to consider any reference data regarding human movement. To address these challenges, in this study, we designed a reward function based on the trajectory optimization rewards (TOR) and bio-inspired rewards, which includes the rewards obtained from reference motion data captured by a single Inertial Moment Unit (IMU) sensor. The sensor was equipped on the participants' pelvis to capture reference motion data. We also adapted the reward function by leveraging previous research on walking simulations for TOR. The experimental results showed that the simulated agents with the modified reward function performed better in mimicking the collected IMU data from participants, which means that the simulated human locomotion was more realistic. As a bio-inspired defined cost, IMU data enhanced the agent's capacity to converge during the training process. As a result, the models' convergence was faster than those developed without reference motion data. Consequently, human locomotion can be simulated more quickly and in a broader range of environments, with a better simulation performance.

Acquisition of bipedal locomotion in a neuromusculoskeletal model with unilateral transtibial amputation

Adaptive locomotion is an essential behavior for animals to survive. The central pattern generator in the spinal cord is responsible for the basic rhythm of locomotion through sensory feedback coordination, resulting in energy-efficient locomotor patterns. Individuals with symmetrical body proportions exhibit an energy-efficient symmetrical gait on flat ground. In contrast, individuals with lower limb amputation, who have morphologically asymmetrical body proportions, exhibit asymmetrical gait patterns. However, it remains unclear how the nervous system adjusts the control of the lower limbs. Thus, in this study, we investigated how individuals with unilateral transtibial amputation control their left and right lower limbs during locomotion using a two-dimensional neuromusculoskeletal model. The model included a musculoskeletal model with 7 segments and 18 muscles, as well as a neural model with a central pattern generator and sensory feedback systems. Specifically, we examined whether individuals with unilateral transtibial amputation acquire prosthetic gait through a symmetric or asymmetric feedback control for the left and right lower limbs. After acquiring locomotion, the metabolic costs of transport and the symmetry of the spatiotemporal gait factors were evaluated. Regarding the metabolic costs of transportation, the symmetric control model showed values approximately twice those of the asymmetric control model, whereas both scenarios showed asymmetry of spatiotemporal gait patterns. Our results suggest that individuals with unilateral transtibial amputation can reacquire locomotion by modifying sensory feedback parameters. In particular, the model reacquired reasonable locomotion for activities of daily living by re-searching asymmetric feedback parameters for each lower limb. These results could provide insight into effective gait assessment and rehabilitation methods to reacquire locomotion in individuals with unilateral transtibial amputation.

Be Careful What You Wish for: Cost Function Sensitivity in Predictive Simulations for Assistive Device Design

Software packages that use optimization to predict the motion of dynamic systems are powerful tools for studying human movement. These “predictive simulations” are gaining popularity in parameter optimization studies for designing assistive devices such as exoskeletons. The cost function is a critical component of the optimization problem and can dramatically affect the solution. Many cost functions have been proposed that are biologically inspired and that produce reasonable solutions, but which may lead to different conclusions in some contexts. We used OpenSim Moco to generate predictive simulations of human walking using several cost functions, each of which produced a reasonable trajectory of the human model. We then augmented the model with motors that generated hip flexion, knee flexion, or ankle plantarflexion torques, and repeated the predictive simulations to determine the optimal motor torques. The model was assumed to be planar and bilaterally symmetric to reduce computation time. Peak torques varied from 41.3 to 79.0 N·m for the hip flexion motors, from 48.0 to 94.2 N·m for the knee flexion motors, and from 42.6 to 79.8 N·m for the ankle plantarflexion motors, which could have important design consequences. This study highlights the importance of evaluating the robustness of results from predictive simulations.

Performance of the CYBERLEGs motorized lower limb prosthetic device during simulated daily activities

Background

The CYBERLEGs-gamma (CLs-ɣ) prosthesis has been developed to investigate the possibilities of powerful active prosthetics in restoring human gait capabilities after lower limb amputation.

Objective

The objective of this study was to determine the performance of the CLs-ɣ prosthesis during simulated daily activities.

Methods

Eight participants with a transfemoral amputation (age: 55 ± 15 years, K-level 3, registered under: NCT03376919) performed a familiarization session, an experimental session with their current prosthesis, three training sessions with the CLs-ɣ prosthesis and another experimental session with the CLs-ɣ prosthesis. Participants completed a stair-climbing-test, a timed-up-and-go-test, a sit-to stand-test, a 2-min dual-task and a 6-min treadmill walk test.

Results

Comparisons between the two experimental sessions showed that stride length significantly increased during walking with the CLs-ɣ prosthesis (p = .012) due to a greater step length of the amputated leg (p = .035). Although a training period with the prototype was included, preferred walking speed was significantly slower (p = .018), the metabolic cost of transport was significantly higher (p = .028) and reaction times significantly worsened (p = .012) when walking with the CLs-ɣ compared to the current prosthesis.

Conclusions

It can be stated that a higher physical and cognitive effort were required when wearing the CLs-ɣ prosthesis. Positive outcomes were observed regarding stride length and stair ambulation. Future prosthetics development should minimize the weight of the device and integrate customized control systems. A recommendation for future research is to include several shorter training periods or a prolonged adaptation period.

Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation

Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This “Learn to Move” competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research

Disentangling the energetic costs of step time asymmetry and step length asymmetry in human walking

The metabolic cost of walking in healthy individuals increases with spatiotemporal gait asymmetries. Pathological gait, such as post-stroke, often has asymmetry in step length and step time which may contribute to an increased energy cost. But paradoxically, enforcing step length symmetry does not reduce metabolic cost of post-stroke walking. The isolated and interacting costs of asymmetry in step time and step length remain unclear, because previous studies did not simultaneously enforce spatial and temporal gait asymmetries. Here, we delineate the isolated costs of asymmetry in step time and step length in healthy human walking. We first show that the cost of step length asymmetry is predicted by the cost of taking two non-preferred step lengths (one short and one long), but that step time asymmetry adds an extra cost beyond the cost of non-preferred step times. The metabolic power of step time asymmetry is about 2.5 times greater than the cost of step length asymmetry. Furthermore, the costs are not additive when walking with asymmetric step time and asymmetric step length: the metabolic power of concurrent asymmetry in step length and step time is driven by the cost of step time asymmetry alone. The metabolic power of asymmetry is explained by positive mechanical power produced during single support phases to compensate for a net loss of center of mass power incurred during double support phases. These data may explain why metabolic cost remains invariant to step length asymmetry in post-stroke walking and suggest how effects of asymmetry on energy cost can be attenuated.

Improving the energy economy of human running with powered and unpowered ankle exoskeleton assistance

Exoskeletons that reduce energetic cost could make recreational running more enjoyable and improve running performance. Although there are many ways to assist runners, the best approaches remain unclear. In our study, we used a tethered ankle exoskeleton emulator to optimize both powered and spring-like exoskeleton characteristics while participants ran on a treadmill. We expected powered conditions to provide large improvements in energy economy and for spring-like patterns to provide smaller benefits achievable with simpler devices. We used human-in-the-loop optimization to attempt to identify the best exoskeleton characteristics for each device type and individual user, allowing for a well-controlled comparison. We found that optimized powered assistance improved energy economy by 24.7 ± 6.9% compared with zero torque and 14.6 ± 7.7% compared with running in normal shoes. Optimized powered torque patterns for individuals varied substantially, but all resulted in relatively high mechanical work input (0.36 ± 0.09 joule kilogram^-1 per step) and late timing of peak torque (75.7 ± 5.0% stance). Unexpectedly, spring-like assistance was ineffective, improving energy economy by only 2.1 ± 2.4% compared with zero torque and increasing metabolic rate by 11.1 ± 2.8% compared with control shoes. The energy savings we observed imply that running velocity could be increased by as much as 10% with no added effort for the user and could influence the design of future products.

Load carrying with flexible bamboo poles: optimization of a coupled oscillator system

In Asia, flexible bamboo poles are routinely used to carry substantial loads on the shoulder. Various advantages have been attributed to this load-carrying strategy (e.g. reduced energy consumption), but experimental evidence remains inconsistent – possibly because carriers in previous studies were inexperienced. Theoretical models typically neglect the individual's capacity to optimize interactions with the oscillating load, leaving the complete dynamics underexplored. This study used a trajectory optimization model to predict gait adaptations that minimize work-based costs associated with carrying compliant loads and compared the outcomes with naturally selected gait adaptations of experienced pole carriers. Gait parameters and load interactions (e.g. relative amplitude and frequency, phase) were measured in rural farmworkers in Vietnam. Participants carried a range of loads with compliant and rigid poles and the energetic consequences of step frequency adjustments were evaluated using the model. When carrying large loads, the empirical step frequency changes associated with pole type (compliant versus rigid) were largely consistent with model predictions, in terms of direction (increase or decrease) and magnitude (by how much). Work-minimizing strategies explain changes in leg compliance, harmonic frequency oscillations and fluctuations in energetic cost associated with carrying loads on a compliant bamboo pole.

Effects of extended powered knee prosthesis stance time via visual feedback on gait symmetry of individuals with unilateral amputation: a preliminary study

BACKGROUND

Establishing gait symmetry is a major aim of amputee rehabilitation and may be more attainable with powered prostheses. Though, based on previous work, we postulate that users transfer a previously-learned motor pattern across devices, limiting the functionality of more advanced prostheses. The objective of this study was to preliminarily investigate the effect of increased stance time via visual feedback on amputees' gait symmetry using powered and passive knee prostheses.

METHODS

Five individuals with transfemoral amputation or knee disarticulation walked at their self-selected speed on a treadmill. Visual feedback was used to promote an increase in the amputated-limb stance time. Individuals were fit with a commercially-available powered prosthesis by a certified prosthetist and practiced walking during a prior visit. The same protocol was completed with a passive knee and powered knee prosthesis on separate days. We used repeated-measures, two-way ANOVA (alpha = 0.05) to test for significant effects of the feedback and device factors. Our main outcome measures were stance time asymmetry, peak anterior-posterior ground reaction forces, and peak anterior propulsion asymmetry.

RESULTS

Increasing the amputated-limb stance time via visual feedback significantly improved the stance time symmetry (p = 0.012) and peak propulsion symmetry (p = 0.036) of individuals walking with both prostheses. With the powered knee prosthesis, the highest feedback target elicited 36% improvement in stance time symmetry, 22% increase in prosthesis-side peak propulsion, and 47% improvement in peak propulsion symmetry compared to a no feedback condition. The changes with feedback were not different with the passive prosthesis, and the main effects of device/ prosthesis type were not statistically different. However, subject by device interactions were significant, indicating individuals did not respond consistently with each device (e.g. prosthesis-side propulsion remained comparable to or was greater with the powered versus passive prosthesis for different subjects). Overall, prosthesis-side peak propulsion averaged across conditions was 31% greater with the powered prosthesis and peak propulsion asymmetry improved by 48% with the powered prosthesis.

CONCLUSIONS

Increasing prosthesis-side stance time via visual feedback favorably improved individuals' temporal and propulsive symmetry. The powered prosthesis commonly enabled greater propulsion, but individuals adapted to each device with varying behavior, requiring further investigation.

Predictive Simulation of Human Walking Augmented by a Powered Ankle Exoskeleton

The human ankle provides significant positive power during the stance phase of walking, which has resulted in studies focusing on methods to reduce the energetic walking cost by augmenting the ankle with exoskeletons. Recently, a few devices have successfully reduced the metabolic cost of walking by replacing part of the biological ankle plantar flexor torque. Despite these achievements, development of assistive ankle devices remains challenging, partly because the current practice of design and control of powered exoskeletons is highly time and effort consuming, which prevents quickly exploring different design and control parameters. Predictive simulations using musculoskeletal models coupled with robotic devices may facilitate the process of design and control of assistive devices. In this study, we simulate human walking augmented by a powered ankle exoskeleton. The walking problem was formulated as a predictive dynamic optimization in which both the optimal assistive device torque and the gait were solved simultaneously. Cases with exoskeletons assisting one ankle and both ankles were considered. The results showed that the energetic cost of walking could be reduced by 45% with one ankle augmented, and by 52% with both ankles augmented. This study contributes towards the goal of providing optimal assistive torque through external devices and theoretical peak reductions that could be expected from such devices.

Dynamic optimization of Gait with a Generalized Lower-Limb Prosthesis Model

Predictive simulation of gait is a promising tool for robotic lower limb prosthesis design, but has been limited in its application to models of existing design types. We propose a modeling approach to find optimal prosthesis dynamics in gait simulations without constraining the prosthesis to follow kinematics allowed by a specific joint mechanism. To accomplish this, we render a transtibial prosthetic device as the composition of its resultant forces and moments as they act upon the prosthetic foot and socket and allow3 degree-of-freedom planar motion. The model is implemented into a human musculoskeletal model and used to solve dynamic optimizations of muscle and prosthesis controls to minimize muscle effort and loading on the residual limb during walking. The emphasis on muscle effort vs. limb loading is varied in the minimization objective and the resulting optimal prosthesis dynamics are compared. We found that muscle effort and socket loading measures were reduced for our prosthesis model compared to a revolute joint prosthesis model. We interpret large displacements in the linear axes to transfer energy to the plantarflexion action before toe-off and reduce loading at the socket-limb interface. Our results suggest this approach could assist in the design of non-biomimetic prostheses but requires experimental validation to assess our modeling assumptions, as well as progress toward increased fidelity of predictive simulation approaches more generally.

Design Optimization in Lower Limb Prostheses: A Review

This paper aims to develop a knowledge base and identify the promising research pathways toward designing lower limb prostheses for optimal biomechanical and clinical outcomes. It is based on the literature search representing the state of the art in the lower limb prosthesis joint design and biomechanical analysis. Current design solutions are organized in terms of fulfilling four key functional roles: body support, propulsion, task flexibility, and loading relief. Biomechanical analyses of these designs reveal that the hypothesized outcomes are not consistently observed. We suggest that these outcomes may be improved by incorporating tools that can predict user performance metrics to optimize the device during the initial design process. We also note that the scope of the solution space of most current designs is limited by focusing on the anthropomorphic design approaches that do not account for the person's altered anatomy post-amputation. The effects of the prosthetic joint behavior on whole-body gait biomechanics and user experience are likewise under-explored. Two research paths to support the goal of better predicting the user outcomes are proposed: experimental parameterization of designs and model-based simulations. However, while work in these areas has introduced promising new possibilities, connecting both to improve real-world performance remains a challenge.

Bilevel Optimization for Cost Function Determination in Dynamic Simulation of Human Gait

Predictive simulation based on dynamic optimization using musculoskeletal models is a powerful approach for studying human gait. Predictive musculoskeletal simulation may be used for a variety of applications from designing assistive devices to testing theories of motor control. However, the underlying cost function for the predictive optimization is unknown and is generally assumed a priori. Alternatively, the underlying cost function can be determined from among a family of possible cost functions, representing an inverse optimal control problem that may be solved using a bilevel optimization approach. In this study, a nested evolutionary approach is proposed to solve the bilevel optimization problem. The lower level optimization is solved by a direct collocation method, and the upper level is solved by a genetic algorithm. We demonstrate our approach to solve different bilevel optimization problems, including finding the weights among three common performance criteria in the cost function for normal human walking. The proposed approach was found to be effective at solving the bilevel optimization problems. This approach should provide practical utility in designing assistive devices to aid mobility, and could yield insights about the control of human walking.

The Challenges and Achievements of Experimental Implementation of an Active Transfemoral Prosthesis Based on Biological Quasi-Stiffness: The CYBERLEGs Beta-Prosthesis

The CYBERLEGs Beta-Prosthesis is an active transfemoral prosthesis that can provide the full torque required for reproducing average level ground walking at both the knee and ankle in the sagittal plane. The prosthesis attempts to produce a natural level ground walking gait that approximates the joint torques and kinematics of a non-amputee while maintaining passively compliant joints, the stiffnesses of which were derived from biological quasi-stiffness measurements. The ankle of the prosthesis consists of a series elastic actuator with a parallel spring and the knee is composed of three different systems that must compliment each other to generate the correct joint behavior: a series elastic actuator, a lockable parallel spring and an energy transfer mechanism. Bench testing of this new prosthesis was completed and demonstrated that the device was able to create the expected torque-angle characteristics for a normal walker under ideal conditions. The experimental trials with four amputees walking on a treadmill to validate the behavior of the prosthesis proved that although the prosthesis could be controlled in a way that allowed all subjects to walk, the accurate timing and kinematic requirements of the output of the device limited the efficacy of using springs with quasi-static stiffnesses. Modification of the control and stiffness of the series springs could provide better performance in future work.