1
|
Wechsler I, Wolf A, Shanbhag J, Leyendecker S, Eskofier BM, Koelewijn AD, Wartzack S, Miehling J. Bridging the sim2real gap. Investigating deviations between experimental motion measurements and musculoskeletal simulation results-a systematic review. Front Bioeng Biotechnol 2024; 12:1386874. [PMID: 38919383 PMCID: PMC11196827 DOI: 10.3389/fbioe.2024.1386874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 05/21/2024] [Indexed: 06/27/2024] Open
Abstract
Musculoskeletal simulations can be used to estimate biomechanical variables like muscle forces and joint torques from non-invasive experimental data using inverse and forward methods. Inverse kinematics followed by inverse dynamics (ID) uses body motion and external force measurements to compute joint movements and the corresponding joint loads, respectively. ID leads to residual forces and torques (residuals) that are not physically realistic, because of measurement noise and modeling assumptions. Forward dynamic simulations (FD) are found by tracking experimental data. They do not generate residuals but will move away from experimental data to achieve this. Therefore, there is a gap between reality (the experimental measurements) and simulations in both approaches, the sim2real gap. To answer (patho-) physiological research questions, simulation results have to be accurate and reliable; the sim2real gap needs to be handled. Therefore, we reviewed methods to handle the sim2real gap in such musculoskeletal simulations. The review identifies, classifies and analyses existing methods that bridge the sim2real gap, including their strengths and limitations. Using a systematic approach, we conducted an electronic search in the databases Scopus, PubMed and Web of Science. We selected and included 85 relevant papers that were sorted into eight different solution clusters based on three aspects: how the sim2real gap is handled, the mathematical method used, and the parameters/variables of the simulations which were adjusted. Each cluster has a distinctive way of handling the sim2real gap with accompanying strengths and limitations. Ultimately, the method choice largely depends on various factors: available model, input parameters/variables, investigated movement and of course the underlying research aim. Researchers should be aware that the sim2real gap remains for both ID and FD approaches. However, we conclude that multimodal approaches tracking kinematic and dynamic measurements may be one possible solution to handle the sim2real gap as methods tracking multimodal measurements (some combination of sensor position/orientation or EMG measurements), consistently lead to better tracking performances. Initial analyses show that motion analysis performance can be enhanced by using multimodal measurements as different sensor technologies can compensate each other's weaknesses.
Collapse
Affiliation(s)
- Iris Wechsler
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Alexander Wolf
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Julian Shanbhag
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
- Chair of Autonomous Systems and Mechatronics, Department of Electrical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sandro Wartzack
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jörg Miehling
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
2
|
Nitschke M, Dorschky E, Leyendecker S, Eskofier BM, Koelewijn AD. Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations. Front Bioeng Biotechnol 2024; 12:1285845. [PMID: 38628437 PMCID: PMC11018991 DOI: 10.3389/fbioe.2024.1285845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 01/18/2024] [Indexed: 04/19/2024] Open
Abstract
Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2 mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables ( ≥ 0.99 ) . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Eva Dorschky
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
- Institute of AI for Health, Helmholtz Zentrum München—German Research Center for Environmental Health, Neuherberg, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
3
|
Puchaud P, Michaud B, Begon M. The interplay of fatigue dynamics and task achievement using optimal control predictive simulation. Hum Mov Sci 2024; 94:103182. [PMID: 38401336 DOI: 10.1016/j.humov.2024.103182] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 12/22/2023] [Accepted: 01/26/2024] [Indexed: 02/26/2024]
Abstract
Predictive simulation of human motion could provide insight into optimal techniques. In repetitive or long-duration tasks, these simulations must predict fatigue-induced adaptation. However, most studies minimize cost function terms related to actuator activations, assuming it minimizes fatigue. An additional modeling layer is needed to consider the previous use of muscles to reveal adaptive strategies to the decreased force production capability. Here, we propose interfacing Xia's three-compartment fatigue dynamics model with rigid-body dynamics. A stabilization invariant was added to Xia's model. We simulated the maximum repetition of dumbbell biceps curls as an optimal control problem (OCP) using direct multiple shooting. We explored three cost functions (minimizing torque, fatigue, or both) and two OCP formulations (full-horizon and sliding-horizon approaches). We adapted Xia's model by adding a stabilization invariant coefficients S=105 for direct multiple shooting. Sliding-horizon OCPs achieved 20 to 21 repetitions. The kinematic strategy slowly deviated from a plausible dumbbell lifting task to a swinging strategy as fatigue onset increasingly compromised the humerus to remain vertical. In full-horizon OCPs, the latter kinematic strategy was used over the whole motion, resulting in 32 repetitions. We showed that sliding-horizon OCPs revealed a reactive strategy to fatigue when only torque was included in the cost function, whereas an anticipatory strategy was revealed when the fatigue term was included in the cost function. Overall, the proposed approach has the potential to be a valuable tool in optimizing performance and helping reduce fatigue-related injuries in a variety of fields.
Collapse
Affiliation(s)
- P Puchaud
- Laboratoire de Simulation et Modélisation du Mouvement, Département de Kinésiologie, Université de Montréal, Laval, Canada.
| | - B Michaud
- Laboratoire de Simulation et Modélisation du Mouvement, Département de Kinésiologie, Université de Montréal, Laval, Canada
| | - M Begon
- Laboratoire de Simulation et Modélisation du Mouvement, Département de Kinésiologie, Université de Montréal, Laval, Canada
| |
Collapse
|
4
|
Seethapathi N, Jain AK, Srinivasan M. Walking speeds are lower for short distance and turning locomotion: Experiments and modeling in low-cost prosthesis users. PLoS One 2024; 19:e0295993. [PMID: 38166012 PMCID: PMC10760709 DOI: 10.1371/journal.pone.0295993] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2021] [Accepted: 12/04/2023] [Indexed: 01/04/2024] Open
Abstract
Preferred walking speed is a widely-used performance measure for people with mobility issues, but is usually measured in straight line walking for fixed distances or durations, and without explicitly accounting for turning. However, daily walking involves walking for bouts of different distances and walking with turning, with prior studies showing that short bouts with at most 10 steps could be 40% of all bouts and turning steps could be 8-50% of all steps. Here, we studied walking in a straight line for short distances (4 m to 23 m) and walking in circles (1 m to 3 m turning radii) in people with transtibial amputation or transfemoral amputation using a passive ankle-foot prosthesis (Jaipur Foot). We found that the study participants' preferred walking speeds are lower for shorter straight-line walking distances and lower for circles of smaller radii, which is analogous to earlier results in subjects without amputation. Using inverse optimization, we estimated the cost of changing speeds and turning such that the observed preferred walking speeds in our experiments minimizes the total cost of walking. The inferred costs of changing speeds and turning were larger for subjects with amputation compared to subjects without amputation in a previous study, specifically, being 4x to 8x larger for the turning cost and being highest for subjects with transfemoral amputation. Such high costs inferred by inverse optimization could potentially include non-energetic costs such as due to joint or interfacial stress or stability concerns, as inverse optimization cannot distinguish such terms from true metabolic cost. These experimental findings and models capturing the experimental trends could inform prosthesis design and rehabilitation therapy to better assist changing speeds and turning tasks. Further, measuring the preferred speed for a range of distances and radii could be a more comprehensive subject-specific measure of walking performance than commonly used straight line walking metrics.
Collapse
Affiliation(s)
- Nidhi Seethapathi
- Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States of America
- Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, United States of America
| | - Anil Kumar Jain
- Santokba Durlabhji Memorial Hospital, Jaipur, Rajasthan, India
| | - Manoj Srinivasan
- Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, United States of America
| |
Collapse
|
5
|
He Y, Liang M, Fang Y, Fekete G, Baker JS, Gu Y. Lumbar and pelvis movement comparison between cross-court and long-line topspin forehand in table tennis: based on musculoskeletal model. Front Bioeng Biotechnol 2023; 11:1185177. [PMID: 37404682 PMCID: PMC10315575 DOI: 10.3389/fbioe.2023.1185177] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Accepted: 06/09/2023] [Indexed: 07/06/2023] Open
Abstract
Introduction: Cross-court and the long-line topspin forehand is the common and basic stroke skill in table tennis. The purpose of this study was to investigate the differences in lumbar and pelvis movements between cross-court and long-line topspin forehand strokes in table tennis based on musculoskeletal demands using OpenSim. Materials and Methods: The eight-camera Vicon system and Kistler force platform were used to measure kinematics and kinetics in the lumbar and pelvis movement of sixteen participants (Weight: 69.89 ± 1.58 kg; Height: 1.73 ± 0.03 m; Age: 22.89 ± 2.03 years; BMI: 23.45 ± 0.69 kg/m2; Experience: 8.33 ± 0.71 years) during cross-court and long-line topspin forehand play. The data was imputed into OpenSim providing the establishment of the Giat2392 musculoskeletal model for simulation. One-dimensional statistical parametric mapping and independent samples t-test was performed in MATLAB and SPSS to analyze the kinematics and kinetics. Results: The results show that the range of motion, peak moment, and maximum angle of the lumbar and pelvis movement in cross-court play were significantly higher than in the long-line stroke play. The moment of long-line in the sagittal and frontal plane was significantly higher than cross-court play in the early stroke phase. Conclusion: The lumbar and pelvis embody greater weight transfer and greater energy production mechanisms when players performed cross-court compared to long-line topspin forehand. Beginners could enhance their motor control strategies in forehand topspin skills and master this skill more easily based on the results of this study.
Collapse
Affiliation(s)
- Yuqi He
- Research Academy of Medicine Combining Sports, Ningbo No.2 Hospital, Ningbo, China
- Faculty of Engineering, University of Pannonia, Veszprém, Hungary
- Savaria Institute of Technology, Eötvös Loránd University, Szombathely, Hungary
| | - Minjun Liang
- Research Academy of Medicine Combining Sports, Ningbo No.2 Hospital, Ningbo, China
- Faculty of Sports Science, Ningbo University, Ningbo, China
| | - Yufei Fang
- Research Academy of Medicine Combining Sports, Ningbo No.2 Hospital, Ningbo, China
| | - Gusztáv Fekete
- Savaria Institute of Technology, Eötvös Loránd University, Szombathely, Hungary
| | - Julien S. Baker
- Department of Sport, Physical Education and Health, Hong Kong Baptist University, Hong Kong, China
| | - Yaodong Gu
- Research Academy of Medicine Combining Sports, Ningbo No.2 Hospital, Ningbo, China
- Faculty of Sports Science, Ningbo University, Ningbo, China
| |
Collapse
|
6
|
Heinrich D, van den Bogert AJ, Mössner M, Nachbauer W. Model-based estimation of muscle and ACL forces during turning maneuvers in alpine skiing. Sci Rep 2023; 13:9026. [PMID: 37270655 DOI: 10.1038/s41598-023-35775-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 05/23/2023] [Indexed: 06/05/2023] Open
Abstract
In alpine skiing, estimation of the muscle forces and joint loads such as the forces in the ACL of the knee are essential to quantify the loading pattern of the skier during turning maneuvers. Since direct measurement of these forces is generally not feasible, non-invasive methods based on musculoskeletal modeling should be considered. In alpine skiing, however, muscle forces and ACL forces have not been analyzed during turning maneuvers due to the lack of three dimensional musculoskeletal models. In the present study, a three dimensional musculoskeletal skier model was successfully applied to track experimental data of a professional skier. During the turning maneuver, the primary activated muscles groups of the outside leg, bearing the highest loads, were the gluteus maximus, vastus lateralis as well as the medial and lateral hamstrings. The main function of these muscles was to generate the required hip extension and knee extension moments. The gluteus maximus was also the main contributor to the hip abduction moment when the hip was highly flexed. Furthermore, the lateral hamstrings and gluteus maximus contributed to the hip external rotation moment in addition to the quadratus femoris. Peak ACL forces reached 211 N on the outside leg with the main contribution in the frontal plane due to an external knee abduction moment. Sagittal plane contributions were low due to consistently high knee flexion (> 60[Formula: see text]), substantial co-activation of the hamstrings and the ground reaction force pushing the anteriorly inclined tibia backwards with respect to the femur. In conclusion, the present musculoskeletal simulation model provides a detailed insight into the loading of a skier during turning maneuvers that might be used to analyze appropriate training loads or injury risk factors such as the speed or turn radius of the skier, changes of the equipment or neuromuscular control parameters.
Collapse
Affiliation(s)
- Dieter Heinrich
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria.
| | | | - Martin Mössner
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| | - Werner Nachbauer
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| |
Collapse
|
7
|
Wang H, Basu A, Durandau G, Sartori M. A Wearable Real-time Kinematic and Kinetic Measurement Sensor Setup for Human Locomotion. WEARABLE TECHNOLOGIES 2023; 4:e11. [PMID: 37091825 PMCID: PMC7614461 DOI: 10.1017/wtc.2023.7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/25/2023]
Abstract
Current laboratory-based setups (optical marker cameras + force plates) for human motion measurement require participants to stay in a constrained capture region which forbids rich movement types. This study established a fully wearable system, based on commercially available sensors (inertial measurement units + pressure insoles) that can measure both kinematic and kinetic motion data simultaneously and support wireless frame-by-frame streaming. In addition, its capability and accuracy were tested against a conventional laboratory-based setup. An experiment was conducted, with 9 participants wearing the wearable measurement system and performing 13 daily motion activities, from slow walking to fast running, together with vertical jump, squat, lunge and single-leg landing, inside the capture space of the laboratory-based motion capture system. The recorded sensor data were post-processed to obtain joint angles, ground reaction forces (GRFs), and joint torques (via multi-body inverse dynamics). Compared to the laboratory-based system, the established wearable measurement system can measure accurate information of all lower limb joint angles (Pearson's r = 0.929), vertical GRFs (Pearson's r = 0.954), and ankle joint torques (Pearson's r = 0.917). Center of pressure (CoP) in the anterior-posterior direction and knee joint torques were fairly matched (Pearson's r = 0.683 and 0.612, respectively). Calculated hip joint torques and measured medial-lateral CoP did not match with the laboratory-based system (Pearson's r = 0.21 and 0.47, respectively). Furthermore, both raw and processed datasets are openly accessible (https://doi.org/10.5281/zenodo.6457662). Documentation, data processing codes, and guidelines to establish the real-time wearable kinetic measurement system are also shared (https://github.com/HuaweiWang/WearableMeasurementSystem).
Collapse
Affiliation(s)
- Huawei Wang
- Department of Biomechanical Engineering, University of Twente, Enschede, the Netherlands
| | - Akash Basu
- Department of Biomechanical Engineering, University of Twente, Enschede, the Netherlands
| | - Guillaume Durandau
- Department of Mechanical Engineering, McGill University, Montreal, Canada
| | - Massimo Sartori
- Department of Biomechanical Engineering, University of Twente, Enschede, the Netherlands
| |
Collapse
|
8
|
Zhao Y, Zhang M, Wu H, He X, Todoh M. Neuromechanics-Based Neural Feedback Controller for Planar Arm Reaching Movements. Bioengineering (Basel) 2023; 10:bioengineering10040436. [PMID: 37106623 PMCID: PMC10136284 DOI: 10.3390/bioengineering10040436] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 03/22/2023] [Accepted: 03/24/2023] [Indexed: 04/29/2023] Open
Abstract
Based on the principles of neuromechanics, human arm movements result from the dynamic interaction between the nervous, muscular, and skeletal systems. To develop an effective neural feedback controller for neuro-rehabilitation training, it is important to consider both the effects of muscles and skeletons. In this study, we designed a neuromechanics-based neural feedback controller for arm reaching movements. To achieve this, we first constructed a musculoskeletal arm model based on the actual biomechanical structure of the human arm. Subsequently, a hybrid neural feedback controller was developed that mimics the multifunctional areas of the human arm. The performance of this controller was then validated through numerical simulation experiments. The simulation results demonstrated a bell-shaped movement trajectory, consistent with the natural motion of human arm movements. Furthermore, the experiment testing the tracking ability of the controller revealed real-time errors within one millimeter, with the tensile force generated by the controller's muscles being stable and maintained at a low value, thereby avoiding the issue of muscle strain that can occur due to excessive excitation during the neurorehabilitation process.
Collapse
Affiliation(s)
- Yongkun Zhao
- Division of Human Mechanical Systems and Design, Graduate School of Engineering, Hokkaido University, Sapporo 060-8628, Japan
- Division of Bioengineering, Graduate School of Engineering Science, Osaka University, Osaka 560-8531, Japan
| | - Mingquan Zhang
- State Key Laboratory of Bioelectronics, Jiangsu Provincial Key Laboratory of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
| | - Haijun Wu
- Division of Mechanical and Aerospace Engineering, Faculty of Engineering, Hokkaido University, Sapporo 060-8628, Japan
| | - Xiangkun He
- Department of Bioengineering, Faculty of Engineering, Imperial College London, London SW7 2AZ, UK
| | - Masahiro Todoh
- Division of Mechanical and Aerospace Engineering, Faculty of Engineering, Hokkaido University, Sapporo 060-8628, Japan
| |
Collapse
|
9
|
Nitschke M, Marzilger R, Leyendecker S, Eskofier BM, Koelewijn AD. Change the direction: 3D optimal control simulation by directly tracking marker and ground reaction force data. PeerJ 2023; 11:e14852. [PMID: 36778146 PMCID: PMC9912948 DOI: 10.7717/peerj.14852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 01/13/2023] [Indexed: 02/10/2023] Open
Abstract
Optimal control simulations of musculoskeletal models can be used to reconstruct motions measured with optical motion capture to estimate joint and muscle kinematics and kinetics. These simulations are mutually and dynamically consistent, in contrast to traditional inverse methods. Commonly, optimal control simulations are generated by tracking generalized coordinates in combination with ground reaction forces. The generalized coordinates are estimated from marker positions using, for example, inverse kinematics. Hence, inaccuracies in the estimated coordinates are tracked in the simulation. We developed an approach to reconstruct arbitrary motions, such as change of direction motions, using optimal control simulations of 3D full-body musculoskeletal models by directly tracking marker and ground reaction force data. For evaluation, we recorded three trials each of straight running, curved running, and a v-cut for 10 participants. We reconstructed the recordings with marker tracking simulations, coordinate tracking simulations, and inverse kinematics and dynamics. First, we analyzed the convergence of the simulations and found that the wall time increased three to four times when using marker tracking compared to coordinate tracking. Then, we compared the marker trajectories, ground reaction forces, pelvis translations, joint angles, and joint moments between the three reconstruction methods. Root mean squared deviations between measured and estimated marker positions were smallest for inverse kinematics (e.g., 7.6 ± 5.1 mm for v-cut). However, measurement noise and soft tissue artifacts are likely also tracked in inverse kinematics, meaning that this approach does not reflect a gold standard. Marker tracking simulations resulted in slightly higher root mean squared marker deviations (e.g., 9.5 ± 6.2 mm for v-cut) than inverse kinematics. In contrast, coordinate tracking resulted in deviations that were nearly twice as high (e.g., 16.8 ± 10.5 mm for v-cut). Joint angles from coordinate tracking followed the estimated joint angles from inverse kinematics more closely than marker tracking (e.g., root mean squared deviation of 1.4 ± 1.8 deg vs. 3.5 ± 4.0 deg for v-cut). However, we did not have a gold standard measurement of the joint angles, so it is unknown if this larger deviation means the solution is less accurate. In conclusion, we showed that optimal control simulations of change of direction running motions can be created by tracking marker and ground reaction force data. Marker tracking considerably improved marker accuracy compared to coordinate tracking. Therefore, we recommend reconstructing movements by directly tracking marker data in the optimal control simulation when precise marker tracking is required.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Robert Marzilger
- Division Positioning and Networks, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, Nuremberg, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
10
|
Venne A, Bailly F, Charbonneau E, Dowling-Medley J, Begon M. Optimal estimation of complex aerial movements using dynamic optimisation. Sports Biomech 2023; 22:300-315. [PMID: 35670189 DOI: 10.1080/14763141.2022.2066015] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
Abstract
When estimating full-body motion from experimental data, inverse kinematics followed by inverse dynamics does not guarantee dynamical consistency of the resulting motion, especially in movements where the trajectory depends heavily on the initial state, such as in free-fall. Our objective was to estimate dynamically consistent joint kinematics and kinetics of complex aerial movements. A 42-degrees-of-freedom model with 95 markers was personalised for five elite trampoline athletes performing various backward and forward twisting somersaults. Using dynamic optimisation, our algorithm estimated joint angles, velocities and torques by tracking the recorded marker positions. Kinematics, kinetics, angular and linear momenta, and marker tracking difference were compared to results of an Extended Kalman Filter (EKF) followed by inverse dynamics. Angular momentum and horizontal linear momentum were conserved throughout the estimated motion, as per free-fall dynamics. Marker tracking difference went from 17 ± 4 mm for the EKF to 36 ± 11 mm with dynamic optimisation tracking the experimental markers, and to 49 ± 9 mm with dynamic optimisation tracking EKF joint angles. Joint angles from the dynamic optimisations were similar to those of the EKF, and joint torques were smoother. This approach satisfies the dynamics of complex aerial rigid-body movements while remaining close to the experimental 3D marker dataset.
Collapse
Affiliation(s)
- André Venne
- Laboratoire de Simulation et Modélisation du Mouvement, Université de Montréal, QC, Canada
| | - François Bailly
- National Institute for Research in Computer Science and Automation, CaminTeam, Montpellier, France
| | - Eve Charbonneau
- Laboratoire de Simulation et Modélisation du Mouvement, Université de Montréal, QC, Canada
| | | | - Mickaël Begon
- Laboratoire de Simulation et Modélisation du Mouvement, Université de Montréal, QC, Canada
| |
Collapse
|
11
|
Denizdurduran B, Markram H, Gewaltig MO. Optimum trajectory learning in musculoskeletal systems with model predictive control and deep reinforcement learning. BIOLOGICAL CYBERNETICS 2022; 116:711-726. [PMID: 35951117 PMCID: PMC9691497 DOI: 10.1007/s00422-022-00940-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 07/04/2022] [Indexed: 05/13/2023]
Abstract
From the computational point of view, musculoskeletal control is the problem of controlling high degrees of freedom and dynamic multi-body system that is driven by redundant muscle units. A critical challenge in the control perspective of skeletal joints with antagonistic muscle pairs is finding methods robust to address this ill-posed nonlinear problem. To address this computational problem, we implemented a twofold optimization and learning framework to be specialized in addressing the redundancies in the muscle control . In the first part, we used model predictive control to obtain energy efficient skeletal trajectories to mimick human movements. The second part is to use deep reinforcement learning to obtain a sequence of stimulus to be given to muscles in order to obtain the skeletal trajectories with muscle control. We observed that the desired stimulus to muscles is only efficiently constructed by integrating the state and control input in a closed-loop setting as it resembles the proprioceptive integration in the spinal cord circuits. In this work, we showed how a variety of different reference trajectories can be obtained with optimal control and how these reference trajectories are mapped to the musculoskeletal control with deep reinforcement learning. Starting from the characteristics of human arm movement to obstacle avoidance experiment, our simulation results confirm the capabilities of our optimization and learning framework for a variety of dynamic movement trajectories. In summary, the proposed framework is offering a pipeline to complement the lack of experiments to record human motion-capture data as well as study the activation range of muscles to replicate the specific trajectory of interest. Using the trajectories from optimal control as a reference signal for reinforcement learning implementation has allowed us to acquire optimum and human-like behaviour of the musculoskeletal system which provides a framework to study human movement in-silico experiments. The present framework can also allow studying upper-arm rehabilitation with assistive robots given that one can use healthy subject movement recordings as reference to work on the control architecture of assistive robotics in order to compensate behavioural deficiencies. Hence, the framework opens to possibility of replicating or complementing labour-intensive, time-consuming and costly experiments with human subjects in the field of movement studies and digital twin of rehabilitation.
Collapse
Affiliation(s)
- Berat Denizdurduran
- Alpine Intuition Sarl, Route de Crochy 20, 1024 Ecublens, Switzerland
- Blue Brain Project, École polytechnique fédérale de Lausanne (EPFL), Campus Biotech, 1202 Geneva, Switzerland
| | - Henry Markram
- Blue Brain Project, École polytechnique fédérale de Lausanne (EPFL), Campus Biotech, 1202 Geneva, Switzerland
| | - Marc-Oliver Gewaltig
- Blue Brain Project, École polytechnique fédérale de Lausanne (EPFL), Campus Biotech, 1202 Geneva, Switzerland
| |
Collapse
|
12
|
Johnson RT, Bianco NA, Finley JM. Patterns of asymmetry and energy cost generated from predictive simulations of hemiparetic gait. PLoS Comput Biol 2022; 18:e1010466. [PMID: 36084139 PMCID: PMC9491609 DOI: 10.1371/journal.pcbi.1010466] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2022] [Revised: 09/21/2022] [Accepted: 08/03/2022] [Indexed: 11/18/2022] Open
Abstract
Hemiparesis, defined as unilateral muscle weakness, often occurs in people post-stroke or people with cerebral palsy, however it is difficult to understand how this hemiparesis affects movement patterns as it often presents alongside a variety of other neuromuscular impairments. Predictive musculoskeletal modeling presents an opportunity to investigate how impairments affect gait performance assuming a particular cost function. Here, we use predictive simulation to quantify the spatiotemporal asymmetries and changes to metabolic cost that emerge when muscle strength is unilaterally reduced and how reducing spatiotemporal symmetry affects metabolic cost. We modified a 2-D musculoskeletal model by uniformly reducing the peak isometric muscle force unilaterally. We then solved optimal control simulations of walking across a range of speeds by minimizing the sum of the cubed muscle excitations. Lastly, we ran additional optimizations to test if reducing spatiotemporal asymmetry would result in an increase in metabolic cost. Our results showed that the magnitude and direction of effort-optimal spatiotemporal asymmetries depends on both the gait speed and level of weakness. Also, the optimal speed was 1.25 m/s for the symmetrical and 20% weakness models but slower (1.00 m/s) for the 40% and 60% weakness models, suggesting that hemiparesis can account for a portion of the slower gait speed seen in people with hemiparesis. Modifying the cost function to minimize spatiotemporal asymmetry resulted in small increases (~4%) in metabolic cost. Overall, our results indicate that spatiotemporal asymmetry may be optimal for people with hemiparesis. Additionally, the effect of speed and the level of weakness on spatiotemporal asymmetry may help explain the well-known heterogenous distribution of spatiotemporal asymmetries observed in the clinic. Future work could extend our results by testing the effects of other neuromuscular impairments on optimal gait strategies, and therefore build a more comprehensive understanding of the gait patterns observed in clinical populations.
Collapse
Affiliation(s)
- Russell T. Johnson
- Division of Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, California, United States of America
- * E-mail:
| | - Nicholas A. Bianco
- Department of Mechanical Engineering, Stanford University, Palo Alto, California, United States of America
| | - James M. Finley
- Division of Biokinesiology and Physical Therapy, University of Southern California, Los Angeles, California, United States of America
- Department of Biomedical Engineering, University of Southern California, Los Angeles, California, United States of America
- Neuroscience Graduate Program, University of Southern California, Los Angeles, California, United States of America
| |
Collapse
|
13
|
Koelewijn AD, Selinger JC. Predictive Simulations to Replicate Human Gait Adaptations and Energetics With Exoskeletons. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1931-1940. [PMID: 35797329 DOI: 10.1109/tnsre.2022.3189038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Robotic exoskeletons have the potential to restore and enhance human mobility. However, optimally controlling these devices, to work in concert with human users, is challenging. Accurate model simulations of the interaction between exoskeletons and users may expedite the design process and improve control. Here, as a proof of principle, we tested if we could use predictive simulations to replicate human gait adaptations and changes in energy expenditure from an experiment where participants walked with exoskeletons. We recreated a past experimental paradigm, where robotic exoskeletons were used to shift people's energetically optimal step frequency to frequencies higher and lower than normally preferred. To match the experimental controller, we modelled knee-worn exoskeletons that applied resistive torques, either proportional or inversely proportional to step frequency-decreasing or increasing the energy optimal step frequency, respectively. We were able to replicate the experiment, finding higher and lower optimal step frequencies than in natural walking under each respective condition. Our simulated resistive torques and objective landscapes resembled the measured experimental resistive torque and energy landscapes. Individual muscle energetics revealed distinct coordination strategies consistent with each exoskeleton controller condition. Increasing the accuracy of step frequency and energetic predictions was best achieved by increasing the number of virtual participants (varying whole-body anthropometrics), rather than the number of muscle parameter sets (varying muscle anthropometrics). In future, our approach can be used to design controllers in advance of human testing, to help identify reasonable solution spaces or tailor design to individual users.
Collapse
|
14
|
Heinrich D, Van den Bogert AJ, Nachbauer W. Estimation of Joint Moments During Turning Maneuvers in Alpine Skiing Using a Three Dimensional Musculoskeletal Skier Model and a Forward Dynamics Optimization Framework. Front Bioeng Biotechnol 2022; 10:894568. [PMID: 35814020 PMCID: PMC9269104 DOI: 10.3389/fbioe.2022.894568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 06/02/2022] [Indexed: 11/13/2022] Open
Abstract
In alpine skiing, estimation of the joint moments acting onto the skier is essential to quantify the loading of the skier during turning maneuvers. In the present study, a novel forward dynamics optimization framework is presented to estimate the joint moments acting onto the skier incorporating a three dimensional musculoskeletal model (53 kinematic degrees of freedom, 94 muscles). Kinematic data of a professional skier performing a turning maneuver were captured and used as input data to the optimization framework. In the optimization framework, the musculoskeletal model of the skier was applied to track the experimental data of a skier and to estimate the underlying joint moments of the skier at the hip, knee and ankle joints of the outside and inside leg as well as the lumbar joint. During the turning maneuver the speed of the skier was about 14 m/s with a minimum turn radius of about 16 m. The highest joint moments were observed at the lumbar joint with a maximum of 1.88 Nm/kg for lumbar extension. At the outside leg, the highest joint moments corresponded to the hip extension moment with 1.27 Nm/kg, the knee extension moment with 1.02 Nm/kg and the ankle plantarflexion moment with 0.85 Nm/kg. Compared to the classical inverse dynamics analysis, the present framework has four major advantages. First, using a forward dynamic optimization framework the underlying kinematics of the skier as well as the corresponding ground reaction forces are dynamically consistent. Second, the present framework can cope with incomplete data (i.e., without ground reaction force data). Third, the computation of the joint moments is less sensitive to errors in the measurement data. Fourth, the computed joint moments are constrained to stay within the physiological limits defined by the musculoskeletal model.
Collapse
Affiliation(s)
- Dieter Heinrich
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
- *Correspondence: Dieter Heinrich,
| | | | - Werner Nachbauer
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| |
Collapse
|
15
|
Chan YS, Teo YX, Gouwanda D, Nurzaman SG, Gopalai AA, Thannirmalai S. Musculoskeletal modelling and simulation of oil palm fresh fruit bunch harvesting. Sci Rep 2022; 12:8010. [PMID: 35568759 PMCID: PMC9107475 DOI: 10.1038/s41598-022-12088-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 05/03/2022] [Indexed: 11/09/2022] Open
Abstract
Oil palm harvesting is a labor-intensive activity and yet it was rarely investigated. Studies showed that complementing human motion analysis with musculoskeletal modelling and simulation can provide valuable information about the dynamics of the joints and muscles. Therefore, this study aims to be the first to create and evaluate an upper extremity musculoskeletal model of the oil palm harvesting motion and to assess the associated Musculoskeletal Disorder (MSD) risk. Tests were conducted at a Malaysia oil palm plantation. Six Inertial Measurement Units (IMU) and Surface Electromyography (sEMG) were used to collect kinematics of the back, shoulder and elbow joints and to measure the muscle activations of longissimus, multifidus, biceps and triceps. The simulated joint angles and muscle activations were validated against the commercial motion capture tool and sEMG, respectively. The muscle forces, joint moments and activations of rectus abdominis, iliocostalis, external oblique, internal oblique and latissimus dorsi were investigated. Findings showed that the longissimus, iliocostalis and rectus abdominis were the primary muscles relied on during harvesting. The harvesters were exposed to a higher risk of MSD while performing back flexion and back rotation. These findings provide insights into the dynamical behavior of the upper extremity muscles and joints that can potentially be used to derive ways to improve the ergonomics of oil palm harvesting, minimize the MSD risk and to design and develop assistive engineering and technological devices or tools for this activity.
Collapse
Affiliation(s)
- Yon Sin Chan
- School of Engineering, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
| | - Yu Xuan Teo
- School of Engineering, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
| | - Darwin Gouwanda
- School of Engineering, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia.
- Monash Industry Palm Oil Research Platform, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia.
| | - Surya Girinatha Nurzaman
- School of Engineering, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
- Monash Industry Palm Oil Research Platform, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
| | - Alpha Agape Gopalai
- School of Engineering, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
- Monash Industry Palm Oil Research Platform, Monash University Malaysia, Bandar Sunway, 47500, Selangor, Malaysia
| | | |
Collapse
|
16
|
Koelewijn AD, Van Den Bogert AJ. Antagonistic co-contraction can minimize muscular effort in systems with uncertainty. PeerJ 2022; 10:e13085. [PMID: 35415011 PMCID: PMC8995038 DOI: 10.7717/peerj.13085] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 02/17/2022] [Indexed: 01/12/2023] Open
Abstract
Muscular co-contraction of antagonistic muscle pairs is often observed in human movement, but it is considered inefficient and it can currently not be predicted in simulations where muscular effort or metabolic energy are minimized. Here, we investigated the relationship between minimizing effort and muscular co-contraction in systems with random uncertainty to see if muscular co-contraction can minimize effort in such system. We also investigated the effect of time delay in the muscle, by varying the time delay in the neural control as well as the activation time constant. We solved optimal control problems for a one-degree-of-freedom pendulum actuated by two identical antagonistic muscles, using forward shooting, to find controller parameters that minimized muscular effort while the pendulum remained upright in the presence of noise added to the moment at the base of the pendulum. We compared a controller with and without feedforward control. Task precision was defined by bounding the root mean square deviation from the upright position, while different perturbation levels defined task difficulty. We found that effort was minimized when the feedforward control was nonzero, even when feedforward control was not necessary to perform the task, which indicates that co-contraction can minimize effort in systems with uncertainty. We also found that the optimal level of co-contraction increased with time delay, both when the activation time constant was increased and when neural time delay was added. Furthermore, we found that for controllers with a neural time delay, a different trajectory was optimal for a controller with feedforward control than for one without, which indicates that simulation trajectories are dependent on the controller architecture. Future movement predictions should therefore account for uncertainty in dynamics and control, and carefully choose the controller architecture. The ability of models to predict co-contraction from effort or energy minimization has important clinical and sports applications. If co-contraction is undesirable, one should aim to remove the cause of co-contraction rather than the co-contraction itself.
Collapse
Affiliation(s)
- Anne D. Koelewijn
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany
- Parker-Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States
| | - Antonie J. Van Den Bogert
- Parker-Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States
| |
Collapse
|