1
|
Nitschke M, Dorschky E, Leyendecker S, Eskofier BM, Koelewijn AD. Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations. Front Bioeng Biotechnol 2024; 12:1285845. [PMID: 38628437 PMCID: PMC11018991 DOI: 10.3389/fbioe.2024.1285845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 01/18/2024] [Indexed: 04/19/2024] Open
Abstract
Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2 mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables ( ≥ 0.99 ) . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Eva Dorschky
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
- Institute of AI for Health, Helmholtz Zentrum München—German Research Center for Environmental Health, Neuherberg, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
2
|
Gambietz M, Nitschke M, Miehling J, Koelewijn AD. Contributing Components of Metabolic Energy Models to Metabolic Cost Estimations in Gait. IEEE Trans Biomed Eng 2024; 71:1228-1236. [PMID: 37938950 DOI: 10.1109/tbme.2023.3331271] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2023]
Abstract
OBJECTIVE As metabolic cost is a primary factor influencing humans' gait, we want to deepen our understanding of metabolic energy expenditure models. Therefore, this paper identifies the parameters and input variables, such as muscle or joint states, that contribute to accurate metabolic cost estimations. METHODS We explored the parameters of four metabolic energy expenditure models in a Monte Carlo sensitivity analysis. Then, we analysed the model parameters by their calculated sensitivity indices, physiological context, and the resulting metabolic rates during the gait cycle. The parameter combination with the highest accuracy in the Monte Carlo simulations represented a quasi-optimized model. In the second step, we investigated the importance of input parameters and variables by analysing the accuracy of neural networks trained with different input features. RESULTS Power-related parameters were most influential in the sensitivity analysis and the neural network-based feature selection. We observed that the quasi-optimized models produced negative metabolic rates, contradicting muscle physiology. Neural network-based models showed promising abilities but have been unable to match the accuracy of traditional metabolic energy expenditure models. CONCLUSION We showed that power-related metabolic energy expenditure model parameters and inputs are most influential during gait. Furthermore, our results suggest that neural network-based metabolic energy expenditure models are viable. However, bigger datasets are required to achieve better accuracy. SIGNIFICANCE As there is a need for more accurate metabolic energy expenditure models, we explored which musculoskeletal parameters are essential when developing a model to estimate metabolic energy.
Collapse
|
3
|
Mohr M, Federolf P, Heinrich D, Nitschke M, Raschner C, Scharbert J, Koelewijn AD. Author Correction: An 8-week injury prevention exercise program combined with change-of-direction technique training limits movement patterns associated with anterior cruciate ligament injury risk. Sci Rep 2024; 14:5426. [PMID: 38443557 PMCID: PMC10915125 DOI: 10.1038/s41598-024-56031-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/07/2024] Open
Affiliation(s)
- M Mohr
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria.
| | - P Federolf
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - D Heinrich
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - M Nitschke
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - C Raschner
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - J Scharbert
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - A D Koelewijn
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
4
|
Mohr M, Federolf P, Heinrich D, Nitschke M, Raschner C, Scharbert J, Koelewijn AD. An 8-week injury prevention exercise program combined with change-of-direction technique training limits movement patterns associated with anterior cruciate ligament injury risk. Sci Rep 2024; 14:3115. [PMID: 38326644 PMCID: PMC10850483 DOI: 10.1038/s41598-024-53640-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 02/03/2024] [Indexed: 02/09/2024] Open
Abstract
Knee ligament sprains are common during change-of-direction (COD) maneuvers in multidirectional team sports. This study aimed to compare the effects of an 8-week injury prevention exercise program containing COD-specific exercises and a similar program containing linear sprint exercises on injury- and performance-related variables during a 135° COD task. We hypothesized that the COD-specific training would lead to (H1) stronger reductions in biomechanical variables associated with anterior cruciate ligament (ACL) injury risk during COD, i.e. knee abduction moment and angle, hip internal rotation angle and lateral trunk lean, and (H2) more effective improvements in COD performance according to the COD completion time, executed angle, ground contact time, and approach speed. Twenty-two sports science students (40% female) completed biomechanical assessments of COD movement strategies before and after participating in two supervised 25-min training sessions per week over 8 weeks. We observed significant 'training x group' interaction effects in support of H1: the COD-specific training but not the linear sprint training led to reduced peak knee abduction moments (interaction, p = 0.027), initial knee abduction (interaction, p < 0.001), and initial lateral trunk lean angles (interaction, p < 0.001) compared to baseline. Although the COD-specific training resulted in sharper executed angles (interaction, p < 0.001), the sprint-specific training group showed reduced COD completion (interaction, p = 0.037) and ground contact times (interaction, p < 0.001). In conclusion, a combination of generic and COD-specific injury prevention training resulted in COD technique adaptations that can help to avoid ACL injury-prone COD movements but may negatively affect COD speed.
Collapse
Affiliation(s)
- M Mohr
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria.
| | - P Federolf
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - D Heinrich
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - M Nitschke
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - C Raschner
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - J Scharbert
- Department of Sport Science, Universität Innsbruck, Fürstenweg 185, 6020, Innsbruck, Austria
| | - A D Koelewijn
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
5
|
Shanbhag J, Wolf A, Wechsler I, Fleischmann S, Winkler J, Leyendecker S, Eskofier BM, Koelewijn AD, Wartzack S, Miehling J. Methods for integrating postural control into biomechanical human simulations: a systematic review. J Neuroeng Rehabil 2023; 20:111. [PMID: 37605197 PMCID: PMC10440942 DOI: 10.1186/s12984-023-01235-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2022] [Accepted: 08/09/2023] [Indexed: 08/23/2023] Open
Abstract
Understanding of the human body's internal processes to maintain balance is fundamental to simulate postural control behaviour. The body uses multiple sensory systems' information to obtain a reliable estimate about the current body state. This information is used to control the reactive behaviour to maintain balance. To predict a certain motion behaviour with knowledge of the muscle forces, forward dynamic simulations of biomechanical human models can be utilized. We aim to use predictive postural control simulations to give therapy recommendations to patients suffering from postural disorders in the future. It is important to know which types of modelling approaches already exist to apply such predictive forward dynamic simulations. Current literature provides different models that aim to simulate human postural control. We conducted a systematic literature research to identify the different approaches of postural control models. The different approaches are discussed regarding their applied biomechanical models, sensory representation, sensory integration, and control methods in standing and gait simulations. We searched on Scopus, Web of Science and PubMed using a search string, scanned 1253 records, and found 102 studies to be eligible for inclusion. The included studies use different ways for sensory representation and integration, although underlying neural processes still remain unclear. We found that for postural control optimal control methods like linear quadratic regulators and model predictive control methods are used less, when models' level of details is increasing, and nonlinearities become more important. Considering musculoskeletal models, reflex-based and PD controllers are mainly applied and show promising results, as they aim to create human-like motion behaviour considering physiological processes.
Collapse
Affiliation(s)
- Julian Shanbhag
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
| | - Alexander Wolf
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Iris Wechsler
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sophie Fleischmann
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jürgen Winkler
- Department of Molecular Neurology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Bjoern M Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Anne D Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Sandro Wartzack
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Jörg Miehling
- Engineering Design, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
6
|
Nitschke M, Marzilger R, Leyendecker S, Eskofier BM, Koelewijn AD. Change the direction: 3D optimal control simulation by directly tracking marker and ground reaction force data. PeerJ 2023; 11:e14852. [PMID: 36778146 PMCID: PMC9912948 DOI: 10.7717/peerj.14852] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 01/13/2023] [Indexed: 02/10/2023] Open
Abstract
Optimal control simulations of musculoskeletal models can be used to reconstruct motions measured with optical motion capture to estimate joint and muscle kinematics and kinetics. These simulations are mutually and dynamically consistent, in contrast to traditional inverse methods. Commonly, optimal control simulations are generated by tracking generalized coordinates in combination with ground reaction forces. The generalized coordinates are estimated from marker positions using, for example, inverse kinematics. Hence, inaccuracies in the estimated coordinates are tracked in the simulation. We developed an approach to reconstruct arbitrary motions, such as change of direction motions, using optimal control simulations of 3D full-body musculoskeletal models by directly tracking marker and ground reaction force data. For evaluation, we recorded three trials each of straight running, curved running, and a v-cut for 10 participants. We reconstructed the recordings with marker tracking simulations, coordinate tracking simulations, and inverse kinematics and dynamics. First, we analyzed the convergence of the simulations and found that the wall time increased three to four times when using marker tracking compared to coordinate tracking. Then, we compared the marker trajectories, ground reaction forces, pelvis translations, joint angles, and joint moments between the three reconstruction methods. Root mean squared deviations between measured and estimated marker positions were smallest for inverse kinematics (e.g., 7.6 ± 5.1 mm for v-cut). However, measurement noise and soft tissue artifacts are likely also tracked in inverse kinematics, meaning that this approach does not reflect a gold standard. Marker tracking simulations resulted in slightly higher root mean squared marker deviations (e.g., 9.5 ± 6.2 mm for v-cut) than inverse kinematics. In contrast, coordinate tracking resulted in deviations that were nearly twice as high (e.g., 16.8 ± 10.5 mm for v-cut). Joint angles from coordinate tracking followed the estimated joint angles from inverse kinematics more closely than marker tracking (e.g., root mean squared deviation of 1.4 ± 1.8 deg vs. 3.5 ± 4.0 deg for v-cut). However, we did not have a gold standard measurement of the joint angles, so it is unknown if this larger deviation means the solution is less accurate. In conclusion, we showed that optimal control simulations of change of direction running motions can be created by tracking marker and ground reaction force data. Marker tracking considerably improved marker accuracy compared to coordinate tracking. Therefore, we recommend reconstructing movements by directly tracking marker data in the optimal control simulation when precise marker tracking is required.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Robert Marzilger
- Division Positioning and Networks, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS, Nuremberg, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department of Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
7
|
Dorschky E, Camomilla V, Davis J, Federolf P, Reenalda J, Koelewijn AD. Perspective on "in the wild" movement analysis using machine learning. Hum Mov Sci 2023; 87:103042. [PMID: 36493569 DOI: 10.1016/j.humov.2022.103042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 09/01/2022] [Accepted: 11/19/2022] [Indexed: 12/12/2022]
Abstract
Recent advances in wearable sensing and machine learning have created ample opportunities for "in the wild" movement analysis in sports, since the combination of both enables real-time feedback to be provided to athletes and coaches, as well as long-term monitoring of movements. The potential for real-time feedback is useful for performance enhancement or technique analysis, and can be achieved by training efficient models and implementing them on dedicated hardware. Long-term monitoring of movement can be used for injury prevention, among others. Such applications are often enabled by training a machine learned model from large datasets that have been collected using wearable sensors. Therefore, in this perspective paper, we provide an overview of approaches for studies that aim to analyze sports movement "in the wild" using wearable sensors and machine learning. First, we discuss how a measurement protocol can be set up by answering six questions. Then, we discuss the benefits and pitfalls and provide recommendations for effective training of machine learning models from movement data, focusing on data pre-processing, feature calculation, and model selection and tuning. Finally, we highlight two application domains where "in the wild" data recording was combined with machine learning for injury prevention and technique analysis, respectively.
Collapse
Affiliation(s)
- Eva Dorschky
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Valentina Camomilla
- Department of Movement, Human and Health Sciences, University of Rome "Foro Italico", Rome, Italy
| | - Jesse Davis
- Department of Computer Science and Leuven.AI, KU Leuven, Leuven, Belgium
| | - Peter Federolf
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| | - Jasper Reenalda
- Biomedical Signal and Systems group, University of Twente, Enschede, The Netherlands; Roessingh Research and Development, Enschede, The Netherlands
| | - Anne D Koelewijn
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
| |
Collapse
|
8
|
Koelewijn AD, Selinger JC. Predictive Simulations to Replicate Human Gait Adaptations and Energetics With Exoskeletons. IEEE Trans Neural Syst Rehabil Eng 2022; 30:1931-1940. [PMID: 35797329 DOI: 10.1109/tnsre.2022.3189038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Robotic exoskeletons have the potential to restore and enhance human mobility. However, optimally controlling these devices, to work in concert with human users, is challenging. Accurate model simulations of the interaction between exoskeletons and users may expedite the design process and improve control. Here, as a proof of principle, we tested if we could use predictive simulations to replicate human gait adaptations and changes in energy expenditure from an experiment where participants walked with exoskeletons. We recreated a past experimental paradigm, where robotic exoskeletons were used to shift people's energetically optimal step frequency to frequencies higher and lower than normally preferred. To match the experimental controller, we modelled knee-worn exoskeletons that applied resistive torques, either proportional or inversely proportional to step frequency-decreasing or increasing the energy optimal step frequency, respectively. We were able to replicate the experiment, finding higher and lower optimal step frequencies than in natural walking under each respective condition. Our simulated resistive torques and objective landscapes resembled the measured experimental resistive torque and energy landscapes. Individual muscle energetics revealed distinct coordination strategies consistent with each exoskeleton controller condition. Increasing the accuracy of step frequency and energetic predictions was best achieved by increasing the number of virtual participants (varying whole-body anthropometrics), rather than the number of muscle parameter sets (varying muscle anthropometrics). In future, our approach can be used to design controllers in advance of human testing, to help identify reasonable solution spaces or tailor design to individual users.
Collapse
|
9
|
Koelewijn AD, Van Den Bogert AJ. Antagonistic co-contraction can minimize muscular effort in systems with uncertainty. PeerJ 2022; 10:e13085. [PMID: 35415011 PMCID: PMC8995038 DOI: 10.7717/peerj.13085] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 02/17/2022] [Indexed: 01/12/2023] Open
Abstract
Muscular co-contraction of antagonistic muscle pairs is often observed in human movement, but it is considered inefficient and it can currently not be predicted in simulations where muscular effort or metabolic energy are minimized. Here, we investigated the relationship between minimizing effort and muscular co-contraction in systems with random uncertainty to see if muscular co-contraction can minimize effort in such system. We also investigated the effect of time delay in the muscle, by varying the time delay in the neural control as well as the activation time constant. We solved optimal control problems for a one-degree-of-freedom pendulum actuated by two identical antagonistic muscles, using forward shooting, to find controller parameters that minimized muscular effort while the pendulum remained upright in the presence of noise added to the moment at the base of the pendulum. We compared a controller with and without feedforward control. Task precision was defined by bounding the root mean square deviation from the upright position, while different perturbation levels defined task difficulty. We found that effort was minimized when the feedforward control was nonzero, even when feedforward control was not necessary to perform the task, which indicates that co-contraction can minimize effort in systems with uncertainty. We also found that the optimal level of co-contraction increased with time delay, both when the activation time constant was increased and when neural time delay was added. Furthermore, we found that for controllers with a neural time delay, a different trajectory was optimal for a controller with feedforward control than for one without, which indicates that simulation trajectories are dependent on the controller architecture. Future movement predictions should therefore account for uncertainty in dynamics and control, and carefully choose the controller architecture. The ability of models to predict co-contraction from effort or energy minimization has important clinical and sports applications. If co-contraction is undesirable, one should aim to remove the cause of co-contraction rather than the co-contraction itself.
Collapse
Affiliation(s)
- Anne D. Koelewijn
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany,Parker-Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States
| | - Antonie J. Van Den Bogert
- Parker-Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States
| |
Collapse
|
10
|
Koelewijn AD, Audu M, del-Ama AJ, Colucci A, Font-Llagunes JM, Gogeascoechea A, Hnat SK, Makowski N, Moreno JC, Nandor M, Quinn R, Reichenbach M, Reyes RD, Sartori M, Soekadar S, Triolo RJ, Vermehren M, Wenger C, Yavuz US, Fey D, Beckerle P. Adaptation Strategies for Personalized Gait Neuroprosthetics. Front Neurorobot 2021; 15:750519. [PMID: 34975445 PMCID: PMC8716811 DOI: 10.3389/fnbot.2021.750519] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Accepted: 11/18/2021] [Indexed: 11/13/2022] Open
Abstract
Personalization of gait neuroprosthetics is paramount to ensure their efficacy for users, who experience severe limitations in mobility without an assistive device. Our goal is to develop assistive devices that collaborate with and are tailored to their users, while allowing them to use as much of their existing capabilities as possible. Currently, personalization of devices is challenging, and technological advances are required to achieve this goal. Therefore, this paper presents an overview of challenges and research directions regarding an interface with the peripheral nervous system, an interface with the central nervous system, and the requirements of interface computing architectures. The interface should be modular and adaptable, such that it can provide assistance where it is needed. Novel data processing technology should be developed to allow for real-time processing while accounting for signal variations in the human. Personalized biomechanical models and simulation techniques should be developed to predict assisted walking motions and interactions between the user and the device. Furthermore, the advantages of interfacing with both the brain and the spinal cord or the periphery should be further explored. Technological advances of interface computing architecture should focus on learning on the chip to achieve further personalization. Furthermore, energy consumption should be low to allow for longer use of the neuroprosthesis. In-memory processing combined with resistive random access memory is a promising technology for both. This paper discusses the aforementioned aspects to highlight new directions for future research in gait neuroprosthetics.
Collapse
Affiliation(s)
- Anne D. Koelewijn
- Biomechanical Data Analysis and Creation (BIOMAC) Group, Machine Learning and Data Analytics Lab, Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Musa Audu
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Antonio J. del-Ama
- Applied Mathematics, Materials Science and Technology and Electronic Technology Department, Rey Juan Carlos University, Mostoles, Spain
| | - Annalisa Colucci
- Clinical Neurotechnology Lab, Neuroscience Research Center (NWFZ), Department of Psychiatry and Neurosciences, Charité - Universita¨tsmedizin Berlin, Berlin, Germany
| | - Josep M. Font-Llagunes
- Biomechanical Engineering Lab, Department of Mechanical Engineering and Research Centre for Biomedical Engineering, Universitat Politècnica de Catalunya, Barcelona, Spain
- Institut de Recerca Sant Joan de Déu, Esplugues de Llobregat, Spain
| | - Antonio Gogeascoechea
- Department of Biomechanical Engineering, Faculty of Engineering Technology, University of Twente, Enschede, Netherlands
| | - Sandra K. Hnat
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Nathan Makowski
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Physical Medicine and Rehabilitation, MetroHealth Medical Center, Cleveland, OH, United States
| | - Juan C. Moreno
- Neural Rehabilitation Group, Department of Translational Neuroscience, Cajal Institute, CSIC, Madrid, Spain
| | - Mark Nandor
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Mechanical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Roger Quinn
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Mechanical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Marc Reichenbach
- Chair of Computer Engineering, Brandenburg University of Technology Cottbus-Senftenberg, Cottbus, Germany
- Chair for Computer Architecture, Department of Computer Science, Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Ryan-David Reyes
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Massimo Sartori
- Department of Biomechanical Engineering, Faculty of Engineering Technology, University of Twente, Enschede, Netherlands
| | - Surjo Soekadar
- Clinical Neurotechnology Lab, Neuroscience Research Center (NWFZ), Department of Psychiatry and Neurosciences, Charité - Universita¨tsmedizin Berlin, Berlin, Germany
| | - Ronald J. Triolo
- Department of Veterans Affairs, Louis Stokes Clevel and Veterans Affairs Medical Center, Advanced Platform Technology Center, Cleveland, OH, United States
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States
| | - Mareike Vermehren
- Clinical Neurotechnology Lab, Neuroscience Research Center (NWFZ), Department of Psychiatry and Neurosciences, Charité - Universita¨tsmedizin Berlin, Berlin, Germany
| | - Christian Wenger
- IHP-Leibniz Institut Fuer Innovative Mikroelektronik, Frankfurt (Oder), Germany
| | - Utku S. Yavuz
- Biomedical Signals and Systems Group, University of Twente, Enschede, Netherlands
| | - Dietmar Fey
- Chair for Computer Architecture, Department of Computer Science, Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Philipp Beckerle
- Chair of Autonomous Systems and Mechatronics, Department of Electrical Engineering, Artificial Intelligence in Biomedical Engineering, Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
11
|
Koelewijn AD, Ijspeert AJ. Exploring the Contribution of Proprioceptive Reflexes to Balance Control in Perturbed Standing. Front Bioeng Biotechnol 2020; 8:866. [PMID: 32984265 PMCID: PMC7485384 DOI: 10.3389/fbioe.2020.00866] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/29/2020] [Accepted: 06/06/2020] [Indexed: 11/17/2022] Open
Abstract
Humans control balance using different feedback loops involving the vestibular system, the visual system, and proprioception. In this article, we focus on proprioception and explore the contribution of reflexes based on force and length feedback to standing balance. In particular, we address the questions of how much proprioception alone could explain balance control, and whether one modality, force or length feedback, is more important than the other. A sagittal plane neuro-musculoskeletal model was developed with six degrees of freedom and nine muscles in each leg. A controller was designed using proprioceptive reflexes and a dead zone. No feedback control was applied inside the dead zone. Reflexes were active once the center of mass moved outside the dead zone. Controller parameters were found by solving an optimization problem, where effort was minimized while the neuro-musculoskeletal model should remain standing upright on a perturbed platform. The ground was perturbed with random square pulses in the sagittal plane with different amplitudes and durations. The optimization was solved for three controllers: using force and length feedback (base model), using only force feedback, and using only length feedback. Simulations were compared to human data from previous work, where an experiment with the same perturbation signal was performed. The optimized controller yielded a similar posture, since average joint angles were within 5 degrees of the experimental average joint angles. The joint angles of the base model, the length only model, and the force only model correlated weakly (ankle) to moderately with the experimental joint angles. The ankle moment correlated weakly to moderately with the experimental ankle moment, while the hip and knee moment were only weakly correlated, or not at all. The time series of the joint angles showed that the length feedback model was better able to explain the experimental joint angles than the force feedback model. Changes in time delay affected the correlation of the joint angles and joint moments. The objective of effort minimization yielded lower joint moments than in the experiment, suggesting that other objectives are also important in balance control, which cause an increase in effort and thus larger joint moments.
Collapse
Affiliation(s)
- Anne D Koelewijn
- Biorobotics Laboratory, Institute of Bioengineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.,Machine Learning and Data Analytics Lab, Faculty of Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Auke J Ijspeert
- Biorobotics Laboratory, Institute of Bioengineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| |
Collapse
|
12
|
Dorschky E, Nitschke M, Martindale CF, van den Bogert AJ, Koelewijn AD, Eskofier BM. CNN-Based Estimation of Sagittal Plane Walking and Running Biomechanics From Measured and Simulated Inertial Sensor Data. Front Bioeng Biotechnol 2020; 8:604. [PMID: 32671032 PMCID: PMC7333079 DOI: 10.3389/fbioe.2020.00604] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2020] [Accepted: 05/18/2020] [Indexed: 12/12/2022] Open
Abstract
Machine learning is a promising approach to evaluate human movement based on wearable sensor data. A representative dataset for training data-driven models is crucial to ensure that the model generalizes well to unseen data. However, the acquisition of sufficient data is time-consuming and often infeasible. We present a method to create realistic inertial sensor data with corresponding biomechanical variables by 2D walking and running simulations. We augmented a measured inertial sensor dataset with simulated data for the training of convolutional neural networks to estimate sagittal plane joint angles, joint moments, and ground reaction forces (GRFs) of walking and running. When adding simulated data, the root mean square error (RMSE) of the test set of hip, knee, and ankle joint angles decreased up to 17%, 27% and 23%, the RMSE of knee and ankle joint moments up to 6% and the RMSE of anterior-posterior and vertical GRF up to 2 and 6%. Simulation-aided estimation of joint moments and GRFs was limited by inaccuracies of the biomechanical model. Improving the physics-based model and domain adaptation learning may further increase the benefit of simulated data. Future work can exploit biomechanical simulations to connect different data sources in order to create representative datasets of human movement. In conclusion, machine learning can benefit from available domain knowledge on biomechanical simulations to supplement cumbersome data collections.
Collapse
Affiliation(s)
- Eva Dorschky
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Christine F. Martindale
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | | | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
13
|
Koelewijn AD, van den Bogert AJ. A solution method for predictive simulations in a stochastic environment. J Biomech 2020; 104:109759. [PMID: 32312556 DOI: 10.1016/j.jbiomech.2020.109759] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2019] [Revised: 03/13/2020] [Accepted: 03/18/2020] [Indexed: 11/18/2022]
Abstract
Predictive gait simulations currently do not account for environmental or internal noise. We describe a method to solve predictive simulations of human movements in a stochastic environment using a collocation method. The optimization is performed over multiple noisy episodes of the trajectory, instead of a single episode in a deterministic environment. Each episode used the same control parameters. The method was verified on a torque-driven pendulum swing-up problem. A different optimal trajectory was found in a stochastic environment than in the deterministic environment. Next, it was applied to gait to show its application in predictive simulation of human movement. We show that, unlike in a deterministic model, a nonzero minimum foot clearance during swing is predicted by a minimum-effort criterion in a stochastic environment. The predicted amount of foot clearance increased with the noise amplitude.
Collapse
Affiliation(s)
- Anne D Koelewijn
- Department of Mechanical Engineering, Cleveland State University, USA; Biorobotics Laboratory, Institute of Bioengineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland; Machine Learning and Data Analytics Lab, Faculty of Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany.
| | | |
Collapse
|
14
|
Koelewijn AD, Heinrich D, van den Bogert AJ. Metabolic cost calculations of gait using musculoskeletal energy models, a comparison study. PLoS One 2019; 14:e0222037. [PMID: 31532796 PMCID: PMC6750598 DOI: 10.1371/journal.pone.0222037] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2019] [Accepted: 08/20/2019] [Indexed: 11/18/2022] Open
Abstract
This paper compares predictions of metabolic energy expenditure in gait using seven metabolic energy expenditure models to assess their correlation with experimental data. Ground reaction forces, marker data, and pulmonary gas exchange data were recorded for six walking trials at combinations of two speeds, 0.8 m/s and 1.3 m/s, and three inclines, -8% (downhill), level, and 8% (uphill). The metabolic cost, calculated with the metabolic energy models was compared to the metabolic cost from the pulmonary gas exchange rates. A repeated measures correlation showed that all models correlated well with experimental data, with correlations of at least 0.9. The model by Bhargava et al. (J Biomech, 2004: 81-88) and the model by Lichtwark and Wilson (J Exp Biol, 2005: 2831-3843) had the highest correlation, 0.95. The model by Margaria (Int Z Angew Physiol Einschl Arbeitsphysiol, 1968: 339-351) predicted the increase in metabolic cost following a change in dynamics best in absolute terms.
Collapse
Affiliation(s)
- Anne D. Koelewijn
- Parker Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States of America
- Biorobotics Laboratory, Institute of Bioengineering, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
| | - Dieter Heinrich
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| | - Antonie J. van den Bogert
- Parker Hannifin Laboratory for Human Motion and Control, Department of Mechanical Engineering, Cleveland State University, Cleveland, Ohio, United States of America
- * E-mail:
| |
Collapse
|
15
|
Koelewijn AD, Dorschky E, van den Bogert AJ. A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Comput Methods Biomech Biomed Engin 2018; 21:521-531. [PMID: 30027769 DOI: 10.1080/10255842.2018.1490954] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Whether humans minimize metabolic energy in gait is unknown. Gradient-based optimization could be used to predict gait without using walking data but requires a twice differentiable metabolic energy model. Therefore, the metabolic energy model of Umberger et al. ( 2003 ) was adapted to be twice differentiable. Predictive simulations of a reaching task and gait were solved using this continuous model and by minimizing effort. The reaching task simulation showed that energy minimization predicts unrealistic movements when compared to effort minimization. The predictive gait simulations showed that objectives other than metabolic energy are also important in gait.
Collapse
Affiliation(s)
- Anne D Koelewijn
- a Department of Mechanical Engineering , Cleveland State University , 1960 E 24th St , Cleveland , OH , 44115 , USA
| | - Eva Dorschky
- b Department of Computer Science , Friedrich-Alexander-Universität Nürnberg , Erlangen , Germany
| | - Antonie J van den Bogert
- a Department of Mechanical Engineering , Cleveland State University , 1960 E 24th St , Cleveland , OH , 44115 , USA
| |
Collapse
|