1
|
Nitschke M, Dorschky E, Leyendecker S, Eskofier BM, Koelewijn AD. Estimating 3D kinematics and kinetics from virtual inertial sensor data through musculoskeletal movement simulations. Front Bioeng Biotechnol 2024; 12:1285845. [PMID: 38628437 PMCID: PMC11018991 DOI: 10.3389/fbioe.2024.1285845] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 01/18/2024] [Indexed: 04/19/2024] Open
Abstract
Portable measurement systems using inertial sensors enable motion capture outside the lab, facilitating longitudinal and large-scale studies in natural environments. However, estimating 3D kinematics and kinetics from inertial data for a comprehensive biomechanical movement analysis is still challenging. Machine learning models or stepwise approaches performing Kalman filtering, inverse kinematics, and inverse dynamics can lead to inconsistencies between kinematics and kinetics. We investigated the reconstruction of 3D kinematics and kinetics of arbitrary running motions from inertial sensor data using optimal control simulations of full-body musculoskeletal models. To evaluate the feasibility of the proposed method, we used marker tracking simulations created from optical motion capture data as a reference and for computing virtual inertial data such that the desired solution was known exactly. We generated the inertial tracking simulations by formulating optimal control problems that tracked virtual acceleration and angular velocity while minimizing effort without requiring a task constraint or an initial state. To evaluate the proposed approach, we reconstructed three trials each of straight running, curved running, and a v-cut of 10 participants. We compared the estimated inertial signals and biomechanical variables of the marker and inertial tracking simulations. The inertial data was tracked closely, resulting in low mean root mean squared deviations for pelvis translation (≤20.2 mm), angles (≤1.8 deg), ground reaction forces (≤1.1 BW%), joint moments (≤0.1 BWBH%), and muscle forces (≤5.4 BW%) and high mean coefficients of multiple correlation for all biomechanical variables ( ≥ 0.99 ) . Accordingly, our results showed that optimal control simulations tracking 3D inertial data could reconstruct the kinematics and kinetics of individual trials of all running motions. The simulations led to mutually and dynamically consistent kinematics and kinetics, which allows researching causal chains, for example, to analyze anterior cruciate ligament injury prevention. Our work proved the feasibility of the approach using virtual inertial data. When using the approach in the future with measured data, the sensor location and alignment on the segment must be estimated, and soft-tissue artifacts are potential error sources. Nevertheless, we demonstrated that optimal control simulation tracking inertial data is highly promising for estimating 3D kinematics and kinetics for a comprehensive biomechanical analysis.
Collapse
Affiliation(s)
- Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Eva Dorschky
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Sigrid Leyendecker
- Institute of Applied Dynamics, Department of Mechanical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
- Institute of AI for Health, Helmholtz Zentrum München—German Research Center for Environmental Health, Neuherberg, Germany
| | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
2
|
Seifer AK, Dorschky E, Küderle A, Moradi H, Hannemann R, Eskofier BM. EarGait: Estimation of Temporal Gait Parameters from Hearing Aid Integrated Inertial Sensors. Sensors (Basel) 2023; 23:6565. [PMID: 37514858 PMCID: PMC10383770 DOI: 10.3390/s23146565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 07/14/2023] [Accepted: 07/18/2023] [Indexed: 07/30/2023]
Abstract
Wearable sensors are able to monitor physical health in a home environment and detect changes in gait patterns over time. To ensure long-term user engagement, wearable sensors need to be seamlessly integrated into the user's daily life, such as hearing aids or earbuds. Therefore, we present EarGait, an open-source Python toolbox for gait analysis using inertial sensors integrated into hearing aids. This work contributes a validation for gait event detection algorithms and the estimation of temporal parameters using ear-worn sensors. We perform a comparative analysis of two algorithms based on acceleration data and propose a modified version of one of the algorithms. We conducted a study with healthy young and elderly participants to record walking data using the hearing aid's integrated sensors and an optical motion capture system as a reference. All algorithms were able to detect gait events (initial and terminal contacts), and the improved algorithm performed best, detecting 99.8% of initial contacts and obtaining a mean stride time error of 12 ± 32 ms. The existing algorithms faced challenges in determining the laterality of gait events. To address this limitation, we propose modifications that enhance the determination of the step laterality (ipsi- or contralateral), resulting in a 50% reduction in stride time error. Moreover, the improved version is shown to be robust to different study populations and sampling frequencies but is sensitive to walking speed. This work establishes a solid foundation for a comprehensive gait analysis system integrated into hearing aids that will facilitate continuous and long-term home monitoring.
Collapse
Affiliation(s)
- Ann-Kristin Seifer
- Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91052 Erlangen, Germany
| | - Eva Dorschky
- Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91052 Erlangen, Germany
| | - Arne Küderle
- Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91052 Erlangen, Germany
| | - Hamid Moradi
- Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91052 Erlangen, Germany
| | | | - Björn M Eskofier
- Machine Learning and Data Analytics Lab (MaD Lab), Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 91052 Erlangen, Germany
| |
Collapse
|
3
|
Dorschky E, Camomilla V, Davis J, Federolf P, Reenalda J, Koelewijn AD. Perspective on "in the wild" movement analysis using machine learning. Hum Mov Sci 2023; 87:103042. [PMID: 36493569 DOI: 10.1016/j.humov.2022.103042] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2022] [Revised: 09/01/2022] [Accepted: 11/19/2022] [Indexed: 12/12/2022]
Abstract
Recent advances in wearable sensing and machine learning have created ample opportunities for "in the wild" movement analysis in sports, since the combination of both enables real-time feedback to be provided to athletes and coaches, as well as long-term monitoring of movements. The potential for real-time feedback is useful for performance enhancement or technique analysis, and can be achieved by training efficient models and implementing them on dedicated hardware. Long-term monitoring of movement can be used for injury prevention, among others. Such applications are often enabled by training a machine learned model from large datasets that have been collected using wearable sensors. Therefore, in this perspective paper, we provide an overview of approaches for studies that aim to analyze sports movement "in the wild" using wearable sensors and machine learning. First, we discuss how a measurement protocol can be set up by answering six questions. Then, we discuss the benefits and pitfalls and provide recommendations for effective training of machine learning models from movement data, focusing on data pre-processing, feature calculation, and model selection and tuning. Finally, we highlight two application domains where "in the wild" data recording was combined with machine learning for injury prevention and technique analysis, respectively.
Collapse
Affiliation(s)
- Eva Dorschky
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Valentina Camomilla
- Department of Movement, Human and Health Sciences, University of Rome "Foro Italico", Rome, Italy
| | - Jesse Davis
- Department of Computer Science and Leuven.AI, KU Leuven, Leuven, Belgium
| | - Peter Federolf
- Department of Sport Science, University of Innsbruck, Innsbruck, Austria
| | - Jasper Reenalda
- Biomedical Signal and Systems group, University of Twente, Enschede, The Netherlands; Roessingh Research and Development, Enschede, The Netherlands
| | - Anne D Koelewijn
- Machine Learning and Data Analytics (MaD) Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
| |
Collapse
|
4
|
Nitschke M, Luckfiel T, Schlarb H, Dorschky E, Koelewijn A. Prediction of the effect of midsole stiffness and energy return using trajectory optimisation. Footwear Science 2021. [DOI: 10.1080/19424280.2021.1917684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Marlies Nitschke
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | | | | | - Eva Dorschky
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| | - Anne Koelewijn
- Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
| |
Collapse
|
5
|
Dorschky E, Nitschke M, Martindale CF, van den Bogert AJ, Koelewijn AD, Eskofier BM. CNN-Based Estimation of Sagittal Plane Walking and Running Biomechanics From Measured and Simulated Inertial Sensor Data. Front Bioeng Biotechnol 2020; 8:604. [PMID: 32671032 PMCID: PMC7333079 DOI: 10.3389/fbioe.2020.00604] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2020] [Accepted: 05/18/2020] [Indexed: 12/12/2022] Open
Abstract
Machine learning is a promising approach to evaluate human movement based on wearable sensor data. A representative dataset for training data-driven models is crucial to ensure that the model generalizes well to unseen data. However, the acquisition of sufficient data is time-consuming and often infeasible. We present a method to create realistic inertial sensor data with corresponding biomechanical variables by 2D walking and running simulations. We augmented a measured inertial sensor dataset with simulated data for the training of convolutional neural networks to estimate sagittal plane joint angles, joint moments, and ground reaction forces (GRFs) of walking and running. When adding simulated data, the root mean square error (RMSE) of the test set of hip, knee, and ankle joint angles decreased up to 17%, 27% and 23%, the RMSE of knee and ankle joint moments up to 6% and the RMSE of anterior-posterior and vertical GRF up to 2 and 6%. Simulation-aided estimation of joint moments and GRFs was limited by inaccuracies of the biomechanical model. Improving the physics-based model and domain adaptation learning may further increase the benefit of simulated data. Future work can exploit biomechanical simulations to connect different data sources in order to create representative datasets of human movement. In conclusion, machine learning can benefit from available domain knowledge on biomechanical simulations to supplement cumbersome data collections.
Collapse
Affiliation(s)
- Eva Dorschky
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Marlies Nitschke
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Christine F. Martindale
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | | | - Anne D. Koelewijn
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| | - Bjoern M. Eskofier
- Machine Learning and Data Analytics Lab, Department of Computer Science, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Erlangen, Germany
| |
Collapse
|
6
|
Dorschky E, Nitschke M, Seifer AK, van den Bogert AJ, Eskofier BM. Estimation of gait kinematics and kinetics from inertial sensor data using optimal control of musculoskeletal models. J Biomech 2019; 95:109278. [DOI: 10.1016/j.jbiomech.2019.07.022] [Citation(s) in RCA: 48] [Impact Index Per Article: 9.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2019] [Revised: 07/16/2019] [Accepted: 07/18/2019] [Indexed: 11/24/2022]
|
7
|
Dorschky E, Krüger D, Kurfess N, Schlarb H, Wartzack S, Eskofier BM, van den Bogert AJ. Optimal control simulation predicts effects of midsole materials on energy cost of running. Comput Methods Biomech Biomed Engin 2019; 22:869-879. [PMID: 30987457 DOI: 10.1080/10255842.2019.1601179] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
Abstract
Testing sports equipment with athletes is costly, time-consuming, hazardous and sometimes impracticable. We propose a method for virtual testing of running shoes and predict how midsoles made of BOOSTTM affect energy cost of running. We contribute a visco-elastic contact model and identified model parameters based on load-displacement measurements. We propose a virtual study using optimal control simulation of musculoskeletal models. The predicted reduction in energy cost of ∼1% for BOOSTTM in comparison to conventional materials is consistent with experimental studies. This indicates that the proposed method is capable of replacing experimental studies in the future.
Collapse
Affiliation(s)
- Eva Dorschky
- a Machine Learning and Data Analytics Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) , Erlangen , Germany
| | - Daniel Krüger
- b Engineering Design , Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) , Erlangen , Germany
| | - Nicolai Kurfess
- a Machine Learning and Data Analytics Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) , Erlangen , Germany
| | | | - Sandro Wartzack
- b Engineering Design , Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) , Erlangen , Germany
| | - Bjoern M Eskofier
- a Machine Learning and Data Analytics Lab , Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) , Erlangen , Germany
| | | |
Collapse
|
8
|
Koelewijn AD, Dorschky E, van den Bogert AJ. A metabolic energy expenditure model with a continuous first derivative and its application to predictive simulations of gait. Comput Methods Biomech Biomed Engin 2018; 21:521-531. [PMID: 30027769 DOI: 10.1080/10255842.2018.1490954] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Whether humans minimize metabolic energy in gait is unknown. Gradient-based optimization could be used to predict gait without using walking data but requires a twice differentiable metabolic energy model. Therefore, the metabolic energy model of Umberger et al. ( 2003 ) was adapted to be twice differentiable. Predictive simulations of a reaching task and gait were solved using this continuous model and by minimizing effort. The reaching task simulation showed that energy minimization predicts unrealistic movements when compared to effort minimization. The predictive gait simulations showed that objectives other than metabolic energy are also important in gait.
Collapse
Affiliation(s)
- Anne D Koelewijn
- a Department of Mechanical Engineering , Cleveland State University , 1960 E 24th St , Cleveland , OH , 44115 , USA
| | - Eva Dorschky
- b Department of Computer Science , Friedrich-Alexander-Universität Nürnberg , Erlangen , Germany
| | - Antonie J van den Bogert
- a Department of Mechanical Engineering , Cleveland State University , 1960 E 24th St , Cleveland , OH , 44115 , USA
| |
Collapse
|
9
|
Reymann MP, Dorschky E, Groh BH, Martindale C, Blank P, Eskofier BM. Blood glucose level prediction based on support vector regression using mobile platforms. Annu Int Conf IEEE Eng Med Biol Soc 2017; 2016:2990-2993. [PMID: 28268941 DOI: 10.1109/embc.2016.7591358] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
The correct treatment of diabetes is vital to a patient's health: Staying within defined blood glucose levels prevents dangerous short- and long-term effects on the body. Mobile devices informing patients about their future blood glucose levels could enable them to take counter-measures to prevent hypo or hyper periods. Previous work addressed this challenge by predicting the blood glucose levels using regression models. However, these approaches required a physiological model, representing the human body's response to insulin and glucose intake, or are not directly applicable to mobile platforms (smart phones, tablets). In this paper, we propose an algorithm for mobile platforms to predict blood glucose levels without the need for a physiological model. Using an online software simulator program, we trained a Support Vector Regression (SVR) model and exported the parameter settings to our mobile platform. The prediction accuracy of our mobile platform was evaluated with pre-recorded data of a type 1 diabetes patient. The blood glucose level was predicted with an error of 19 % compared to the true value. Considering the permitted error of commercially used devices of 15 %, our algorithm is the basis for further development of mobile prediction algorithms.
Collapse
|