1
|
Imhauser CW, Baumann AP, (Cheryl) Liu X, Bischoff JE, Verdonschot N, Fregly BJ, Elmasry SS, Abdollahi NN, Hume DR, Rooks NB, Schneider MTY, Zaylor W, Besier TF, Halloran JP, Shelburne KB, Erdemir A. Reproducibility in modeling and simulation of the knee: Academic, industry, and regulatory perspectives. J Orthop Res 2023; 41:2569-2578. [PMID: 37350016 PMCID: PMC11345941 DOI: 10.1002/jor.25652] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 04/23/2023] [Accepted: 05/30/2023] [Indexed: 06/24/2023]
Abstract
Stakeholders in the modeling and simulation (M&S) community organized a workshop at the 2019 Annual Meeting of the Orthopaedic Research Society (ORS) entitled "Reproducibility in Modeling and Simulation of the Knee: Academic, Industry, and Regulatory Perspectives." The goal was to discuss efforts among these stakeholders to address irreproducibility in M&S focusing on the knee joint. An academic representative from a leading orthopedic hospital in the United States described a multi-institutional, open effort funded by the National Institutes of Health to assess model reproducibility in computational knee biomechanics. A regulatory representative from the United States Food and Drug Administration indicated the necessity of standards for reproducibility to increase utility of M&S in the regulatory setting. An industry representative from a major orthopedic implant company emphasized improving reproducibility by addressing indeterminacy in personalized modeling through sensitivity analyses, thereby enhancing preclinical evaluation of joint replacement technology. Thought leaders in the M&S community stressed the importance of data sharing to minimize duplication of efforts. A survey comprised 103 attendees revealed strong support for the workshop and for increasing emphasis on computational modeling at future ORS meetings. Nearly all survey respondents (97%) considered reproducibility to be an important issue. Almost half of respondents (45%) tried and failed to reproduce the work of others. Two-thirds of respondents (67%) declared that individual laboratories are most responsible for ensuring reproducible research whereas 44% thought that journals are most responsible. Thought leaders and survey respondents emphasized that computational models must be reproducible and credible to advance knee M&S.
Collapse
Affiliation(s)
- Carl W. Imhauser
- Department of Biomechanics, Hospital for Special Surgery, New York, NY, USA
| | - Andrew P. Baumann
- U.S. Food and Drug Administration, Center for Devices and Radiological Health, Office of Science and Engineering Laboratories, Division of Applied Mechanics, Silver Spring, MD
| | | | | | - Nico Verdonschot
- Technical Medical Institute at University of Twente, Enschede, The Netherlands
- Orthopaedic Research Lab, Radboud University Medical Centre, Nijmegen, The Netherlands
| | | | - Shady S. Elmasry
- Department of Biomechanics, Hospital for Special Surgery, New York, NY, USA
- Department of Mechanical Design and Production, Faculty of Engineering, Cairo University, Egypt
| | - Neda N. Abdollahi
- Center for Human Machine Systems, Cleveland State University, Cleveland, OH, USA
- Department of Mechanical Engineering, Cleveland State University, Cleveland, OH, USA
- Department of Biomedical Engineering, Lerner Research Institute, Cleveland Clinic, Cleveland, OH, USA
| | - Donald R. Hume
- Department of Mechanical and Materials Engineering, University of Denver, Denver, CO, USA
- Center for Orthopaedic Biomechanics, University of Denver, Denver, CO, USA
| | - Nynke B. Rooks
- Auckland Bioengineering Institute, University of Auckland, Auckland, NZ
| | | | - William Zaylor
- Center for Human Machine Systems, Cleveland State University, Cleveland, OH, USA
- Department of Mechanical Engineering, Cleveland State University, Cleveland, OH, USA
| | - Thor F. Besier
- Auckland Bioengineering Institute, University of Auckland, Auckland, NZ
- Department of Engineering Science, Faculty of Engineering, University of Auckland, Auckland, NZ
| | - Jason P. Halloran
- Applied Sciences Laboratory, Institute for Shock Physics, Washington State University, Spokane, WA, USA
| | - Kevin B. Shelburne
- Department of Mechanical and Materials Engineering, University of Denver, Denver, CO, USA
- Center for Orthopaedic Biomechanics, University of Denver, Denver, CO, USA
| | - Ahmet Erdemir
- Department of Biomedical Engineering, Lerner Research Institute, Cleveland Clinic, Cleveland, OH, USA
- Computational Biomodeling (CoBi) Core, Lerner Research Institute, Cleveland Clinic, USA
| |
Collapse
|
3
|
Abstract
Medical and scientific advances are predicated on new knowledge that is robust and reliable and that serves as a solid foundation on which further advances can be built. In biomedical research, we are in the midst of a revolution with the generation of new data and scientific publications at a previously unprecedented rate. However, unfortunately, there is compelling evidence that the majority of these discoveries will not stand the test of time. To a large extent, this reproducibility crisis in basic and preclinical research may be as a result of failure to adhere to good scientific practice and the desperation to publish or perish. This is a multifaceted, multistakeholder problem. No single party is solely responsible, and no single solution will suffice. Here we review the reproducibility problems in basic and preclinical biomedical research, highlight some of the complexities, and discuss potential solutions that may help improve research quality and reproducibility.
Collapse
Affiliation(s)
- C. Glenn Begley
- From the TetraLogic Pharmaceuticals Corporation, Malvern, PA; and Departments of Medicine and Health Research and Policy, Stanford University School of Medicine, Department of Statistics, Stanford University School of Humanities and Sciences, and Meta-Research Innovation Center at Stanford (METRICS), CA
| | - John P.A. Ioannidis
- From the TetraLogic Pharmaceuticals Corporation, Malvern, PA; and Departments of Medicine and Health Research and Policy, Stanford University School of Medicine, Department of Statistics, Stanford University School of Humanities and Sciences, and Meta-Research Innovation Center at Stanford (METRICS), CA
| |
Collapse
|