Improving prediction model systematic review methodology: Letter to the Editor

Including Modifiable and Nonmodifiable Factors Improves Injury Risk Assessment in Professional Baseball Pitchers

OBJECTIVES

To (1) evaluate an injury risk model that included modifiable and nonmodifiable factors into an arm injury risk prediction model in Minor League Baseball (MiLB) pitchers and (2) compare model performance separately for predicting the incidence of elbow and shoulder injuries.

DESIGN

Prospective cohort.

METHODS

A 10-year MiLB injury risk study was conducted. Pitchers were evaluated during preseason, and pitches and arm injuries were documented prospectively. Nonmodifiable variables included arm injury history, professional experience, arm dominance, year, and humeral torsion. Modifiable variables included BMI, pitch count, total range of motion, and horizontal adduction. We compared modifiable, nonmodifiable, and combined model performance by R2, calibration (best = 1.00), and discrimination (area under the curve [AUC]; higher number is better). Sensitivity analysis included only arm injuries sustained in the first 90 days.

RESULTS

In this study, 407 MiLB pitchers (141 arm injuries) were included. Arm injury incidence was 0.27 injuries per 1000 pitches. The arm injury model (calibration 1.05 [0.81-1.30]; AUC: 0.74 [0.69-0.80]) had improved performance compared to only using modifiable predictors (calibration: 0.91 [0.68-1.14]; AUC: 0.67 [0.62-0.73]) and only shoulder range of motion (calibration: 0.52 [0.29, 0.75]; AUC: 0.52 [0.46, 58]). Elbow injury model demonstrated improved performance (calibration: 1.03 [0.76-1.33]; AUC: 0.76 [0.69-0.83]) compared to the shoulder injury model (calibration: 0.46 [0.22-0.69]; AUC: 0.62 [95% CI: 0.55, 0.69]). The sensitivity analysis demonstrated improved model performance compared to the arm injury model.

CONCLUSION

Arm injury risk is influenced by modifiable and nonmodifiable risk factors. The most accurate way to identify professional pitchers who are at risk for arm injury is to use a model that includes modifiable and nonmodifiable risk factors. J Orthop Sports Phys Ther 2022;52(9):630-640. Epub: 9 July 2022. doi:10.2519/jospt.2022.11072.

Just How Confident Can We Be in Predicting Sports Injuries? A Systematic Review of the Methodological Conduct and Performance of Existing Musculoskeletal Injury Prediction Models in Sport

BACKGROUND

An increasing number of musculoskeletal injury prediction models are being developed and implemented in sports medicine. Prediction model quality needs to be evaluated so clinicians can be informed of their potential usefulness.

OBJECTIVE

To evaluate the methodological conduct and completeness of reporting of musculoskeletal injury prediction models in sport.

METHODS

A systematic review was performed from inception to June 2021. Studies were included if they: (1) predicted sport injury; (2) used regression, machine learning, or deep learning models; (3) were written in English; (4) were peer reviewed.

RESULTS

Thirty studies (204 models) were included; 60% of studies utilized only regression methods, 13% only machine learning, and 27% both regression and machine learning approaches. All studies developed a prediction model and no studies externally validated a prediction model. Two percent of models (7% of studies) were low risk of bias and 98% of models (93% of studies) were high or unclear risk of bias. Three studies (10%) performed an a priori sample size calculation; 14 (47%) performed internal validation. Nineteen studies (63%) reported discrimination and two (7%) reported calibration. Four studies (13%) reported model equations for statistical predictions and no machine learning studies reported code or hyperparameters.

CONCLUSION

Existing sport musculoskeletal injury prediction models were poorly developed and have a high risk of bias. No models could be recommended for use in practice. The majority of models were developed with small sample sizes, had inadequate assessment of model performance, and were poorly reported. To create clinically useful sports musculoskeletal injury prediction models, considerable improvements in methodology and reporting are urgently required.