Use of machine learning to assess the predictive value of 3 commonly used clinical measures to quantify outcomes after total shoulder arthroplasty

Accuracy of machine learning to predict the outcomes of shoulder arthroplasty: a systematic review

BACKGROUND

Artificial intelligence (AI) uses computer systems to simulate cognitive capacities to accomplish goals like problem-solving and decision-making. Machine learning (ML), a branch of AI, makes algorithms find connections between preset variables, thereby producing prediction models. ML can aid shoulder surgeons in determining which patients may be susceptible to worse outcomes and complications following shoulder arthroplasty (SA) and align patient expectations following SA. However, limited literature is available on ML utilization in total shoulder arthroplasty (TSA) and reverse TSA.

METHODS

A systematic literature review in accordance with PRISMA guidelines was performed to identify primary research articles evaluating ML's ability to predict SA outcomes. With duplicates removed, the initial query yielded 327 articles, and after applying inclusion and exclusion criteria, 12 articles that had at least 1 month follow-up time were included.

RESULTS

ML can predict 30-day postoperative complications with a 90% accuracy, postoperative range of motion with a higher-than-85% accuracy, and clinical improvement in patient-reported outcome measures above minimal clinically important differences with a 93%-99% accuracy. ML can predict length of stay, operative time, discharge disposition, and hospitalization costs.

CONCLUSION

ML can accurately predict outcomes and complications following SA and healthcare utilization. Outcomes are highly dependent on the type of algorithms used, data input, and features selected for the model.

LEVEL OF EVIDENCE

III.

Evaluating the fairness and accuracy of machine learning-based predictions of clinical outcomes after anatomic and reverse total shoulder arthroplasty

BACKGROUND

Machine learning (ML)-based clinical decision support tools (CDSTs) make personalized predictions for different treatments; by comparing predictions of multiple treatments, these tools can be used to optimize decision making for a particular patient. However, CDST prediction accuracy varies for different patients and also for different treatment options. If these differences are sufficiently large and consistent for a particular subcohort of patients, then that bias may result in those patients not receiving a particular treatment. Such level of bias would deem the CDST "unfair." The purpose of this study is to evaluate the "fairness" of ML CDST-based clinical outcomes predictions after anatomic (aTSA) and reverse total shoulder arthroplasty (rTSA) for patients of different demographic attributes.

METHODS

Clinical data from 8280 shoulder arthroplasty patients with 19,249 postoperative visits was used to evaluate the prediction fairness and accuracy associated with the following patient demographic attributes: ethnicity, sex, and age at the time of surgery. Performance of clinical outcome and range of motion regression predictions were quantified by the mean absolute error (MAE) and performance of minimal clinically important difference (MCID) and substantial clinical benefit classification predictions were quantified by accuracy, sensitivity, and the F1 score. Fairness of classification predictions leveraged the "four-fifths" legal guideline from the US Equal Employment Opportunity Commission and fairness of regression predictions leveraged established MCID thresholds associated with each outcome measure.

RESULTS

For both aTSA and rTSA clinical outcome predictions, only minor differences in MAE were observed between patients of different ethnicity, sex, and age. Evaluation of prediction fairness demonstrated that 0 of 486 MCID (0%) and only 3 of 486 substantial clinical benefit (0.6%) classification predictions were outside the 20% fairness boundary and only 14 of 972 (1.4%) regression predictions were outside of the MCID fairness boundary. Hispanic and Black patients were more likely to have ML predictions out of fairness tolerance for aTSA and rTSA. Additionally, patients <60 years old were more likely to have ML predictions out of fairness tolerance for rTSA. No disparate predictions were identified for sex and no disparate regression predictions were observed for forward elevation, internal rotation score, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form score, or global shoulder function.

CONCLUSION

The ML algorithms analyzed in this study accurately predict clinical outcomes after aTSA and rTSA for patients of different ethnicity, sex, and age, where only 1.4% of regression predictions and only 0.3% of classification predictions were out of fairness tolerance using the proposed fairness evaluation method and acceptance criteria. Future work is required to externally validate these ML algorithms to ensure they are equally accurate for all legally protected patient groups.

Initial clinical experience with a predictive clinical decision support tool for anatomic and reverse total shoulder arthroplasty

PURPOSE

Clinical decision support tools (CDSTs) are software that generate patient-specific assessments that can be used to better inform healthcare provider decision making. Machine learning (ML)-based CDSTs have recently been developed for anatomic (aTSA) and reverse (rTSA) total shoulder arthroplasty to facilitate more data-driven, evidence-based decision making. Using this shoulder CDST as an example, this external validation study provides an overview of how ML-based algorithms are developed and discusses the limitations of these tools.

METHODS

An external validation for a novel CDST was conducted on 243 patients (120F/123M) who received a personalized prediction prior to surgery and had short-term clinical follow-up from 3 months to 2 years after primary aTSA (n = 43) or rTSA (n = 200). The outcome score and active range of motion predictions were compared to each patient's actual result at each timepoint, with the accuracy quantified by the mean absolute error (MAE).

RESULTS

The results of this external validation demonstrate the CDST accuracy to be similar (within 10%) or better than the MAEs from the published internal validation. A few predictive models were observed to have substantially lower MAEs than the internal validation, specifically, Constant (31.6% better), active abduction (22.5% better), global shoulder function (20.0% better), active external rotation (19.0% better), and active forward elevation (16.2% better), which is encouraging; however, the sample size was small.

CONCLUSION

A greater understanding of the limitations of ML-based CDSTs will facilitate more responsible use and build trust and confidence, potentially leading to greater adoption. As CDSTs evolve, we anticipate greater shared decision making between the patient and surgeon with the aim of achieving even better outcomes and greater levels of patient satisfaction.

Impact of Deltoid Computer Tomography Image Data on the Accuracy of Machine Learning Predictions of Clinical Outcomes after Anatomic and Reverse Total Shoulder Arthroplasty

Background: Despite the importance of the deltoid to shoulder biomechanics, very few studies have quantified the three-dimensional shape, size, or quality of the deltoid muscle, and no studies have correlated these measurements to clinical outcomes after anatomic (aTSA) and/or reverse (rTSA) total shoulder arthroplasty in any statistically/scientifically relevant manner. Methods: Preoperative computer tomography (CT) images from 1057 patients (585 female, 469 male; 799 primary rTSA and 258 primary aTSA) of a single platform shoulder arthroplasty prosthesis (Equinoxe; Exactech, Inc., Gainesville, FL) were analyzed in this study. A machine learning (ML) framework was used to segment the deltoid muscle for 1057 patients and quantify 15 different muscle characteristics, including volumetric (size, shape, etc.) and intensity-based Hounsfield (HU) measurements. These deltoid measurements were correlated to postoperative clinical outcomes and utilized as inputs to train/test ML algorithms used to predict postoperative outcomes at multiple postoperative timepoints (1 year, 2-3 years, and 3-5 years) for aTSA and rTSA. Results: Numerous deltoid muscle measurements were demonstrated to significantly vary with age, gender, prosthesis type, and CT image kernel; notably, normalized deltoid volume and deltoid fatty infiltration were demonstrated to be relevant to preoperative and postoperative clinical outcomes after aTSA and rTSA. Incorporating deltoid image data into the ML models improved clinical outcome prediction accuracy relative to ML algorithms without image data, particularly for the prediction of abduction and forward elevation after aTSA and rTSA. Analyzing ML feature importance facilitated rank-ordering of the deltoid image measurements relevant to aTSA and rTSA clinical outcomes. Specifically, we identified that deltoid shape flatness, normalized deltoid volume, deltoid voxel skewness, and deltoid shape sphericity were the most predictive image-based features used to predict clinical outcomes after aTSA and rTSA. Many of these deltoid measurements were found to be more predictive of aTSA and rTSA postoperative outcomes than patient demographic data, comorbidity data, and diagnosis data. Conclusions: While future work is required to further refine the ML models, which include additional shoulder muscles, like the rotator cuff, our results show promise that the developed ML framework can be used to evolve traditional CT-based preoperative planning software into an evidence-based ML clinical decision support tool.

Impact of accumulating risk factors on the incidence of dislocation after primary reverse total shoulder arthroplasty using a medial glenoid-lateral humerus onlay prosthesis

BACKGROUND

The aim of this study was to facilitate preoperative identification of patients at risk for dislocation after reverse total shoulder arthroplasty (rTSA) using the Equinoxe rTSA prosthesis (medialized glenoid, lateralized onlay humerus with a 145° neck-shaft angle) and quantify the impact of accumulating risk factors on the occurrence of dislocation.

METHODS

We retrospectively analyzed 10,023 primary rTSA patients from an international multicenter database of a single platform shoulder prosthesis and quantified the dislocation rate associated with multiple combinations of previously identified risk factors. To adapt our statistical results for prospective identification of patients most at-risk for dislocation, we stratified our data set by multiple risk factor combinations and calculated the odds ratio for each cohort to quantify the impact of accumulating risk factors on dislocation.

RESULTS

Of the 10,023 primary rTSA patients, 136 (52 female, 83 male, 1 unknown) were reported to have a dislocation for a rate of 1.4%. Patients with zero risk factors were rare, where only 12.7% of patients (1268 of 10,023) had no risk factors, and only 0.5% of these (6 of 1268) had a report of dislocation. The dislocation rate increased in patient cohorts with an increasing number of risk factors. Specifically, the dislocation rate increased from 0.9% for a patient cohort with 1 risk factor to 1.0% for 2 risk factors, 1.6% for 3 risk factors, 2.7% for 4 risk factors, 5.3% for 5 risk factors, and 7.3% for 6 risk factors. Stratifying dislocation rate by multiple risk factor combinations identified numerous cohorts with either an elevated risk or a diminished risk for dislocation.

DISCUSSION

This multicenter study of 10,023 rTSA patients demonstrated that 1.4% of the patients experienced dislocation with one specific medialized glenoid-lateralized humerus onlay rTSA prosthesis. Stratifying patients by multiple combinations of risk factors demonstrated the impact of accumulating risk factors on the incidence of dislocation. rTSA patients with the greatest risk of dislocation were those of male sex, age ≤67 years at the time of surgery, patients with body mass index ≥31, patients who received cemented humeral stems, patients who received glenospheres having a diameter >40 mm, and/or patients who received expanded or laterally offset glenospheres. Patients with these risk factors who are considering rTSA using a medial glenoid-lateral humerus should be made aware of their elevated dislocation risk profile.

The Shoulder Arthroplasty Smart Score Correlates Well With Legacy Outcome Scores Without a Ceiling Effect

BACKGROUND

The Shoulder Arthroplasty Smart (SAS) score is a new, validated machine learning-derived outcome measure that requires six input parameters. The SAS score has the potential to replace legacy shoulder outcome scores.

METHODS

We conducted a retrospective review of a multinational shoulder arthroplasty database of one platform shoulder prosthesis (used interchangeably for anatomic and reverse total shoulder arthroplasty). All primary shoulder arthroplasties with a minimum of two-year follow-up and an available SAS score were evaluated. Associations between scoring systems were assessed using Pearson correlations, with 95% confidence intervals stratified by time point (preoperatively and postoperatively at 2- and 5-year follow-ups, respectively) and procedure (anatomic verses reverse total shoulder arthroplasty). Conversion equations were developed using the best-fit line from linear regression analysis. Ceiling effects were assessed based on two definitions: (1) >15% of participants scoring the maximal possible score and (2) a standardized distance less than 1.0, whereby the standardized distance is calculated by subtracting the mean from the maximal score and dividing by the standard deviation.

RESULTS

Two thousand four hundred six shoulders were evaluated at 4,553 clinical encounters. For preoperatively collected data, the SAS score correlated strongly with the Constant (R = 0.83), University of California at Los Angeles (R = 0.85), and Shoulder Pain and Disability Index (R = -0.70) scores and moderately with the American Shoulder and Elbow Surgeons (R = 0.69) and Simple Shoulder Test (R = 0.65) scores. The SAS score strongly correlated (R > 0.7) with all legacy outcome scores collected at 2- and 5-year postoperative visits. Score predictions made using the conversion equations between the SAS score and legacy outcome scores strongly correlated with their actual values. Neither the SAS nor the Constant score were influenced by ceiling effects. All other outcome scores evaluated demonstrated ceiling effects.

CONCLUSION

The SAS score correlates well with legacy shoulder scores after primary shoulder arthroplasty while mitigating ceiling effects. Surgeons may decrease patient questionnaire burden by using the brief six-question SAS score.

Factors associated with improvement or loss of internal rotation after reverse shoulder arthroplasty

BACKGROUND

Internal rotation (IR) with reverse total shoulder arthroplasty (rTSA) can be unpredictable. Identifying the factors associated with loss of or improved IR could aid preoperative patient counseling. This study quantifies the change in IR experienced by rTSA patients with nonfracture indications and identifies the patient, implant, and operative factors associated with IR loss or gain at 2-year minimum follow-up.

METHODS

A total of 1978 primary rTSA patients were analyzed from an international database of a single rTSA prosthesis to quantify IR at 2 years' minimum follow-up. rTSA patients were divided into 2 cohorts based on their preoperative IR score, with group 1 patients having less active IR as defined by a preoperative IR score ≤3 and group 2 patients having greater active IR as defined by a preoperative IR score ≥4 (ie, L5 or higher). For both group 1 and 2 patients, univariate and multivariate analyses were performed to quantify the risk factors associated with IR loss after rTSA.

RESULTS

Overall, 58.9% of rTSA patients experienced IR improvement and 17.0% lost IR after rTSA. The occurrence of IR loss or gain was dependent on preoperative IR score, as 73.2% of group 1 patients improved IR and only 40.1% of group 2 patients improved IR, whereas 31.0% of group 2 patients lost IR and only 6.3% of group 1 patients lost IR after rTSA. Numerous risk factors for IR loss were identified. For group 1 patients, male sex (P = .004, odds ratio [OR] = 2.056), tobacco usage (P = .004, OR = 0.348), larger humeral stem diameter (P = .008, OR = 0.852), and not having subscapularis repaired (P = .002, OR = 2.654) were significant risk factors for IR loss. For group 2 patients, male sex (P = .005, OR = 1.656), higher body mass index (P = .002, OR = 0.946), a diagnosis other than osteoarthritis (P < .001, OR = 2.189), nonaugmented baseplate usage (P < .001, OR = 2.116), and not having subscapularis repaired (P < .001, OR = 3.052) were significant risk factors for IR loss.

CONCLUSION

The majority of patients improve IR after rTSA in the nonfracture setting. rTSA patients with substantial IR prior to surgery had a greater probability for losing IR compared to patients with poor preoperative IR. Numerous risk factors for IR loss were identified; these risk factors are useful for counseling patients considering rTSA, as some patients are more likely to lose IR than others.

Using machine learning to predict internal rotation after anatomic and reverse total shoulder arthroplasty

BACKGROUND

Improvement in internal rotation (IR) after anatomic (aTSA) and reverse (rTSA) total shoulder arthroplasty is difficult to predict, with rTSA patients experiencing greater variability and more limited IR improvements than aTSA patients. The purpose of this study is to quantify and compare the IR score for aTSA and rTSA patients and create supervised machine learning that predicts IR after aTSA and rTSA at multiple postoperative time points.

METHODS

Clinical data from 2270 aTSA and 4198 rTSA patients were analyzed using 3 supervised machine learning techniques to create predictive models for internal rotation as measured by the IR score at 6 postoperative time points. Predictions were performed using the full input feature set and 2 minimal input feature sets. The mean absolute error (MAE) quantified the difference between actual and predicted IR scores for each model at each time point. The predictive accuracy of the XGBoost algorithm was also quantified by its ability to distinguish which patients would achieve clinical improvement greater than the minimal clinically important difference (MCID) and substantial clinical benefit (SCB) patient satisfaction thresholds for IR score at 2-3 years after surgery.

RESULTS

rTSA patients had significantly lower mean IR scores and significantly less mean IR score improvement than aTSA patients at each postoperative time point. Both aTSA and rTSA patients experienced significant improvements in their ability to perform activities of daily living (ADLs); however, aTSA patients were significantly more likely to perform these ADLs. Using a minimal feature set of preoperative inputs, our machine learning algorithms had equivalent accuracy when predicting IR score for both aTSA (0.92-1.18 MAE) and rTSA (1.03-1.25 MAE) from 3 months to >5 years after surgery. Furthermore, these predictive algorithms identified with 90% accuracy for aTSA and 85% accuracy for rTSA which patients will achieve MCID IR score improvement and predicted with 85% accuracy for aTSA patients and 77% accuracy for rTSA which patients will achieve SCB IR score improvement at 2-3 years after surgery.

DISCUSSION

Our machine learning study demonstrates that active internal rotation can be accurately predicted after aTSA and rTSA at multiple postoperative time points using a minimal feature set of preoperative inputs. These predictive algorithms accurately identified which patients will, and will not, achieve clinical improvement in IR score that exceeds the MCID and SCB patient satisfaction thresholds.

Reverse Shoulder Arthroplasty Biomechanics

The reverse total shoulder arthroplasty (rTSA) prosthesis has been demonstrated to be a viable treatment option for a variety of end-stage degenerative conditions of the shoulder. The clinical success of this prosthesis is at least partially due to its unique biomechanical advantages. As taught by Paul Grammont, the medialized center of rotation fixed-fulcrum prosthesis increases the deltoid abductor moment arm lengths and improves deltoid efficiency relative to the native shoulder. All modern reverse shoulder prostheses utilize this medialized center of rotation (CoR) design concept; however, some differences in outcomes and complications have been observed between rTSA prostheses. Such differences in outcomes can at least partially be explained by the impact of glenoid and humeral prosthesis design parameters, surgical technique, implant positioning, patient-specific bone morphology, and usage in humeral and glenoid bone loss situations on reverse shoulder biomechanics. Ultimately, a better understanding of the reverse shoulder biomechanical principles will guide future innovations and further improve clinical outcomes.

Risk factors for complications and revision surgery after anatomic and reverse total shoulder arthroplasty

BACKGROUND

Complications and revisions following anatomic (aTSA) and reverse (rTSA) total shoulder arthroplasty have deleterious effects on patient function and satisfaction. The purpose of this study is to evaluate patient-specific, implant-specific and technique-specific risk factors for intraoperative complications, postoperative complications, and the occurrence of revisions after aTSA and rTSA.

METHODS

A total of 2964 aTSA and 5616 rTSA patients were enrolled in an international database of primary shoulder arthroplasty. Intra- and postoperative complications, as well as revisions, were reported and evaluated. Multivariate analyses were performed to quantify the risk factors associated with complications and revisions.

RESULTS

aTSA patients had a significantly higher complication rate (P = .0026) and a significantly higher revision rate (P < .0001) than rTSA patients, but aTSA patients also had a significantly longer average follow-up (P < .0001) than rTSA patients. No difference (P = .2712) in the intraoperative complication rate was observed between aTSA and rTSA patients. Regarding intraoperative complications, female sex (odds ratio [OR] 2.0, 95% confidence interval [CI] 1.17-3.68) and previous shoulder surgery (OR 2.9, 95% CI 1.73-4.90) were identified as significant risk factors. In regard to postoperative complications, younger age (OR 0.987, 95% CI 0.977-0.996), diagnosis of rheumatoid arthritis (OR 1.76, 95% 1.12-2.65), and previous shoulder surgery (OR 1.42, 95% CI 1.16-1.72) were noted to be risks factors. Finally, in regard to revision surgery, younger age (OR 0.964, 95% CI 0.933-0.998), more glenoid retroversion (OR 1.03, 95% CI 1.001-1.058), larger humeral stem size (OR 1.09, 95% CI 1.01-1.19), larger humeral liner thickness or offset (OR 1.50, 95% CI 1.18-1.96), larger glenosphere diameter (OR 1.16, 95% CI 1.07-1.26), and more intraoperative blood loss (OR 1.002, 95% CI 1.001-1.004) were noted to be risk factors.

CONCLUSIONS

Studying the impact of numerous patient- and implant-specific risk factors and determining their impact on complications and revision shoulder arthroplasty can assist surgeons in counseling patients and guide patient expectations following aTSA or rTSA. Care should be taken in patients with a history of previous shoulder surgery, who are at increased risk of both intra- and postoperative complications.