Evaluating the fairness and accuracy of machine learning-based predictions of clinical outcomes after anatomic and reverse total shoulder arthroplasty

BACKGROUND

Machine learning (ML)-based clinical decision support tools (CDSTs) make personalized predictions for different treatments; by comparing predictions of multiple treatments, these tools can be used to optimize decision making for a particular patient. However, CDST prediction accuracy varies for different patients and also for different treatment options. If these differences are sufficiently large and consistent for a particular subcohort of patients, then that bias may result in those patients not receiving a particular treatment. Such level of bias would deem the CDST "unfair." The purpose of this study is to evaluate the "fairness" of ML CDST-based clinical outcomes predictions after anatomic (aTSA) and reverse total shoulder arthroplasty (rTSA) for patients of different demographic attributes.

METHODS

Clinical data from 8280 shoulder arthroplasty patients with 19,249 postoperative visits was used to evaluate the prediction fairness and accuracy associated with the following patient demographic attributes: ethnicity, sex, and age at the time of surgery. Performance of clinical outcome and range of motion regression predictions were quantified by the mean absolute error (MAE) and performance of minimal clinically important difference (MCID) and substantial clinical benefit classification predictions were quantified by accuracy, sensitivity, and the F1 score. Fairness of classification predictions leveraged the "four-fifths" legal guideline from the US Equal Employment Opportunity Commission and fairness of regression predictions leveraged established MCID thresholds associated with each outcome measure.

RESULTS

For both aTSA and rTSA clinical outcome predictions, only minor differences in MAE were observed between patients of different ethnicity, sex, and age. Evaluation of prediction fairness demonstrated that 0 of 486 MCID (0%) and only 3 of 486 substantial clinical benefit (0.6%) classification predictions were outside the 20% fairness boundary and only 14 of 972 (1.4%) regression predictions were outside of the MCID fairness boundary. Hispanic and Black patients were more likely to have ML predictions out of fairness tolerance for aTSA and rTSA. Additionally, patients <60 years old were more likely to have ML predictions out of fairness tolerance for rTSA. No disparate predictions were identified for sex and no disparate regression predictions were observed for forward elevation, internal rotation score, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form score, or global shoulder function.

CONCLUSION

The ML algorithms analyzed in this study accurately predict clinical outcomes after aTSA and rTSA for patients of different ethnicity, sex, and age, where only 1.4% of regression predictions and only 0.3% of classification predictions were out of fairness tolerance using the proposed fairness evaluation method and acceptance criteria. Future work is required to externally validate these ML algorithms to ensure they are equally accurate for all legally protected patient groups.

Stratification of the Minimal Clinically Important Difference, Substantial Clinical Benefit, and Patient Acceptable Symptomatic State after Total Shoulder Arthroplasty by Implant Type, Preoperative Diagnosis, and Sex

BACKGROUND

Clinical significance, as opposed to statistical significance, has increasingly been utilized to evaluate outcomes after total shoulder arthroplasty (TSA). The purpose of this study was to identify thresholds of the minimal clinically important difference (MCID), substantial clinical benefit (SCB), and patient acceptable symptomatic state (PASS) for TSA outcome metrics and determine if these thresholds are influenced by prosthesis type (anatomic or reverse TSA), sex, or preoperative diagnosis.

METHODS

A prospectively collected international multicenter database inclusive of 38 surgeons was queried for patients receiving a primary aTSA or rTSA between 2003 and 2021. Prospectively, outcome metrics including ASES, shoulder function score (SFS), SST, UCLA, Constant, VAS Pain, shoulder arthroplasty smart (SAS) score, forward flexion, abduction, external rotation, and internal rotation was recorded preoperatively and at each follow-up. A patient satisfaction question was administered at each follow-up. Anchor-based MCID, SCB, and PASS were calculated as defined previously overall and according to implant type, preoperative diagnosis, and sex. The percentage of patients achieving thresholds was also quantified.

RESULTS

A total of 5,851 total shoulder arthroplasties including aTSA (n=2,236) and rTSA (n=3,615) were included in the study cohort. The following were identified as MCID thresholds for the overall (aTSA + rTSA irrespective of diagnosis or sex) cohort: VAS Pain (-1.5), SFS (1.2), SST (2.1), Constant (7.2), ASES (13.9), UCLA (8.2), SPADI (-21.5), and SAS (7.3), Abduction (13°), Forward elevation (16°), External rotation (4°), Internal rotation score (0.2). SCB thresholds for the overall cohort were: VAS Pain (-3.3), SFS (2.9), SST 3.8), Constant (18.9), ASES (33.1), UCLA (12.3), SPADI (-44.7), and SAS (18.2), Abduction (30°), Forward elevation (31°), External rotation (12°), Internal rotation score (0.9). PASS thresholds for the overall cohort were: VAS Pain (0.8), SFS (7.3), SST (9.2), Constant (64.2), ASES (79.5), UCLA (29.5), SPADI (24.7), and SAS (72.5), Abduction (104°), Forward elevation (130°), External rotation (30°), Internal rotation score (3.2). MCID, SCB, and PASS thresholds varied depending on preoperative diagnosis and sex.

CONCLUSION

MCID, SCB, and PASS thresholds vary depending on implant type, preoperative diagnosis, and sex. A comprehensive understanding of these differences as well as identification of clinically-relevant thresholds for legacy and novel metrics is essential to assist surgeons in evaluating their patient's outcomes, interpreting the literature, and counseling their patients preoperatively regarding expectations for improvement. Given that PASS thresholds are fragile and vary greatly depending on cohort variability, caution should be exercised in conflating them across different studies.

Impact of Deltoid Computer Tomography Image Data on the Accuracy of Machine Learning Predictions of Clinical Outcomes after Anatomic and Reverse Total Shoulder Arthroplasty

Background: Despite the importance of the deltoid to shoulder biomechanics, very few studies have quantified the three-dimensional shape, size, or quality of the deltoid muscle, and no studies have correlated these measurements to clinical outcomes after anatomic (aTSA) and/or reverse (rTSA) total shoulder arthroplasty in any statistically/scientifically relevant manner. Methods: Preoperative computer tomography (CT) images from 1057 patients (585 female, 469 male; 799 primary rTSA and 258 primary aTSA) of a single platform shoulder arthroplasty prosthesis (Equinoxe; Exactech, Inc., Gainesville, FL) were analyzed in this study. A machine learning (ML) framework was used to segment the deltoid muscle for 1057 patients and quantify 15 different muscle characteristics, including volumetric (size, shape, etc.) and intensity-based Hounsfield (HU) measurements. These deltoid measurements were correlated to postoperative clinical outcomes and utilized as inputs to train/test ML algorithms used to predict postoperative outcomes at multiple postoperative timepoints (1 year, 2-3 years, and 3-5 years) for aTSA and rTSA. Results: Numerous deltoid muscle measurements were demonstrated to significantly vary with age, gender, prosthesis type, and CT image kernel; notably, normalized deltoid volume and deltoid fatty infiltration were demonstrated to be relevant to preoperative and postoperative clinical outcomes after aTSA and rTSA. Incorporating deltoid image data into the ML models improved clinical outcome prediction accuracy relative to ML algorithms without image data, particularly for the prediction of abduction and forward elevation after aTSA and rTSA. Analyzing ML feature importance facilitated rank-ordering of the deltoid image measurements relevant to aTSA and rTSA clinical outcomes. Specifically, we identified that deltoid shape flatness, normalized deltoid volume, deltoid voxel skewness, and deltoid shape sphericity were the most predictive image-based features used to predict clinical outcomes after aTSA and rTSA. Many of these deltoid measurements were found to be more predictive of aTSA and rTSA postoperative outcomes than patient demographic data, comorbidity data, and diagnosis data. Conclusions: While future work is required to further refine the ML models, which include additional shoulder muscles, like the rotator cuff, our results show promise that the developed ML framework can be used to evolve traditional CT-based preoperative planning software into an evidence-based ML clinical decision support tool.

Impact of accumulating risk factors on the incidence of dislocation after primary reverse total shoulder arthroplasty using a medial glenoid-lateral humerus onlay prosthesis

BACKGROUND

The aim of this study was to facilitate preoperative identification of patients at risk for dislocation after reverse total shoulder arthroplasty (rTSA) using the Equinoxe rTSA prosthesis (medialized glenoid, lateralized onlay humerus with a 145° neck-shaft angle) and quantify the impact of accumulating risk factors on the occurrence of dislocation.

METHODS

We retrospectively analyzed 10,023 primary rTSA patients from an international multicenter database of a single platform shoulder prosthesis and quantified the dislocation rate associated with multiple combinations of previously identified risk factors. To adapt our statistical results for prospective identification of patients most at-risk for dislocation, we stratified our data set by multiple risk factor combinations and calculated the odds ratio for each cohort to quantify the impact of accumulating risk factors on dislocation.

RESULTS

Of the 10,023 primary rTSA patients, 136 (52 female, 83 male, 1 unknown) were reported to have a dislocation for a rate of 1.4%. Patients with zero risk factors were rare, where only 12.7% of patients (1268 of 10,023) had no risk factors, and only 0.5% of these (6 of 1268) had a report of dislocation. The dislocation rate increased in patient cohorts with an increasing number of risk factors. Specifically, the dislocation rate increased from 0.9% for a patient cohort with 1 risk factor to 1.0% for 2 risk factors, 1.6% for 3 risk factors, 2.7% for 4 risk factors, 5.3% for 5 risk factors, and 7.3% for 6 risk factors. Stratifying dislocation rate by multiple risk factor combinations identified numerous cohorts with either an elevated risk or a diminished risk for dislocation.

DISCUSSION

This multicenter study of 10,023 rTSA patients demonstrated that 1.4% of the patients experienced dislocation with one specific medialized glenoid-lateralized humerus onlay rTSA prosthesis. Stratifying patients by multiple combinations of risk factors demonstrated the impact of accumulating risk factors on the incidence of dislocation. rTSA patients with the greatest risk of dislocation were those of male sex, age ≤67 years at the time of surgery, patients with body mass index ≥31, patients who received cemented humeral stems, patients who received glenospheres having a diameter >40 mm, and/or patients who received expanded or laterally offset glenospheres. Patients with these risk factors who are considering rTSA using a medial glenoid-lateral humerus should be made aware of their elevated dislocation risk profile.

Prognostic value of the Walch classification for patients before and after shoulder arthroplasty performed for osteoarthritis with an intact rotator cuff

BACKGROUND

The Walch classification is commonly used by surgeons when determining the treatment of osteoarthritis (OA). However, its utility in prognosticating patient clinical state before and after TSA remains unproven. We assessed the prognostic value of the modified Walch glenoid classification on preoperative clinical state and postoperative clinical and radiographic outcomes in total shoulder arthroplasty (TSA).

METHODS

A prospectively collected, multicenter database for a single-platform TSA system was queried for patients with rotator cuff-intact OA and minimum 2 year follow-up after anatomic (aTSA) and reverse TSA (rTSA). Differences in patient-reported outcome scores (Simple Shoulder Test, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form, Shoulder Pain and Disability Index, visual analog scale for pain, Shoulder Function score), combined patient-reported and clinical-input scores (Constant, University of California-Los Angeles shoulder score, Shoulder Arthroplasty Smart Score), active range of motion values (forward elevation [FE], abduction, external rotation [ER], internal rotation [IR], and radiographic outcomes (humeral and glenoid radiolucency line rates, scapula notching rate) were stratified and compared by glenoid deformity type per the Walch classification for aTSA and rTSA cohorts. Comparisons were performed to assess the ability of the Walch classification to predict the preoperative, postoperative, and improved state after TSA.

RESULTS

1008 TSAs were analyzed including 576 aTSA and 432 rTSA. Comparison of outcomes between Walch glenoid types resulted in 15 pairwise comparisons of 12 clinical outcome metrics, yielding 180 total Walch glenoid pairwise comparisons for each clinical state (preoperative, postoperative, improvement). Of the 180 possible pairwise Walch glenoid type and metric comparisons studied for aTSA and rTSA cohorts, <6% and <2% significantly differed in aTSA and rTSA cohorts, respectively. Significant differences based on Walch type were seen after adjustment for multiple pairwise comparisons in the aTSA cohort for FE and ER preoperatively, the Constant score postoperatively, and for abduction, FE, ER, Constant score, and SAS score for pre- to postoperative improvement. In the rTSA cohort, significant differences were only seen in abduction and Constant score both postoperatively and for pre- to postoperative improvement. There were no statistically significant differences in humeral lucency rate, glenoid lucency rate (aTSA), scapular notching rate (rTSA), complication rates, or revision rates between Walch glenoid types after TSA.

CONCLUSION

Although useful for describing degenerative changes to the glenohumeral joint, we demonstrate a weak association between preoperative glenoid morphology according to the Walch classification and clinical state when evaluating patients undergoing TSA for rotator cuff-intact OA. Alternative glenoid classification systems or predictive models should be considered to provide more precise prognoses for patients undergoing TSA for rotator cuff-intact OA.

Maximizing range of motion of reverse total shoulder arthroplasty using design optimization techniques

Range of motion (ROM) obtained after reverse shoulder arthroplasty (RSA) is an important factor in patient satisfaction and success of the procedure. The optimum RSA design that maximizes ROM is currently unknown. Furthermore, it is unknown if the envelope of motion the RSA is optimized for (overall versus forward elevation planes only) will determine its design. We hypothesized that these were potentially competing objectives (maximizing ROM in frontal elevation planes would require sacrifice in posterior elevation planes), and as a result the optimized designs would differ. The objective of this study was to use computer models and design optimization techniques to determine RSA configurations optimized for either case and compare them in terms of design and performance. Design parameters included glenoid lateralization, humeral lateralization, neck-shaft angle, and inferior offset of the center of rotation (COR) and two different cup depths. All optimized designs maximized glenoid lateralization and inferior offset of the COR. Designs optimized specifically for greater forward elevation plane ROM, however, had slightly higher neck-shaft angles and greater humeral lateralization. In terms of performance, the optimized designs provided 31% to 39% increases in ROM in comparison to that of a representative commercially-available Grammot-style prosthesis. It was concluded that RSA designs optimized for overall versus forward elevation plane ROMs will differ, but both offer improvement over a representative commercially available design, regardless of which ROM region is considered.

Investigating the Effects of Demographics on Shoulder Morphology and Density Using Statistical Shape and Density Modeling

A better understanding of how the shape and density of the shoulder vary among members of a population can help design more effective population-based orthopedic implants. The main objective of this study was to develop statistical shape models (SSMs) and statistical density models (SDMs) of the shoulder to describe the main modes of variability in the shape and density distributions of shoulder bones within a population in terms of principal components (PCs). These PC scores were analyzed, and significant correlations were observed between the shape and density distributions of the shoulder and demographics of the population, such as sex and age. Our results demonstrated that when the overall body sizes of male and female donors were matched, males still had, on average, larger scapulae and thicker humeral cortical bones. Moreover, we concluded that age has a weak but significant inverse effect on the density within the entire shoulder. Weak and moderate, but significant, correlations were also found between many modes of shape and density variations in the shoulder. Our results suggested that donors with bigger humeri have bigger scapulae and higher bone density of humeri corresponds with higher bone density in the scapulae. Finally, asymmetry, to some extent, was noted in the shape and density distributions of the contralateral bones of the shoulder. These results can be used to help guide the designs of population-based prosthesis components and pre-operative surgical planning.

An analysis of the safety of performing dobutamine stress echocardiography in an ambulatory setting

Dobutamine echocardiography has become widely used in the past decade in the evaluation of patients with suspected coronary artery disease who are unable to undergo exercise treadmill or bicycle testing. The safety of this procedure has been studied in a hospital-based setting. However, no studies thus far have evaluated the safety of this procedure in an office-based setting, remote from a hospital. We performed dobutamine echocardiography on 127 patients in an office-based setting, remote from a hospital. Throughout the course of this study there were no deaths, myocardial infarctions, sustained episodes of ventricular tachycardia, or syncopal episodes associated with dobutamine infusion. The frequency of noncardiac side effects was 29%, the majority of which were nausea, vomiting, and paresthesias. Three patients had nonsustained ventricular tachycardia, none of whom had symptoms. We conclude that dobutamine echocardiography is safe, well tolerated, and useful in an office-based setting.

Redistribution of the hemidesmosome components alpha 6 beta 4 integrin and bullous pemphigoid antigens during epithelial wound healing

As basal cells of stratified squamous epithelia become migratory in response to wounding, they lose their cell-substrate adhesion junctions, the hemidesmosomes. We report here studies to determine the fate of the hemidesmosome components, alpha 6 beta 4 integrin and the bullous pemphigoid antigens (BPAGs), as recognized by bullous pemphigoid autoantisera (BPA), in migrating epithelium. In addition, we report studies to determine whether relative synthesis and amount of alpha 6 beta 4 is altered during migration. Mouse corneas with 1.5- to 2-mm-diameter central epithelial debridements were allowed to heal in vitro or in vivo for 1-18 h. In order to do preembedding immunoelectron microscopic localization of alpha 6 beta 4, sheets of stationary and migrating corneal epithelium were removed from their basal laminae after organ culture. BPA and antibodies to alpha 6 and beta 4 were used for immunofluorescence microscopy on frozen sections of intact corneas healing in vivo 1-18 h. Both alpha 6 and beta 4 were found to redistribute from their clustered location within hemidesmosomes to a more even distribution within the substrate-associated membrane of basal cells of the tip of the leading edge of migrating epithelium. Behind the tip of the leading edge, basal cells bound the integrin antibodies around their entire membrane. BPAGs moved from their location along the basal cell membrane of stationary epithelium to a diffuse location within the cytoplasm of migrating cells at the leading edge of migration. Quantitative immunoprecipitation and immunoblotting of alpha 6 beta 4 as well as beta 1 integrin from stationary and migrating epithelium were done to determine whether the synthesis or total amount of the integrins were altered during migration. The relative syntheses of alpha 6 beta 4 and beta 1 per milligram of protein or per cell do not appear to differ between stationary and migrating epithelium and the total amount of the beta 4 and beta 1 does not change despite increased rates of protein synthesis in migrating epithelium. Taken together, these studies suggest that as hemidesmosomes disassemble, their clustered integrin component distributes more evenly in the basal cell membrane, the components recognized by BPA and associated with intermediate filaments are released from the membrane, and these events occur in the absence of any measurable change in the synthesis or total amount of the alpha 6 beta 4 component.

Alpha 6 beta 4 integrin heterodimer is a component of hemidesmosomes

Antisera that recognize the alpha 6 and beta 4 subunits of integrins were found by immunoelectron microscopy to localize to hemidesmosomes in the basal cells of mouse corneal epithelium. Immunoprecipitation experiments using extracts of metabolically labeled corneal epithelial cells indicate that the primary alpha 6-subunit-containing integrin heterodimer present is alpha 6 beta 4 and not alpha 6 beta 1. Here we extend previous studies to report that by immunofluorescence microscopy the alpha 6 integrin subunit colocalizes with bullous pemphigoid antigen and type VII collagen in newly forming hemidesmosomes in the developing 17-day fetal rabbit eye. Neither the composition of the anchoring filaments, which span the region between the hemidesmosomal plaque and the lamina densa of basement membrane where the globular domain of type VII collagen is located, nor the extracellular ligand of alpha 6 beta 4 is known. Once anchoring filament proteins are identified, it will be of interest to determine whether any bind to alpha 6 beta 4.

In vitro studies of megakaryocytopoiesis in thrombocytotic disorders of man

Increased numbers of bone marrow megakaryocytes and thrombocytosis are frequently observed in patients with myeloproliferative disorders (MPD). Increased marrow megakaryocytes and thrombocytosis are also noted in a variety of inflammatory and neoplastic disease leading to the phenomenon of reactive thrombocytosis (RT). The pathogenesis of this finding remains incompletely understood. Using methodology developed in our laboratory, we investigated the causative role of megakaryocyte colony-stimulating activity (Meg-CSA) in generating this phenomenon. We also examined the cloning efficiency of colony-forming units-megakaryocyte (CFU-M) and their responsiveness to an exogenous source of Meg-CSA in patients with these diseases. The results of our investigations suggest that: (1) increased production of Meg-CSA is not responsible for the megakaryocyte hyperplasia and thrombocytosis noted in these patients; (2) the intrinsic stem cell defect described in MPD appears to affect the CFU-M of these patients as well, resulting in an effective expansion of the CFU-M pool with consequent megakaryocyte hyperplasia and thrombocytosis; (3) the CFU-M of patients with MPD remain responsive to an exogenous source of Meg-CSA, suggesting that this megakaryocyte hyperplasia may not be entirely autonomous of its effects; and (4) the CFU-M pool in RT is normal both in size and responsiveness to Meg-CSA, suggesting that in these disorders, the stimulus leading to megakaryocyte hyperplasia and thrombocytosis is active at the post-CFU-M level of megakaryocyte differentiation.

Acquired amegakaryocytic thrombocytopenic purpura: a syndrome of diverse etiologies

The possible pathogenetic mechanisms responsible for the production of acquired amegakaryocytic thrombocytopenic purpura (AATP) were investigated in a group of patients with this disorder. Absence of megakaryocytes and small platelet glycoprotein-bearing mononuclear cells, as determined by immunochemical staining of patient marrows with an antisera to platelet glycoproteins, suggested that the defect in AATP occurs in an early progenitor cell of the megakaryocytic lineage. Using an in vitro clonal assay system for negakaryocytic progenitor cells or megakaryocyte colony-forming units (CFU-M), the proliferative capacity of AATP marrow cells was then assessed. Bone marrow cells from three of four patients formed virtually no megakaryocyte colonies, suggesting that in these individuals the AATP was due to an intrinsic defect in the CFU-M. Bone marrow cells from an additional patient, however, formed 12% of the normal numbers of colonies, providing evidence for at least partial integrity of the CFU-M compartment in this patient. Serum specimens from all six patients were screened for their capacity to alter in vitro megakaryocyte colony formation. Five of six sera enhanced colony formation in a stepwise fashion, demonstrating appropriately elevated levels of megakaryocyte colony-stimulating activity. The serum of the patient with partial integrity of the CFU-M compartment, however, stimulated colony formation only at low concentrations. At higher concentrations, this patient's serum actually inhibited the number of colonies cloned, suggesting the presence of a humoral inhibitor to CFU-M. Serum samples from all patients were further screened for such humoral inhibitors of megakaryocyte colony formation using a cytotoxicity assay. The patient whose serum was inhibitory to CFU-M at high concentrations was indeed found to have a complement-dependent serum IgG inhibitor that was cytotoxic to allogeneic and autologous marrow CFU-M but did not alter erythroid colony formation. These-studies suggest that AATP can be due to at least two mechanisms: either an intrinsic effect at the level of the CFU-M or a circulating cytotoxic autoantibody directed against the CFU-M.