Sociodemographic Factors Are Associated with Patient-Reported Outcome Measure Completion in Orthopaedic Surgery: An Analysis of Completion Rates and Determinants Among New Patients

Artificial Intelligence for Clinically Meaningful Outcome Prediction in Orthopedic Research: Current Applications and Limitations

PURPOSE OF REVIEW

Patient-reported outcome measures (PROM) play a critical role in evaluating the success of treatment interventions for musculoskeletal conditions. However, predicting which patients will benefit from treatment interventions is complex and influenced by a multitude of factors. Artificial intelligence (AI) may better anticipate the propensity to achieve clinically meaningful outcomes through leveraging complex predictive analytics that allow for personalized medicine. This article provides a contemporary review of current applications of AI developed to predict clinically significant outcome (CSO) achievement after musculoskeletal treatment interventions.

RECENT FINDINGS

The highest volume of literature exists in the subspecialties of total joint arthroplasty, spine, and sports medicine, with only three studies identified in the remaining orthopedic subspecialties combined. Performance is widely variable across models, with most studies only reporting discrimination as a performance metric. Given the complexity inherent in predictive modeling for this task, including data availability, data handling, model architecture, and outcome selection, studies vary widely in their methodology and results. Importantly, the majority of studies have not been externally validated or demonstrate important methodological limitations, precluding their implementation into clinical settings. A substantial body of literature has accumulated demonstrating variable internal validity, limited scope, and low potential for clinical deployment. The majority of studies attempt to predict the MCID-the lowest bar of clinical achievement. Though a small proportion of models demonstrate promise and highlight the utility of AI, important methodological limitations need to be addressed moving forward to leverage AI-based applications for clinical deployment.

Differences in Baseline Physical Function and Mental Health PROM Scores in Patients With Foot and Ankle Conditions

BACKGROUND

This study sought to establish normative values for baseline physical and mental health by foot and ankle diagnosis using validated PROMIS scores and to compare the correlation between these 2 outcomes across common diagnoses. Additionally, it investigated the effects associated with chronic vs acute conditions and specific diagnoses on mental health.

METHODS

We reviewed baseline PROMIS Physical Function 10a (PF10a) and PROMIS Global-Mental (PGM) scores of 14,245 patients with one of the 10 most common foot and ankle diagnoses seen at our institution between 2016 and 2021. Pearson correlation coefficients were calculated to assess the relationship between PF10a and PGM by diagnosis. A multivariable regression model including age, sex, language, race, ethnicity, education level, income, and Charlson Comorbidity Index was used to determine the associated effect of diagnosis on PGM score.

RESULTS

On unadjusted analysis, patients diagnosed with an ankle fracture had the lowest mean physical function, whereas patients with hallux valgus had the highest (PF10a = 33.9 vs 46.7, P < .001). Patients with foot/ankle osteoarthritis had the lowest mean self-reported mental health, whereas patients with hallux rigidus had the highest (PGM = 49.9 vs 53.4, P < .001). PF10a and PGM scores were significantly positively correlated for all diagnoses; the correlation was strongest in patients diagnosed with foot/ankle osteoarthritis or hammertoes (r = 0.511) and weakest in patients with ankle fractures (r = 0.232) or sprains (r = 0.280). Chronic conditions, including hammertoes (β = -5.1, 95% CI [-5.8, -4.3], P < .001), foot/ankle osteoarthritis (β = -5.0, 95% CI [-5.7, -4.3], P < .001), and hallux valgus (β = -4.8, 95% CI [-5.5, -4.1], P < .001) were associated with the largest negative effects on patients' mental health.

CONCLUSION

Self-reported physical function and mental health varied across common foot and ankle diagnoses and were more tightly correlated in chronic conditions. The associations between diagnosis and mental health scores appear larger for more chronic diagnoses, including those that are generally associated with relatively unimpaired physical function.

Ten-year results from the UK National Ligament Registry: Patient characteristics and factors predicting nonresponders for completion of outcome scores

PURPOSE

The aim of this study was to report the demographic and mechanism of injury data in the UK National Ligament Registry (NLR) at 10 years and determine factors leading to poor compliance with completion of Patient-Reported Outcome Scores (PROMs).

METHODS

A retrospective review was performed for prospectively collected data on the NLR between January 2013 and December 2022. All patients who underwent primary anterior cruciate ligament reconstruction (ACLR) were included. Patient demographics, mechanism of injury and patient compliance with completing PROMs were analysed. Patient characteristics were further analysed in relation to compliance with completing the different PROMs at the predefined time points. Patients were identified as nonresponders if they had not completed either 1- or 2-year postoperative Knee injury and Osteoarthritis Outcome Score (KOOS).

RESULTS

A total of 17,492 patients were included in this study. The average age for patients undergoing ACLR between 2013 and 2022 was 29.4 (SD, 10.3). Seventy percent were men and 30% women. Football was the most common activity associated with an ACL injury. Patient compliance with recording PROMs was 55% preoperatively and 37%, 32% and 24% at 1-, 2- and 5-year postoperative follow-up, respectively. Nonresponders represented 54% of eligible patients. Multivariate analyses showed that sex, age, smoking, time interval between injury and surgery and low socioeconomic status were associated with low compliance with postoperative PROM completion (p < 0.001).

CONCLUSION

This study reports the demographic characteristics for patients on the NLR since 2013. Male sex, young age, increased waiting time between injury and surgery, smoking and lower socioeconomic class were predictors of low compliance with completion of postoperative PROMs on the UK NLR. Understanding the factors that affect patient compliance with PROMs improves our ability to provide targeted interventions and information to specific patient populations with the aim of enhancing inclusiveness and representation of population in the registry.

LEVEL OF EVIDENCE

Level III.

Non-White Race and Concomitant Orthopedic Conditions Are Risk Factors for Failure to Achieve Clinically Relevant Improvement After Total Knee Arthroplasty

BACKGROUND

Data from the American Joint Replacement Registry demonstrate that 1-year minimal clinically important difference (MCID) achievement rates after total knee arthroplasty (TKA) are substantially lower when using general patient reported outcome measures, such as Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS-PF), than joint specific measures. The purpose of this study was to evaluate patient characteristics and outcomes associated with MCID achievement after TKA using the PROMIS-PF measure.

METHODS

A retrospective review of 263 patients undergoing TKA with preoperative and 1-year postoperative PROMIS-PF scores from March 12, 2020 to February 8, 2022 was performed. Three multivariate models were built to evaluate predictors of MCID achievement. Preoperative predictors evaluated included demographics, comorbidities, history of spine and knee surgery, and baseline PROMIS-PF. Postoperative clinical outcomes evaluated included lengths of stay, discharge statuses, complications, and utilizations of other orthopaedic services.

RESULTS

There were 109 patients (41%) who achieved an MCID at 1-year postoperatively. Non-white patients had 2.17 times lower odds of achieving MCID. No clinical outcomes assessed were independently predictive of MCID achievement. During the 1-year postoperative period, 63% of patients sought care for another orthopaedic condition. Patients requiring postoperative injections on another joint had a 2.27 times lower odds of achieving MCID. Those seen for spine conditions postoperatively had a 2.44 lower odds of achieving MCID.

CONCLUSIONS

Race, postoperative injections, and treatment for spine conditions after TKA were independent predictors of failure to achieve MCID. These results may guide preoperative patient consultation and risk-adjustment in future studies using PROMIS-PF as an endpoint for evaluation of TKA outcomes.

Social vulnerability adversely affects emergency-department utilization but not patient-reported outcomes after total joint arthroplasty

INTRODUCTION

Multiple studies demonstrate social deprivation is associated with inferior outcomes after total hip (THA) and total knee (TKA) arthroplasty; its effect on patient-reported outcomes is debated. The primary objective of this study evaluated the relationship between social vulnerability and the PROMIS-PF measure in patients undergoing THA and TKA. A secondary aim compared social vulnerability between patients who required increased resource utilization or experienced complications and those who didn't.

MATERIALS AND METHODS

A retrospective review of 537 patients from March 2020 to February 2022 was performed. The Centers for Disease Control Social Vulnerability Index (SVI) were used to quantify socioeconomic disadvantage. The cohort was split into THA and TKA populations; univariate and multivariate analyses were performed to evaluate primary and secondary outcomes. Statistical significance was assessed at p < 0.05.

RESULTS

48.6% of patients achieved PROMIS-PF MCID at 1-year postoperatively. Higher levels of overall social vulnerability (0.40 vs. 0.28, p = 0.03) were observed in TKA patients returning to the ED within 90-days of discharge. Increased overall SVI (OR = 9.18, p = 0.027) and household characteristics SVI (OR = 9.57, p = 0.015) were independent risk factors for 90-day ED returns after TKA. In THA patients, increased vulnerability in the household type and transportation dimension was observed in patients requiring 90-day ED returns (0.51 vs. 0.37, p = 0.04).

CONCLUSION

Despite an increased risk for 90-day ED returns, patients with increased social vulnerability still obtain good 1-year functional outcomes. Initiatives seeking to mitigate the effect of social deprivation on TJA outcomes should aim to provide safe alternatives to ED care during early recovery.

Does Wound VAC Temporization Offer Patient-Reported Outcomes Similar to Single-Stage Excision Reconstruction After Myxofibrosarcoma Resection?

BACKGROUND

Vacuum-assisted closure (VAC) temporization is a promising technique to achieve local control in aggressive soft tissue sarcomas. Despite its previously reported efficacy, adoption of VAC temporization remains limited, primarily due to the scarce literature on patient-reported outcomes (PROs) supporting its efficacy. This study compared the postoperative PROs after VAC temporization or single-stage (SS) excision and reconstruction for patients undergoing surgical resection for myxofibrosarcoma management.

METHODS

A retrospective analysis of myxofibrosarcoma patients who underwent surgical resections at our institution from 2016 to 2022 was performed. Postoperative PROs collected prospectively for those treated with VAC temporization or SS excision/reconstruction were compared using a visual analog scale (VAS) for pain and three Patient-Reported Outcomes Measurement Information System (PROMIS) questionnaires: Global Health Short-Form Mental (SF Mental), Global Health Short-Form Physical (SF Physical), and Physical Function Short-Form 10a (SF 10a). Absolute and differential (postoperative minus preoperative) scores at the 1-month, 3-month, 6-month, 1-year, and 2-year time points were compared.

RESULTS

The analysis included 79 patients (47 treated with VAC temporization and 32 treated with SS excision/reconstruction). All outcomes were similar between the groups except for physical function 1 year after surgery, in which the differential PROMIS SF 10a scores were higher in the SS group (p = 0.001). All the remaining absolute and differential PROMIS and VAS pain scores were similar between the groups at all time points. Postoperative complications did not differ between the groups.

CONCLUSION

The PROs for physical and mental health, physical function, and pain were similar between the myxofibrosarcoma patients who had VAC temporization and those who had SS excision/reconstruction after surgical resection.

Are Social Deprivation and Low Traditional Health Literacy Associated With Higher PROMIS CAT Completion in Orthopaedic Surgery?

BACKGROUND

The Patient-Reported Outcomes Measurement Information System® (PROMIS®) may be used to assess an individual patient's perspective of their physical, mental, and social health through either standard or computer adaptive testing (CAT) patient questionnaires. These questionnaires are used across disciplines; however, they have seen considerable application in orthopaedic surgery. Patient characteristics associated with PROMIS CAT completion have not been examined within the context of social determinants of health, such as social deprivation or health literacy, nor has patient understanding of the content of PROMIS CAT been assessed.

QUESTIONS/PURPOSES

(1) What patient demographics, including social deprivation, are associated with completion of PROMIS CAT questionnaires? (2) Is health literacy level associated with completion of PROMIS CAT questionnaires? (3) Do patients with lower health literacy have a higher odds of completing PROMIS CAT without fully understanding the content?

METHODS

Between June 2022 and August 2022, a cross-sectional study was performed via a paper survey administered to patients at a single, urban, quaternary academic medical center in orthopaedic subspecialty clinics of foot and ankle, trauma, and hand/upper extremity surgeons. We considered all English-speaking patients aged 18 or older, including those with limited reading and/or writing abilities, as eligible provided they received an iPad in clinic to complete the PROMIS CAT questionnaire as part of their routine standard clinical care or they completed the questionnaire via a patient portal before the visit. In all, 946 patients were considered eligible during the study period and a convenience sample of 36% (339 of 946) of patients was approached for inclusion due to clinic time constraints. Fifteen percent (52 of 339) declined to participate, leaving 85% (287 of 339) of patients for analysis here. Median (range) age of study participants was 49 years (35 to 64). Fifty-eight percent (167 of 287) of study participants self-identified as non-Hispanic Black or African American and 26% (75 of 287) as non-Hispanic White. Even proportions were observed across education levels (high school graduate or less, 29% [82 of 287]; some college, 25% [73 of 287]; college graduate, 25% [71 of 287]; advanced degree, 20% [58 of 287]). Eighteen percent (52 of 287) of patients reported an annual income bracket of USD 0 to 13,000, and 17% (48 of 287) reported more than USD 120,000. Forty-six percent (132 of 287) of patients worked full-time, 21% (59 of 287) were retired, and 23% (66 of 287) were unemployed or on disability. The primary outcome of interest was self-reported PROMIS CAT questionnaire completion grouped as: fully completed, partially completed, or no part completed. Overall, self-reported PROMIS CAT questionnaire completion proportions were: 80% (229 of 287) full completion, 13% (37 of 287) partial completion, and 7% (21 of 287) no part completed. We collected the National Area Deprivation Index (ADI) score and the Brief Health Literacy Screening Tool (BRIEF) as part of the study survey to associate with level of completion. Additionally, patient understanding of PROMIS CAT was assessed through Likert-scaled responses to a study survey question that directly asked whether the patient understood all of the questions on the PROMIS CAT questionnaire. Responses to this question may have been limited by social desirability bias, and hence may overestimate how many individuals genuinely understood the questionnaire content. However, the benefit of this approach was it efficiently allowed us to estimate the ceiling effect of patient comprehension of PROMIS CAT and likely had a high degree of specificity for detecting lack of comprehension.

RESULTS

ADI score adjusted for age was not associated with PROMIS CAT completion (partial completion OR 1.00 [95% CI 0.98 to 1.01]; p = 0.72, no part completed OR 1.01 [95% CI 0.99 to 1.03]; p = 0.45). Patients with lower health literacy scores, however, were more likely to not complete any part of their assigned questionnaires than patients with higher scores (no part completed OR 0.85 [95% CI 0.75 to 0.97]; p = 0.02). Additionally, 74% (26 of 35) of patients who did not fully understand all of the PROMIS CAT questionnaire questions still fully completed them-hence, 11% (26 of 229) of all patients who fully completed PROMIS CAT did not fully understand the content. Among patients self-reporting full completion of PROMIS CAT with health literacy data (99% [227 of 229]), patients with inadequate/marginal health literacy were more likely than patients with adequate health literacy to not fully understand all of the questions (21% [14 of 67] versus 8% [12 of 160], OR 3.26 [95% CI 1.42 to 7.49]; p = 0.005).

CONCLUSION

Within an urban, socioeconomically diverse, orthopaedic patient population, health literacy was associated with PROMIS CAT questionnaire completion. Lower health literacy levels increased the likelihood of not completing any part of the assigned PROMIS CAT questionnaires. Additionally, patients completed PROMIS CAT without fully understanding the questions. This indicates that patient completion does not guarantee comprehension of the questions nor validity of their scores, even more so among patients with low health literacy. This is a substantive concern for fidelity of data gathered from PROMIS CAT.

CLINICAL RELEVANCE

Clinical implementation of the PROMIS CAT in orthopaedic populations will benefit from further research into health literacy to increase questionnaire completion and to ensure that patients understand the content of the questions they are answering, which will increase the internal validity of the outcome measure.

Social Determinants of Health in Maryland Hip Arthroscopy Patients

Background Prior studies have demonstrated racial and socioeconomic disparities in patient-reported outcome measure (PROM) completion rates, and improvement exists across multiple orthopedic conditions. The purpose of this study was to assess whether these disparities are present in patients undergoing hip arthroscopy (HA) procedures. Methods A retrospective study of 306 patients undergoing HA from 2021 to 2023 was performed. Social determinants of health (SDOH) were compared between HA patients and the general Maryland population. Patients were then classified by whether they completed baseline and six-month PROMs (Patient-Reported Outcomes Measurement Information System Physical Function (PROMIS-PF) instrument). Patients who completed PROMIS-PF were classified by whether or not they achieved minimal clinically important difference (MCID) at six months. Demographics and SDOH were compared using univariate analyses between patients who did and did not complete PROMs and between those who did and did not achieve MCID. SDOH were evaluated at the zip-code level using regional health information exchange measures. Results Compared to the Maryland population, HA patients resided in areas of lower social vulnerability. Preoperative and six-month PROMs were completed by 102 (33%) patients. No significant differences in demographics or any SDOH were found between patients who did and did not complete PROMs. Six-month MCID was achieved in 75 of 102 (74%) patients with complete PROMs; no significant differences in demographics or SDOH were observed between patients who did and did not achieve MCID. Conclusions For patients undergoing HA, disparities in patient-reported outcome completion rates and postoperative functional improvement do not appear to be present across demographics and SDOH, indicating equitable care is being delivered.

Non-English Speakers and Socioeconomic Minorities are Significantly Less Likely to Complete Patient-Reported Outcome Measures for Total Hip and Knee Arthroplasty: Analysis of 16,119 Cases

BACKGROUND

The Comprehensive Care for Joint Replacement requires patient-reported outcome measure (PROM) completion for total knee/hip arthroplasty (TKA/THA) patients. A 90% completion rate to avoid penalties was planned for 2023 but has been delayed. Our analysis compares TKA/THA PROM completion and results across demographics. We hypothesized that minority groups would be less likely to complete PROMs.

METHODS

A retrospective review was performed from 2018 to 2021 of 16,119 patients who underwent primary elective TKA or THA at a single institution. Pairwise chi-squared tests, t-tests, analysis of variance, and multiple logistic regression analyses were used to compare PROM completion rates and scores across demographics and surgery type (TKA/THA).

RESULTS

Comparing patients who had (N = 7,664) and did not have (N = 8,455) documented PROMs, completion rates were significantly lower in patients who were women, Black, Hispanic, less educated, used Medicaid insurance, lived in lower income neighborhoods, spoke non-English languages, required an interpreter, and underwent TKA versus THA. After regression analyses, odds ratios for PROM completion remained significantly lower in non-English speakers, Hispanic and Medicaid patients, lower income groups, and patients undergoing TKA. For the 31.8% of patients who completed both preoperative/postoperative PROMs, women, Black, and non-English speaking patients had significantly lower PROM scores for most measures preoperatively and postoperatively despite similar or better improvements after surgery.

CONCLUSION

Patients undergoing TKA and non-English speaking, ethnic, and socioeconomic minorities are less likely to complete PROMs. Strategies to create, validate, and collect PROMs for these populations are needed to avoid exacerbation of healthcare disparities.