Is the Patient-Reported Outcome Measurement Information System Feasible in Bundled Payment for Care Improvement in Total Hip Arthroplasty Patients?

Assessing the measurement properties of PROMIS Computer Adaptive Tests, short forms and legacy patient reported outcome measures in patients undergoing total hip arthroplasty

BACKGROUND

The commonly used ('legacy') PROMs evaluating outcomes of total hip arthroplasty (THA), have several limitations regarding their measurement properties and interpretation of scores. One innovation in PROMs is the use of Computerized Adaptive Testing (CAT). The Patient-Reported Outcomes Measurement Information System (PROMIS®) is a validated system of CATs. The aim of this study was to assess the measurement properties of PROMIS and legacy instruments in patients undergoing THA.

METHODOLOGY

Patients in this multicenter study filled out a questionnaire twice, including Dutch-Flemish PROMIS v1.2 Physical Function (PROMIS-PF) and v1.1 Pain Interference (PROMIS-PI) CATs and short forms, PROMIS v1.0 Pain Intensity, and legacy PROMs (Hip disability and Osteoarthritis Outcome Score (HOOS), HOOS-Physical function Shortform (HOOS-PS), Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), Oxford Hip Score (OHS), and two numeric rating scales measuring pain). The reliability, measurement precision (Standard Error of Measurement (SEM)), smallest detectable change (SDC), and burden of PROMIS instruments were presented head-to-head to legacy PROMs. Furthermore, construct validity was assessed.

RESULTS

208 patients were included. All instruments had a sufficient test-retest reliability (range ICC: 0.83-0.96). The SEM of PROMIS CATs and short forms ranged from 1.8 to 2.2 T-score points, the SEM of legacy instruments 2.6-11.1. The SDC of PROMIS instruments ranged from 2.1 to 7.3 T-score points, the SDC of legacy instruments 7.2-30.9. The construct validity of PROMIS CAT and short forms were found sufficient, except for the PROMIS-PI short form. The burden of PROMIS CATs was smaller than PROMIS short forms (range 4.8-5.2 versus 8-20 items, respectively). The burden of legacy instruments measuring physical functioning ranged from 5 to 40 items.

CONCLUSIONS

The PROMIS-PF is less burdensome, with high measurement precision, and almost no minimal or maximal scores, and an equal reliability compared to legacy instruments measuring physical functioning in patients undergoing THA. The PROMIS Pain Intensity 1a is comparable to the legacy pain instruments in terms of burden, reliability and SDC. Measuring the construct Pain Interference may not have additional value in this population because of its high correlation with instruments measuring physical functioning. The SDC values presented in this study can be used for individual patient monitoring.

Pearls and pitfalls of PROMIS clinically significant outcomes in orthopaedic surgery

INTRODUCTION

Patient-Reported Outcome Measurement Information System (PROMIS) was developed as a uniform and generalizable PROM system using item response theory and computer adaptive testing. We aimed to assess the utilization of PROMIS for clinically significant outcomes (CSOs) measurements and provide insights into its use in orthopaedic research.

MATERIALS AND METHODS

We reviewed PROMIS CSO reports for orthopaedic procedures via PubMed, Cochrane Library, Embase, CINAHL, and Web of Science from inception to 2022, excluding abstracts and missing measurements. Bias was assessed using the Newcastle-Ottawa Scale (NOS) and questionnaire compliance. PROMIS domains, CSO measures, and study populations were described. A meta-analysis compared distribution and anchor-based MCIDs in low-bias (NOS ≥ 7) studies.

RESULTS

Overall, 54 publications from 2016 to 2022 were reviewed. PROMIS CSO studies were observational with increasing publication rates. Evidence-level was II in 10/54, bias low in 51/54, and compliance ≥ 86% in 46/54. Most (28/54) analysed lower extremity procedures. PROMIS domains examined Pain Function (PF) in 44/54, Pain Interference (PI) in 36/54, and Depression (D) in 18/54. Minimal clinically important difference (MCID) was reported in 51/54 and calculated based on distribution in 39/51 and anchor in 29/51. Patient acceptable symptom state (PASS), substantial clinical benefit (SCB), and minimal detectable change (MDC) were reported in ≤ 10/54. MCIDs were not significantly greater than MDCs. Anchor-based MCIDs were greater than distribution based MCIDs (standardized mean difference = 0.44, p < 0.001).

CONCLUSIONS

PROMIS CSOs are increasingly utilized, especially for lower extremity procedures assessing the PF, PI, and D domains using distribution-based MCID. Using more conservative anchor-based MCIDs and reporting MDCs may strengthen results. Researchers should consider unique pearls and pitfalls when assessing PROMIS CSOs.

What is the Impact of Social Deprivation on Mental and Physical Health Before and After Primary Total Knee Arthroplasty?

Background

The purpose of the present study was to investigate the relationship between socioeconomic status and Patient-Reported Outcomes Measurement Information System Global Health (PROMIS-GH) scores before and after primary total knee arthroplasty (TKA). We hypothesized that patients with greater social deprivation would have lower PROMIS-GH scores at 3 months and 1 year following primary TKA.

Methods

We retrospectively reviewed data from patients who underwent unilateral primary TKA and completed PROMIS-GH preoperatively and at 3 months (n = 257) or 1 year (n = 154) postoperatively. Area Deprivation Index (ADI), calculated from 9-digit zip codes, was used to measure social deprivation. Participants were grouped into quartiles by ADI score. Minimal clinically important difference in PROMIS-GH mental (PROMIS-MH) and physical health (PROMIS-PH) component scores were compared between ADI groups.

Results

Participants in the highest ADI quartile (most disadvantaged) had significantly lower PROMIS-MH and PROMIS-PH scores at every time point relative to the lowest ADI quartile (least disadvantaged) (P < .05 for all). Both ADI groups experienced significant improvements in PROMIS-PH following TKA (P < .001 for all), but not in PROMIS-MH (P > .05 for all) at 3-months and 1-year postoperatively. Magnitude of improvement in PROMIS-PH and rates of achievement of minimal clinically important difference did not significantly differ between ADI groups (P > .05 for all).

Conclusions

Socially disadvantaged patients benefit equally from primary TKA but are more likely to have persistently lower 1-year postoperative PROMIS-GH scores relative to less disadvantaged patients. Social deprivation should be accounted for when using PROMIS-GH to assess clinical outcomes for research and quality measures.

Level of Evidence

IV, retrospective cohort study.

Discordance Abounds in Minimum Clinically Important Differences in THA: A Systematic Review

BACKGROUND

The minimum clinically important difference (MCID) is intended to detect a change in a patient-reported outcome measure (PROM) large enough for a patient to appreciate. Their growing use in orthopaedic research stems from the necessity to identify a metric, other than the p value, to better assess the effect size of an outcome. Yet, given that MCIDs are population-specific and that there are multiple calculation methods, there is concern about inconsistencies. Given the increasing use of MCIDs in total hip arthroplasty (THA) research, a systematic review of calculated MCID values and their respective ranges, as well as an assessment of their applications, is important to guide and encourage their use as a critical measure of effect size in THA outcomes research.

QUESTIONS/PURPOSES

We systematically reviewed MCID calculations and reporting in current THA research to answer the following: (1) What are the most-reported PROM MCIDs in THA, and what is their range of values? (2) What proportion of studies report anchor-based versus distribution-based MCID values? (3) What are the most common methods by which anchor-based MCID values are derived? (4) What are the most common derivation methods for distribution-based MCID values? (5) How do the reported medians and corresponding ranges compare between calculation methods for each PROM?

METHODS

The EMBASE, MEDLINE, and PubMed databases were systematically reviewed from inception through March 2022 for THA studies reporting an MCID value for any PROMs. Two independent authors reviewed articles for inclusion. All articles calculating new PROM MCID scores after primary THA were included for data extraction and analysis. MCID values for each PROM, MCID calculation method, number of patients, and study demographics were extracted from each article. In total, 30 articles were included. There were 45 unique PROMs for which 242 MCIDs were reported. These studies had a total of 1,000,874 patients with a median age of 64 years and median BMI of 28.7 kg/m 2 . Women made up 55% of patients in the total study population, and the median follow-up period was 12 months (range 0 to 77 months). The overall risk of bias was assessed as moderate using the modified Methodological Index for Nonrandomized Studies criteria for comparative studies (the mean score for comparative papers in this review was 18 of 24, with higher scores representing better study quality) and noncomparative studies (for these, the mean score was 10 of a possible 16 points, with higher scores representing higher study quality). Calculated values were classified as anchor-based, distribution-based, or not reported. MCID values for each PROM, MCID calculation method, number of patients, and study demographics were extracted from each study. Anchor-based and distribution-based MCIDs were compared for each unique PROM using a Wilcoxon rank sum test, given the non-normal distribution of values.

RESULTS

The Oxford Hip Score (OHS) and the Hip Injury and Osteoarthritis Score (HOOS) Pain and Quality of Life subscore MCIDs were the most frequently reported, comprising 12% (29 of 242), 8% (20 of 242), and 8% (20 of 242), respectively. The EuroQol VAS (EQ-VAS) was the next-most frequently reported (7% [17 of 242]) followed by the EuroQol 5D (EQ-5D) (7% [16 of 242]). The median anchor-based value for the OHS was 9 (IQR 8 to 11), while the median distribution-based value was 6 (IQR 5 to 6). The median anchor-based MCID values for HOOS Pain and Quality of Life were 33 (IQR 28 to 35) and 25 (14 to 27), respectively; the median distribution-based values were 10 (IQR 9 to 10) and 13 (IQR 10 to 14), respectively. Thirty percent (nine of 30) of studies used an anchor-based method to calculate a new MCID, while 40% (12 of 30) used a distribution-based technique. Thirty percent of studies (nine of 30) calculated MCID values using both methods. For studies reporting an anchor-based calculation method, a question assessing pain relief, satisfaction, or quality of life on a five-point Likert scale was the most commonly used anchor (30% [eight of 27]), followed by a receiver operating characteristic curve estimation (22% [six of 27]). For studies using distribution-based calculations, the most common method was one-half the standard deviation of the difference between preoperative and postoperative PROM scores (46% [12 of 26]). Most reported median MCID values (nine of 14) did not differ by calculation method for each unique PROM (p > 0.05). The OHS, HOOS JR, and HOOS Function, Symptoms, and Activities of Daily Living subscores all varied by calculation method, because each anchor-based value was larger than its respective distribution-based value.

CONCLUSION

We found that MCIDs do not vary very much by calculation method across most outcome measurement tools. Additionally, there are consistencies in MCID calculation methods, because most authors used an anchor question with a Likert scale for the anchor-based approach or used one-half the standard deviation of preoperative and postoperative PROM score differences for the distribution-based approach. For some of the most frequently reported MCIDs, however, anchor-based values tend to be larger than distribution-based values for their respective PROMs.

CLINICAL RELEVANCE

We recommend using a 9-point increase as the MCID for the OHS, consistent with the median reported anchor-based value derived from several high-quality studies with large patient groups that used anchor-based approaches for MCID calculations, which we believe are most appropriate for most applications in clinical research. Likewise, we recommend using the anchor-based 33-point and 25-point MCIDs for the HOOS Pain and Quality of Life subscores, respectively. We encourage using anchor-based MCID values of WOMAC Pain, Function, and Stiffness subscores, which were 29, 26, and 30, respectively.

Metrics of Clinically Important Changes in Total Hip Arthroplasty: A Systematic Review

BACKGROUND

Although patient-reported outcome measures (PROMs) have become a regularly used metric, there is little consensus on the methodology used to determine clinically relevant postoperative outcomes. We systematically reviewed the literature for studies that have identified metrics of clinical efficacy after total hip arthroplasty (THA) including minimal clinically important difference (MCID), patient acceptable symptom state (PASS), minimal detectable change (MDC), and substantial clinical benefit (SCB).

METHODS

A systematic review examining quantitative metrics for assessing clinical improvement with PROMs following THA was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines using the MEDLINE database from 2008 to 2020. Inclusion criteria included full texts, English language, primary THA with minimum 1-year follow-up, use of metrics for assessing clinical outcomes with PROMs, and primary derivations of those metrics. Sixteen studies (24,487 THA patients) met inclusion criteria and 11 different PROMs were reported.

RESULTS

MCIDs were calculated using distribution methods in 7 studies (44%), anchor methods in 2 studies (13%), and both methods in 2 studies (13%). MDC was calculated in 2 studies, PASS was reported in 1 study using anchor-based method, and SCB was calculated in 1 study using anchor-based method.

CONCLUSION

There is a lack of consistency in the literature regarding the use and interpretation of PROMs to assess patient satisfaction. MCID was the most frequently reported measure, while MDC, SCB, and PASS were used relatively infrequently. Method of derivation varied based on the PROM used; distribution method was more frequently used for MCID.

One-Year Readmissions Following Total Joint Arthroplasty May Be Associated With Failure to Achieve the Minimal Clinically Important Difference of Patient-Reported Outcomes Measurement Information System Physical, Mental, and Physical-Short Form-10a

The primary aims of our study were to determine if hospital readmissions within one year following primary total joint arthroplasty (TJA) and their relative timing influence patients' ability to achieve the two-year Patient-Reported Outcomes Measurement Information System (PROMIS) physical, PROMIS mental, and PROMIS Physical-Function-Short-Form-10a (SF-10a) minimal clinically important difference (MCID). This is a retrospective study conducted using data from a multi-institutional, arthroplasty registry. Only patients with paired patient-reported outcome measure (PROM) assessments (preoperatively and two years postoperatively) were included. Five separate readmission cohorts were formed: (1) any-cause readmission within one year, (2) any-cause readmission within 90 days, (3) non-index-surgery-related readmission within 90 days, (4) index-surgery-related readmission within one year, and (5) index-surgery-related readmission within 90 days. A propensity score match was used to match each of the patients to one of the 972 patients (1:1 basis) in the non-readmission group. The association between failure to achieve each of the three two-year MCIDs and Readmission status was analyzed using logistic regression. We found that all readmissions within one year and index-surgery-related readmissions within one year resulted in an increased risk of failure to achieve the two-year MCID across all three collected PROMs. Index surgery-related readmissions within 90 days (OR 3.24; 95% CI 1.05-11.05; p=0.048) sustained significantly different rates of two-year PROMIS physical MCID achievement compared to matched controls. Postoperative complications requiring readmission, particularly those related to the joint arthroplasty and those within 90 days of index surgery, significantly impact the ability to achieve the two-year MCID of PROMs.

Projected Prevalence of Obesity in Primary Total Knee Arthroplasty: How Big Will the Problem Get?

BACKGROUND

Obesity is a well-established risk factor for complications following primary total knee arthroplasty (TKA). The purpose of this study is to utilize 3 national databases to develop projections of obesity within the general population and primary TKA patients in the United States through 2029.

METHODS

Data from the National Surgical Quality Improvement Program (NSQIP), the Behavior Risk Factor Surveillance System (BRFSS), and the National Health and Nutrition Examination Survey were queried for years 1999-2019. Current Procedural Terminology code 27447 was used to identify primary TKA patients in NSQIP. Individuals were categorized according to body mass index (kg/m²) by year: normal weight (≤24.9); overweight (25.0-29.9); obese (30.0-39.9); and morbidly obese (≥40). Multinomial logistic regression was used to project categorical body mass index data for years 2020-2029.

RESULTS

A total of 8,372,221 individuals were included (7,986,414 BRFSS, 385,807 NSQIP TKA). From 2011 to 2019, the prevalence of normal weight and overweight individuals declined in the general population (BRFSS) and in primary TKA. Prevalence of obese/morbidly obese individuals increased in the general population from 31% to 36% and in primary TKA from 60% to 64%. Projection models estimate that by 2029, 46% of the general population will be obese/morbidly obese and 69% of primary TKA will be obese/morbidly obese.

CONCLUSION

By 2029, we estimate ≥69% of primary TKA to be obese/morbidly obese. Increased resources dedicated to care pathways and research focused on improving outcomes in obese arthroplasty patients will be necessary as this population continues to grow.

LEVEL OF EVIDENCE

Level III, Retrospective Cohort Study.

Measurement properties of PROMIS short forms for pain and function in total hip arthroplasty patients

Introduction

While the Patient-Reported Outcomes Measurement Information System (PROMIS) is mainly designed for computer adaptive testing, its static short forms (SF) are used when a paper-pencil format is preferred or item banks are not yet translated into the target language. This study examined the measurement properties of the German PROMIS-SF for pain intensity (PAIN), pain interference (PI) and physical function (PF) in total hip arthroplasty (THA) patients.

Methods

SF were collected before and 12 months post-surgery. Higher scores indicate more PAIN, higher PI and better PF. Oxford Hip Score (OHS) was the main reference measure. Six months post-surgery, a subsample completed the SF twice within 14 days to test reliability.

Results

Of 172 eligible patients, 147 consented to participate and received questionnaires; 132 (74 males) returned baseline questionnaires (mean age 65.8 ± 10.2 years) and 116, 12-month questionnaires. Forty-five patients provided test-retest data.

Correlations of all SF with OHS were large (│r│ ≥ 0.7; confidence intervals did not include 0.50). Cronbach’s alpha values were: PAIN, 0.86; PI, 0.93; PF, 0.91. Intraclass correlation coefficients were: PAIN, 0.77; PI, 0.81; PF, 0.69. Standard errors of measurement were: PAIN, 3.8; PI, 2.8; PF, 3.6. Smallest detectable change thresholds were: PAIN, 8.8; PI, 6.6; PF, 8.4. Follow-up data showed a ceiling effect (best score) for PAIN (66%), PI (76%), and PF (66%). SF change scores showed large correlations with OHS change scores (│r│ > 0.6).

Conclusion

Our results provide some evidence of construct validity, and acceptable reliability and responsiveness of PROMIS-SF for pain and function in THA patients. These SF can thus be considered acceptable for use, although patients’ improvement in physical function might be underestimated due to the large follow-up PF score ceiling effects.

Measurement qualities of PROMIS instruments are mainly assessed for computer adaptive testing but not for non-adaptive short questionnaires. As these questionnaires are in use, their measurement properties must also be evaluated. Results from computer adaptive testing cannot simply be transferred.

We studied the measurement qualities of the German PROMIS short questionnaires for pain intensity, pain interference and physical function in patients undergoing hip replacement. We wanted to see how these questionnaires perform when compared to the Oxford Hip Score, a standard questionnaire commonly used to test hip-related disability in these patients.

The three questionnaires can be considered acceptable for use in hip replacement patients, but some limitations do exist. Patient improvement in physical function might be underestimated because many patients reach the highest possible score and further improvements cannot be measured. Also, any small but important improvement in physical function cannot be distinguished from measurement error in individual patients.

Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures

We define the minimal important change (MIC) as a threshold for a minimal within-person change over time above which patients perceive themselves importantly changed. There is a lot of confusion about the concept of MIC, particularly about the concepts of minimal important change and minimal detectable change, which questions the validity of published MIC values. The aims of this study were: (1) to clarify the concept of MIC and how to use it; (2) to provide practical guidance for estimating methodologically sound MIC values; and (3) to improve the applicability of PROMIS by summarizing the available evidence on plausible PROMIS MIC values. We discuss the concept of MIC and how to use it and provide practical guidance for estimating MIC values. In addition, we performed a systematic review in PubMed on MIC values of any PROMIS measure from studies using recommended approaches. A total of 50 studies estimated the MIC of a PROMIS measure, of which 19 studies used less appropriate methods. MIC values of the remaining 31 studies ranged from 0.1 to 12.7 T-score points. We recommend to use the predictive modeling method, possibly supplemented with the vignette-based method, in future MIC studies. We consider a MIC value of 2-6 T-score points for PROMIS measures reasonable to assume at this point. For surgical interventions a higher MIC value might be appropriate. We recommend more high-quality studies estimating MIC values for PROMIS.

Is the Patient-Reported Outcome Measurement Information System Feasible in Bundled Payment for Care Improvement Total Knee Arthroplasty Patients?

BACKGROUND

Several bundled payment plans, like the Bundled Payment for Care Improvement (BPCI) initiative for total joint arthroplasty, have been introduced to decrease costs and improve clinical care. Measuring clinical outcomes with efficient, standardized methodologies is essential to determine the relative value of total joint arthroplasty care. We investigated feasibility and responsiveness of the recently developed Patient-Reported Outcomes Measurement Information System (PROMIS) in total knee arthroplasty (TKA) patients.

METHODS

We included patients with preoperative and 1-year PROMIS Physical Function (PF), Pain Interference (PI), and Depression (DEP) scores who received unilateral primary TKA. Burden was assessed using the number of questions and time required for PROMIS completion. The minimum clinically important difference was defined as 5. Floor/ceiling effects were noted if more than 15% of patients responded with the lowest/highest possible score, respectively. Wilcoxon rank-sum test was used to compare categorical data. Analysis of variance was used for PROMIS comparisons.

RESULTS

In total, 172 knees (54 BPCI) were included. Floor effects were identified for DEP at baseline (non-BPCI) and follow-up (both groups), and for PI at follow-up only (BPCI). Patients required 140 seconds and 16 questions to answer all 3 PROMIS domains. Sixty-seven percent, 60%, and 44% of knees achieved minimum clinically important difference in PI, PF, and DEP scores respectively, with no significant difference between groups. The BPCI cohort was older (P < .001) with a higher American Society of Anesthesiologists score (P = .028). There were no significant differences in scores between BPCI and non-BPCI patients.

CONCLUSION

PROMIS is feasible and time-efficient in BPCI patients undergoing primary TKA. There were no significant differences in outcomes between BPCI and non-BPCI knees.

LEVEL OF EVIDENCE

Level III.