Interpreting Patient-Reported Outcome Measures in Orthopaedic Surgery: A Systematic Review

Defining clinically relevant outcome thresholds for pain and function after osteochondral autograft transplantation of the knee

PURPOSE

To define the minimal clinically important difference (MCID) for measures of pain and function at 2, 5 and 10 years after osteochondral autograft transplantations (OATs).

METHODS

Patients undergoing OATs of the knee were identified from a prospectively maintained cartilage surgery registry. Baseline demographic, injury and surgical factors were collected. Patient-reported outcome scores (PROMs) were collected at baseline, 2-, 5- and 10-year follow-up, including the International Knee Documentation Committee (IKDC) score, Knee Outcome Survey Activities of Daily Living Scale (KOS-ADLS), Marx activity scale and Visual Analogue Scale (VAS) for pain. The MCIDs were quantified for each metric utilizing a distribution-based method equivalent to one-half the standard deviation of the mean change in outcome score. The percentage of patients achieving MCID as a function of time was assessed.

RESULTS

Of 63 consecutive patients who underwent OATs, 47 (74.6%) patients were eligible for follow-up (surgical date before October 2021) and had fully completed preoperative PROMs. A total of 39 patients (83%) were available for a minimum 2-year follow-up, with a mean (±standard deviation) follow-up of 5.8 ± 3.4 years. The MCIDs were determined to be 9.3 for IKDC, 2.5 for Marx, 7.4 for KOS-ADLS and 12.9 for pain. At 2 years, 78.1% of patients achieved MCID for IKDC, 77.8% for Marx, 75% for KOS-ADLS and 57.9% for pain. These results were generally maintained through 10-year follow-ups, with 75% of patients achieving MCID for IKDC, 80% for Marx, 80% for KOS-ADLS and 69.8% for pain.

CONCLUSIONS

The majority of patients achieved a clinically relevant outcome improvement after OATs of the knee, with results sustained through 10-year follow-up. Patients who experience clinically relevant outcome improvement after OATs in the short term continue to experience sustained benefits at longer-term follow-up. These data provide valuable prognostic information when discussing patient candidacy and the expected trajectory of recovery.

LEVEL OF EVIDENCE

Level III.

What outcomes do patients value after orthopaedic trauma: A best-worst scaling choice experiment

INTRODUCTION

Patient-centered treatment and research should focus on the outcomes that matter to patients. The primary aim of this study was to determine the outcome preferences of patients after musculoskeletal trauma. The secondary aim was to identify discrepancies between outcome preferences of surgeons and patients.

METHODS

A Best-worst scaling choice experiment survey was administered to patients with operative lower extremity injuries and orthopaedic surgeons who take trauma call. Participants completed 13 choice sets of 3 randomly-ordered outcomes, including: a full recovery (back to normal) without any problems; a problem that requires additional surgery or hospital stay versus medication or treatment in clinic/emergency department; minimal to moderate versus severe pain for 6 weeks; need for crutches/walker versus wheelchair for 6-12 weeks; being unable to work for 6-12 weeks; requiring 2-4 weeks in a facility; a perfect versus poor or worst-possible EuroQol 5 Dimension (EQ-5D) score at 1 year; and death. Within each set, participants ranked their "most-preferred" to "least-preferred" outcomes. Responses were aggregated to calculate the relative importance, or marginal utility, of each outcome stratified by respondent type.

RESULTS

Fifty-five patients and 65 surgeons participated. The most preferred outcome for patients and surgeons was a full recovery (back to normal) without any problems, followed by minimal to moderate pain for 6 weeks and a perfect EQ-5D score. The least preferred outcomes were death and the worst EQ-5D score, which had similar marginal utility, followed by a poor EQ-5D score and a problem that needs another surgery or stay at a hospital, which also had similar marginal utility. Surgeons, in comparison to patients, assigned a higher marginal utility to perfect EQ-5D scores at one year (3.55 vs. 2.03; p < 0.0001) and a 2-4 week stay in a facility (0.52 vs. -0.21; p = 0.001), and a lower marginal utility to severe pain for 6 weeks (-0.58 vs. -0.08; p = 0.04) and a poor EQ-5D score (-1.88 vs. -1.03; p = 0.02).

CONCLUSIONS

A full recovery (back to normal) without any problems was the most-preferred outcome for both patients and surgeons. Patient-centered care and research should focus on both patients' return to baseline and the avoidance of complications.

LEVEL OF EVIDENCE

N/A.

[Translated article] 4 Domain Sports PROM in Spanish: Cross-cultural adaptation in the Chilean population and reliability analysis

BACKGROUND

Patient-Reported Outcome Measures (PROMs) are tools of increasing interest in the sports population. The purpose of this study was to perform the cross-cultural adaptation and reliability analysis of the 4 Domain Sports Patient-Reported Outcome Measure (4 DSP) into Spanish.

METHODS

A six-stage cross-cultural adaptation protocol was executed to obtain the Spanish version of the 4 DSP (S-4DSP). Subsequently, the questionnaire was administered to a population of 108 postoperative athletes with ACL (Anterior Cruciate Ligament) injuries. The questionnaire was administered again after 30 days. Acceptability, floor and ceiling effects, internal consistency (Cronbach's alpha), and reproducibility (Intraclass Correlation) were evaluated.

RESULTS

The S-4DSP was fully completed by 108 participants (mean age 34±10.75, 26% women), achieving 100% acceptability. No floor effect was detected. The statistical analysis yielded a global Cronbach's alpha for the questionnaire of 0.65, and domain-specific alphas of 0.88, 0.72, 0.27, and 0.68 for the first, second, third, and fourth domains, respectively. The Intraclass Correlation test reached a maximum of 0.94 and a minimum of 0.48 for the first and fifth questions, respectively.

CONCLUSIONS

The S-4DSP is a reliable and useful tool for evaluating Spanish-speaking athletes after ACL reconstruction.

Linking Patient-Reported Outcome Measure Scores to Adverse Event Data to Gain Insight into Overestimation of Postoperative Patient-Reported Outcome Measure Improvement After Total Hip Arthroplasty and Total Knee Arthroplasty Due to Selective Nonresponse

BACKGROUND

The purpose of the study was to gain insight into how clinically relevant improvement in patient-reported outcome measure scores after total hip arthroplasty (THA) and total knee arthroplasty (TKA) may be underestimated or overestimated, we compared patient-reported outcome measure respondents and nonrespondents on their adverse event rates and assessed whether adverse event occurrence was associated with clinically relevant patient-reported outcome measure improvement from those without adverse events.

METHODS

All primary THAs and TKAs performed in 19 Dutch hospitals between January 2017 and December 2019 were included. The hip disability and osteoarthritis outcome score-physical function short form (HOOS-PS) and knee injury and osteoarthritis outcome score-physical function short form (KOOS-PS) were used to assess the physical function after THA and TKA, respectively. Adverse events included 1-year revision, 30-day readmission, 30-day complications, and long (ie, >75th percentile) length of stay. A clinically relevant improvement was defined as at least a 10-point decrease in HOOS-PS and 9 points in KOOS-PS scores. Associations between adverse events and clinically relevant HOOS-PS and KOOS-PS improvement were assessed using binary logistic regression models adjusted for patient characteristics and clustering of patients within hospitals.

RESULTS

There were 20,338 THA and 18,082 TKA procedures included. Adverse events occurred more frequently in HOOS-PS and KOOS-PS nonrespondents than in respondents. The THA patients experiencing revision, complications, or long length of stay were less likely to experience clinically relevant HOOS-PS improvements (odds ratios of 0.11 [0.06 to 0.20], 0.44 [0.30 to 0.63], and 0.66 [0.50 to 0.88], respectively). The TKA patients experiencing revision or long length of stay were less likely to experience clinically relevant KOOS-PS improvements (odds ratios of 0.26 [0.12 to 0.55] and 0.63 [0.50 to 0.80], respectively).

CONCLUSIONS

Clinically relevant HOOS-PS and KOOS-PS improvements are likely overestimated, as nonrespondents had higher adverse event rates which were associated with lower likelihood to achieve clinically relevant HOOS-PS and KOOS-PS improvements.

Evaluation of Outcomes After Anterior Cruciate Ligament Reconstruction: What We Know, What We Have, and What to Consider

Measurement of success following anterior cruciate ligament reconstruction (ACLR) hinges on the appropriate use of high quality and meaningful outcome measures. We identified and categorized over 100 outcome measures for ACLR using the International Classification of Functioning, Disability and Health (ICF) model. The ICF model is a useful framework to facilitate decisions about outcome selection and describe recovery following ACL injury. We outline key considerations when selecting outcome measures during study design (purpose, measurement properties, sample size, global assessment) or evaluating reported outcomes (measurement properties, sample size, magnitude/precision, clinical relevance, applicability), and discuss challenges in outcome measurement following ACLR.

Patient-Reported Outcome Measurement and Minimal Clinically Important Difference for Hand Surgeons

Patient-reported outcome measurement (PROM) tools are used to evaluate health status and response to treatment and have been integral in the effort to improve the quality of care provided. Patient reported outcomes (PROs) have garnered additional attention since becoming a priority of the National Institutes of Health in the early part of this century, and their use in both clinical practice and research has subsequently increased. In the upper extremity, a variety of PRO instruments exist that can assist physicians in their ability to track and/or prognosticate outcomes, make comparisons between treatments as well as strengthen research methodologies, and help determine the value of care. A more complete interpretation of the clinical significance of patient-reported outcome measurements is informed by parameters such as minimal clinically important difference, substantial clinical benefit and patient acceptable symptom state.

Preoperative factors associated with failure to reach the patient acceptable symptom state after anterior cruciate ligament reconstruction in patients aged 40 and older

PURPOSE

To perform a predictive analysis to identify preoperative patient factors associated with failure to achieve a newly defined patient acceptable symptom state (PASS) for the International Knee Documentation Committee (IKDC) Score after anterior cruciate ligament reconstruction (ACLR) in patients aged ≥ 40 years with a minimum of 2-year follow-up.

METHODS

This was a secondary analysis of a retrospective review of all patients aged 40 years or older receiving a primary allograft ACLR at a single institution between the years of 2005 and 2016, with 2-year minimum follow-up. Using an updated PASS threshold of 66.7 for the International Knee Documentation Committee (IKDC) score previously established for this patient cohort, a univariate and multivariate analysis was performed to identify preoperative patient characteristics predictive of failure to achieve PASS.

RESULTS

A total of 197 patients with a mean follow-up of 6.2 ± 2.1 years (range 2.7 - 11.2) were included in the analysis (48.5 ± 5.6 years, 51.8% female, Body Mass Index (BMI) 25.9 ± 4.4). PASS was achieved by 162 patients (82.2%). Patients who failed to achieve PASS more often had lateral compartment cartilage defects (P = 0.001) and lateral meniscus tears (P = 0.004), higher BMIs (P = 0.004), and Workers' Compensation status (P = 0.043) on univariable analysis. Factors predictive of failure to achieve PASS on multivariable analysis included BMI and lateral compartment cartilage defect (OR 1.12 [1.03-1.23], P = 0.013; OR 5.1 [1.87-13.9], P = 0.001).

CONCLUSION

Among patients ≥ 40 years who receive a primary allograft ACLR, patients who fail to achieve PASS more often had lateral compartment cartilage defects and higher BMIs.

LEVEL OF EVIDENCE

Level IV.

All MCIDs Are Wrong, But Some May be Useful

OBJECTIVE

To demonstrate how to apply a baseline-adjusted receiver operator characteristic curve (AROC) analysis for minimum clinically important differences (MCIDs) in an empirical data set and discuss new insights relating to MCIDs.

DESIGN

Retrospective study.

METHODS

This study includes data from 999 active-duty military service patients enrolled in the United States Military Health System's Military Orthopedics Tracking Injuries and Outcomes Network. Anchored MCIDs were calculated using the standard receiver operator characteristic analysis and the AROC analysis for the Patient-Reported Outcome Measure Information System (PROMIS) Pain Interference and Defense and Veterans Pain Rating Scale (DVPRS). Point estimates where confidence intervals (CIs) crossed the 0.5 identity line on the area-under-the-curve (AUC) analysis were considered statistically invalid. MCID estimates where CIs crossed 0 were considered theoretically invalid.

RESULTS

In applying an AROC analysis, the region of AUC and MCID validity for the PROMIS Pain Interference score exists when the baseline score is greater than 61.0 but less than 72.3. For DVPRS, the region of MCID validity is when the baseline score is greater than 5.9 but less than 7.9.

CONCLUSION

Baseline values influence not only the MCID but also the accuracy of the MCID. MCIDs are statistically and theoretically valid for only a discrete range of baseline scores. Our findings suggest that the MCID may be too flawed a construct to accurately benchmark treatment outcomes. J Orthop Sports Phys Ther 2022;52(6):401-407. doi:10.2519/jospt.2022.11193.

Protocol for implementation of the 'AusPROM' recommendations for elective surgery patients: a mixed-methods cohort study

INTRODUCTION

Incorporating patient-reported outcome measures (PROMs) into usual care in hospitals can improve safety and quality. Gaps exist in electronic PROM (ePROM) implementation recommendations, including for elective surgery. The aims are to: (1) understand barriers and enablers to ePROM implementation in hospitals and develop Australian ePROM implementation recommendations (AusPROM); (2) test the feasibility and acceptability of the Quality of Recovery 15 item short-form (QoR-15) PROM for elective surgery patients applying the AusPROM and (3) establish if the QoR-15 PROM has concurrent validity with the EQ-5D-5L.

METHODS AND ANALYSIS

Phase I will identify staff barriers and facilitators for the implementation of the AusPROM recommendations using a Delphi technique. Phase II will determine QoR-15 acceptability for elective surgery patients across four pilot hospitals, using the AusPROM recommendations. For phase II, in addition to a consumer focus group, patients will complete brief acceptability surveys, incorporating the QoR-15, in the week prior to surgery, in the week following surgery and 4 weeks postsurgery. The primary endpoint will be 4 weeks postsurgery. Phase III will be the national implementation of the AusPROM (29 hospitals) and the concurrent validity of the QoR-15 and generic EQ-5D-5L. This protocol adopts the Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trials Protocols guidelines.

ETHICS AND DISSEMINATION

The results will be disseminated via public forums, conferences and peer-reviewed journals. Ethics approval: La Trobe University (HEC20479).

TRIAL REGISTRATION NUMBER

ACTRN12621000298819 (Phase I and II) and ACTRN12621000969864 (Phase III).

Patient-acceptable symptom state for reporting outcomes following unicompartmental knee arthroplasty : a matched pair analysis comparing UKA in ACL-deficient versus ACL-intact knees

AIMS

The patient-acceptable symptom state (PASS) is a level of wellbeing, which is measured by the patient. The aim of this study was to determine if the proportion of patients who achieved an acceptable level of function (PASS) after medial unicompartmental knee arthroplasty (UKA) was different based on the status of the anterior cruciate ligament (ACL) at the time of surgery.

METHODS

A total of 114 patients who underwent UKA for isolated medial osteoarthritis (OA) of the knee were included in the study. Their mean age was 65 years (SD 10). No patient underwent a bilateral procedure. Those who had undergone ACL reconstruction during the previous five years were excluded. The Knee injury Osteoarthritis Outcome Score Activities of Daily Living (KOOS ADL) function score was used as the primary outcome measure with a PASS of 87.5, as described for total knee arthroplasty (TKA). Patients completed all other KOOS subscales, Lysholm score, the Western Ontario and McMaster Universities Osteoarthritis Index, and the Veterans Rand 12-item health survey score. Failure was defined as conversion to TKA.

RESULTS

Survivorship at ten years was 97% in both the ACL-deficient and ACL-intact groups. The mean survival was 16.1 years (95% confidence interval (CI) 15.3 to 16.8) for the ACL-deficient group and 15.6 years (95% CI 14.8 to 16.361) for the ACL-intact group (p = 0.878). At a mean of nine years (SD 3.5) in the ACL-deficient group, 32 patients (87%) reached the PASS for the KOOS ADL. In the ACL-intact group, at a mean of 8.6 years (SD 3) follow-up, 63 patients (85%) reached PASS for the KOOS ADL. There was no significant difference in the percentage of patients who reached PASS for all KOOS subscales and Lysholm between the two groups.

CONCLUSION

PASS was achieved in 85% of all UKAs for KOOS ADL, similar to reports for TKA. Fixed-bearing, medial, non-robotically-assisted UKA resulted in 97% survival at ten years in both the ACL-deficient and ACL-intact groups. There was no significant difference in all outcomes between the two groups. Understanding PASS will allow better communication between surgeons and patients to improve the surgical management of patients with single compartment OA of the knee. This study provides mid- to long-term data supporting the use of PASS to document outcomes following UKA. PASS was met in more than 85% of patients with no differences between ACL-deficient and ACL-intact knees at a mean follow-up of nine years. Cite this article: Bone Joint J 2021;103-B(8):1367-1372.