The PROMIS CAT Demonstrates Responsiveness in Patients After ACL Reconstruction Across Numerous Health Domains

PROMIS physical function and pain perform poorly psychometrically in patients undergoing medial patellofemoral ligament reconstruction

PURPOSE

The purpose of this study was to assess the validity, floor and ceiling effects, and dimensionality of PROMIS Physical Function (PF) and Pain compared to legacy patient reported outcome (PRO) measures in patients undergoing medial patellofemoral ligament (MPFL) reconstruction.

METHODS

Patients who underwent MPFL reconstruction between 2018 to 2020 were retrospectively reviewed. Preoperatively, patients completed the IKDC, VR-12, Kujala, SF-12, KOOS JR, PROMIS PF and Pain surveys. Inter-survey convergence was assessed with Spearman correlations. Psychometric analysis included investigations of inter-survey convergent validity, intra-survey floor and ceiling effects, and Rasch analyses with person-item fit and iterative question elimination model fit testing.

RESULTS

A total of 76 patients (mean age: 22.6 ± 8.4 years) who completed preoperative surveys were included (compliance: 91.7-96.2%). Preoperatively, age was significantly associated with both PROMIS PF (coefficient:  - 0.291, P = 0.005) and Pain scores (coefficient: 0.294, P = 0.002). PROMIS PF had a Very Good correlation with IKDC and PROMIS Pain had a Very Good correlation with KOOS JR. Other correlations ranged from Poor to Good. No significant floor or ceiling effects were observed for any PRO. On iterative question elimination Rasch modelling, only two questions from PROMIS PF remained after 6 rounds of elimination while PROMIS Pain had no questions remaining after 3 rounds of elimination.

CONCLUSION

Preoperative PROMIS PF and Pain provided only Fair to Good correlations with most legacy PROs. Although no significant floor and ceiling effects were observed, PROMIS PF and Pain did not perform well psychometrically in this population of patients undergoing MPFL reconstruction. These results suggest PROMIS questionnaires should be used with caution in this population, especially if used in isolation, when discussing clinical expectations with patients. The authors recommend continued use of legacy PROs specific to assessment of patellar instability and function that have established validity in patellar instability populations.

LEVEL OF EVIDENCE

Level IV.

There is Little or No Association Between Independently Assessed Communication Strategies and Patient Ratings of Clinician Empathy

BACKGROUND

Quality of care is increasingly assessed and incentivized using measures of patient-reported outcomes and experience. Little is known about the association between measurement of clinician communication strategies by trained observers and patient-rated clinician empathy (a patient-reported experience measure). An effective independent measure could help identify and promote clinician behaviors associated with good patient experience of care.

QUESTIONS/PURPOSES

(1) What is the association between independently assessed clinician communication effectiveness and patient-rated clinician empathy? (2) Which factors are associated with independently assessed communication effectiveness?

METHODS

One hundred twenty adult (age > 17 years) new or returning patients seeking musculoskeletal specialty care between September 2019 and January 2020 consented to video recording of their visit followed by completion of questionnaires rating their perceptions of providers' empathy levels in this prospective study. Patients who had operative treatment and those who had nonoperative treatment were included in our sample. We pooled new and returning patients because our prior studies of patient experience found no influence of visit type and because we were interested in the potential influences of familiarity with the clinician on empathy ratings. We did not record the number of patients or baseline data of patients who were approached, but most patients (> 80%) were willing to participate. For 7% (eight of 120 patients), there was a malfunction with the video equipment or files were misplaced, leaving 112 records available for analysis. Patients were seen by one provider among four attending physicians, four residents, or four physician assistants or nurse practitioners. The primary study question addressed the correlation between patient-rated clinician empathy using the Jefferson Scale of Patient Perceptions of Physician Empathy and clinician communication effectiveness, independently rated by two communication scholars using the Liverpool Communication Skills Assessment Scale. Based on a subset of 68 videos (61%), the interrater reliability was considered good for individual items on the Liverpool Communication Skills Assessment Scale (intraclass correlation coefficient [ICC] 0.78 [95% confidence interval (CI) 0.75 to 0.81]) and excellent for the sum of the items (that is, the total score) (ICC = 0.92 [95% CI 0.87 to 0.95]). To account for the potential association of personal factors with empathy ratings, patients completed measures of symptoms of depression (the Patient-Reported Outcome Measurement Information System depression computerized adaptive test), self-efficacy in response to pain (the two-item Pain Self-Efficacy Questionnaire), health anxiety (the five-item Short Health Anxiety Inventory), and basic demographics.

RESULTS

Accounting for potentially confounding variables, including specific clinicians, marital status, and work status in the multivariable analysis, we found higher independent ratings of communication effectiveness had a slight association (odds ratio [OR] 1.1 [95% CI 1.0 to 1.3]; p = 0.02) with higher (dichotomized) ratings of patient-rated clinician empathy, while being single was associated with lower ratings (OR 0.40 [95% CI 0.16 to 0.99]; p = 0.05). Independent ratings of communication effectiveness were slightly higher for women (regression coefficient 1.1 [95% CI 0.05 to 2.2]); in addition, two of the four attending physicians were rated notably higher than the other 10 participants after controlling for confounding variables (differences up to 5.8 points on average [95% CI 2.6 to 8.9] on a 36-point scale).

CONCLUSION

The observation that ratings of communication effectiveness by trained communication scholars have little or no association with patient-rated clinician empathy suggests that either effective communication is insufficient for good patient experience or that the existing measures are inadequate or inappropriate. This line of investigation might be enhanced by efforts to identify clinician behaviors associated with better patient experience, develop reliable and effective measures of clinician behaviors and patient experience, and use those measures to develop training approaches that improve patient experience.

LEVEL OF EVIDENCE

Level I, prognostic study .

Comparison of the PROMIS and iHOT-12 in Determining Satisfaction Levels After Hip Arthroscopy for FAIS

Background

The Patient-Reported Outcomes Measurement Information System (PROMIS) has not been fully evaluated for outcomes assessment after hip arthroscopy to correct femoroacetabular impingement syndrome (FAIS).

Purpose/Hypothesis

The purpose of this study was to compare the accuracy of the PROMIS Physical Function (PF) and Pain Interference (PI) subscales with the 12-Item International Hip Outcome Tool (iHOT-12) to define patients with 3 unique substantial clinical benefit (SCB) scores-patients who reported ≥80%, ≥90%, and 100% satisfaction at 1 year after hip arthroscopy for FAIS. We hypothesized that the iHOT-12 would be more accurate than the PROMIS-PF and PROMIS-PI subscales in identifying these 3 patient groups.

Study Design

Cohort study (diagnosis); Level of evidence, 2.

Methods

We reviewed the records of patients who underwent hip arthroscopy for symptomatic FAIS at 3 centers from January 2019 through June 2021 and had 1-year clinical and radiographic follow-up data. Patients completed the iHOT-12, PROMIS-PF, and PROMIS-PI on initial assessment and at 1 year (±30 days) postoperatively. Postoperative satisfaction was reported on an 11-category scale with anchors defined as "0% satisfied" and "100% satisfied." Receiver operator characteristic analysis was performed to determine the absolute SCB values for the iHOT-12 and PROMIS subscales that would most accurately identify those patients who reported ≥80%, ≥90%, and 100% satisfaction. Area under the curve (AUC) values and 95% CIs for the 3 instruments were compared.

Results

Included were 163 patients (111 [68%] women and 52 [32%] men), with a mean age of 26.1 years. Corresponding absolute SCB scores for patients who reported ≥80%, ≥90%, and 100% satisfaction were as follows: iHOT-12, 68.4, 72.1, 74.7; PROMIS-PF, 45, 47.7, 49.9; and PROMIS-PI, 55.9, 52.4, 51.9. The AUC ranged between 0.67 and 0.82, with overlapping 95% CIs indicating a minimal difference in accuracy between the 3 instruments. Sensitivity and specificity values ranged between 0.61 and 0.82.

Conclusion

The PROMIS-PF and PROMIS-PI subscales were as accurate as the iHOT-12 in defining absolute SCB scores for patients reporting ≥80%, ≥90%, and 100% satisfaction at 1-year follow-up after hip arthroscopy for FAIS.

Clinically Depressed Patients Having Anterior Cruciate Ligament Reconstruction Show Improved but Inferior Rate of Achieving Minimum Clinically Important Difference for Patient-Reported Outcomes Measurement Information System Compared With Situationally Depressed or Nondepressed Patients

PURPOSE

To determine whether the preoperative diagnosis of depression predicted worse postoperative outcomes, including physical therapy (PT) compliance, return-to-sport, and patient-reported outcomes using the Patient-Reported Outcomes Measurement Information System (PROMIS) after anterior cruciate ligament (ACL) reconstruction.

METHODS

A multisurgeon series of consecutive patients who had undergone ACL reconstruction with minimum 2-year follow-up were included. Chart review was conducted to determine depression diagnosis status, demographic data, rehabilitation PT compliance, return to sports, and patient-reported outcome data using PROMIS. Patients who met the PROMIS threshold for mild depression but did not carry a clinical diagnosis of depression were classified as "situationally depressed."

RESULTS

Ninety-five of 115 consecutive patients (81%) met inclusion criteria with an average follow-up of 34 ± 1.9 months. Fourteen patients (15%) had a preoperative diagnosis of depression, whereas 21 (22%) were considered situationally depressed. Clinically depressed patients had a greater rate of PT noncompliance (33.2% ± 17.6% vs 21.9% ± 12.6%; P = .02) and a lower postoperative PROMIS Physical Function (50.8 ± 7.7 vs 57.8 ± 11.0; P = .03 compared with patients without depression. Situationally depressed patients had lower preoperative physical function (35.4 vs 42.5; P = .04) with no differences in postoperative outcomes scores compared to the non-depressed cohort.19/21 (90.5%) of situationally depressed patients had postoperative resolution of their depressive symptoms.

CONCLUSIONS

Situationally depressed patients without a clinical diagnosis of depression can expect significant improvements in both pain and function, as well as a resolution of their depressed mood based on PROMIS scores as they progress through recovery after ACL reconstruction. Clinically depressed patients also experience significant improvements; however, their rate of achieving the minimum clinically important difference for PROMIS outcomes may be less than their nondepressed or situationally depressed counterparts.

LEVEL OF EVIDENCE

III, prognostic comparative trial.

The Current Utilization of Patient-reported Outcome Measurement Information System in Shoulder, Elbow, and Sports Medicine

Clinical research using patient-reported outcome measures has been critical within the field of shoulder, elbow, and sports medicine in helping clinicians deliver evidence-based and value-based medicine. Recently, however, clinicians have advocated for improving the process of obtaining clinically meaningful information from patients while decreasing survey fatigue and increasing compliance. To that end, the National Institutes of Health created the Patient-Reported Outcome Measures Information System (PROMIS) in which a number of institutions and research investigations have adopted for reporting outcomes. A special focus has also been placed on PROMIS Computer Adaptive Testing forms, which tailor questioning through item response theory. The purpose of this study was to provide insight into the utilization, advantages, and disadvantages of PROMIS within the field of shoulder, elbow, and sports medicine and provide a comparison with legacy patient-reported outcome measure measurements.

Preoperative Patient-Reported Outcomes Measurement Information System Computerized Adaptive Testing (PROMIS CAT) Scores Predict Achievement of Minimum Clinically Important Difference Following Anterior Cruciate Ligament Reconstruction Using an Anchor-Based Methodology

PURPOSE

To determine the change in Patient-Reported Outcomes Measurement Information System Computerized Adaptive Testing (PROMIS CAT) scores for physical function, pain interference, and depression that constitute minimum clinically important difference (MCID) using an anchor-based technique and to identify pre-operative clinical thresholds in anchor-based MCID that predict likelihood of achieving MCID following anterior cruciate ligament (ACL) reconstruction.

METHODS

Adult patients aged 18 years or older undergoing ACL reconstruction that completed both preoperative and postoperative PROMIS CAT assessments and an anchor-based questionnaire were identified over a 23-month period. Anchor-based MCID was determined for PROMIS CAT forms for physical function (PROMIS PF CAT), pain interference (PROMIS PI CAT), and depression (PROMIS D CAT).

RESULTS

A total of 137 patients were included for statistical analysis, with pre-operative PROMIS CAT forms completed 27.9 ± 31.2 days before surgery and 492.5 ± 219.9 days postoperatively on average. Statistically significant improvements were observed for all PROMIS CAT domains. PROMIS PF CAT improved from 39.5 ± 8.2 to 55.0 ± 9.7 (P < .0005), PROMIS PI CAT from 59.8 ± 7.2 to 48.2 ± 8.3 (P < .0005), and PROMIS D CAT from 47.9 ± 8.8 to 41.5 ± 8.6 (P < .0005). Anchor-based MCID for each PROMIS CAT form was calculated to be +4.5, -5.4, and -4.1 for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT, respectively. Mean difference between preoperative and postoperative PROMIS CAT scores exceeded MCID for all domains. The percentage of patients achieving MCID for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT was 85%, 72%, and 55%, respectively. After introduction of 95% specificity cutoffs, the percentage of patients achieving MCID for PROMIS PF CAT, PROMIS PI CAT, and PROMIS D CAT increased to 100% (<35.6 cutoff score), 92% (>65.7 cutoff score), and 83% (>57.5 cutoff score), respectively.

CONCLUSIONS

According to anchor-based analysis of PROMIS CAT MCID, ACL reconstruction is effective in improving physical function, pain interference, and depression symptoms. In addition, preoperative PROMIS CAT scores can predict the likelihood of achieving MCID postoperatively.

LEVEL OF EVIDENCE

Level IV, prognostic case series.