Measurement properties of PROMIS short forms for pain and function in orthopedic foot and ankle surgery patients

Ankle Muscle Strength and Gait Function After Dorsal Closing Wedge Calcaneal Osteotomy for Haglund Exostosis-Related Heel Pain

BACKGROUND

Haglund exostosis-related heel pain may be surgically treated with dorsal closing wedge calcaneal osteotomy (DCWCO). Recent reports on this technique show good clinical and self-reported outcomes. However, uncertainty about functional consequences related to ankle muscle strength and gait function due to a shortened Achilles tendon lever arm exists.

METHODS

Fifteen patients (15 feet) with Haglund exostosis-related heel pain were surgically treated with DCWCO and evaluated before and 1 year after surgery. Isometric plantar flexion and dorsiflexion strength was quantified for both the involved and the uninvolved limb. Gait analysis was performed at a self-selected walking speed using a 3D motion capture system including force plates. Self-reported outcomes (Foot Function Index and Global Treatment Outcome) were also assessed.

RESULTS

Before surgery, as well as after surgery, plantar flexion strength of the involved limb was significantly lower compared to the uninvolved limb while dorsiflexion strength did not differ between limbs at both time points. Step length and time, ankle flexion angles, power generation, and propulsive impulses during gait did not differ between limbs both before and after surgery. Propulsive impulse and step length of the involved limb increased from pre- to postsurgery with an effect size of 1.04 and 0.48, respectively, revealing a general improvement in gait dynamics. Total Foot Function Index improved by 48% after surgery, and 80% of patients rated their surgery as "helped" or "helped a lot" (Global Treatment Outcome).

CONCLUSION

In this relatively small cohort, we found that patients treated for Haglund exostosis-related heel pain with DCWCO surgery had minor interlimb differences in gait kinematics and kinetics and generally improved gait dynamics and self-reported function at 1-year follow-up.

LEVEL OF EVIDENCE

Level II, observational prospective cohort study.

Pearls and pitfalls of PROMIS clinically significant outcomes in orthopaedic surgery

INTRODUCTION

Patient-Reported Outcome Measurement Information System (PROMIS) was developed as a uniform and generalizable PROM system using item response theory and computer adaptive testing. We aimed to assess the utilization of PROMIS for clinically significant outcomes (CSOs) measurements and provide insights into its use in orthopaedic research.

MATERIALS AND METHODS

We reviewed PROMIS CSO reports for orthopaedic procedures via PubMed, Cochrane Library, Embase, CINAHL, and Web of Science from inception to 2022, excluding abstracts and missing measurements. Bias was assessed using the Newcastle-Ottawa Scale (NOS) and questionnaire compliance. PROMIS domains, CSO measures, and study populations were described. A meta-analysis compared distribution and anchor-based MCIDs in low-bias (NOS ≥ 7) studies.

RESULTS

Overall, 54 publications from 2016 to 2022 were reviewed. PROMIS CSO studies were observational with increasing publication rates. Evidence-level was II in 10/54, bias low in 51/54, and compliance ≥ 86% in 46/54. Most (28/54) analysed lower extremity procedures. PROMIS domains examined Pain Function (PF) in 44/54, Pain Interference (PI) in 36/54, and Depression (D) in 18/54. Minimal clinically important difference (MCID) was reported in 51/54 and calculated based on distribution in 39/51 and anchor in 29/51. Patient acceptable symptom state (PASS), substantial clinical benefit (SCB), and minimal detectable change (MDC) were reported in ≤ 10/54. MCIDs were not significantly greater than MDCs. Anchor-based MCIDs were greater than distribution based MCIDs (standardized mean difference = 0.44, p < 0.001).

CONCLUSIONS

PROMIS CSOs are increasingly utilized, especially for lower extremity procedures assessing the PF, PI, and D domains using distribution-based MCID. Using more conservative anchor-based MCIDs and reporting MDCs may strengthen results. Researchers should consider unique pearls and pitfalls when assessing PROMIS CSOs.

PROMIS pain intensity and interference after pelvic organ prolapse surgery

INTRODUCTION AND HYPOTHESIS

Patient-Reported Outcome Measurement Information System (PROMIS) questionnaires provide valid comparisons across disciplines. Pain measures can be used to track functional outcomes. Limited PROMIS pain data exist in gynecological surgery. We sought to use pain intensity and pain interference short forms to assess pain and recovery after pelvic organ prolapse surgery.

METHODS

The PROMIS pain intensity and pain interference questionnaires were given to patients undergoing uterosacral ligament suspension (USLS), sacrospinous ligament fixation (SSLF) or minimally invasive sacrocolpopexy (MISC) at baseline, 1 week, and 6 weeks postoperatively. Clinical minimally important change was defined as a difference of 2-6 T-score points. Mean pain intensity and pain interference T-scores were compared at baseline, 1 week and 6 weeks with ANOVA. Multiple linear regression assessed 1-week scores adjusted for apical suspension type, advanced prolapse, concurrent hysterectomy, concurrent anterior or posterior repair, and concurrent sling.

RESULTS

At 1 week, all apical suspension groups showed minimally important change in pain intensity and pain interference T-scores. Between groups at 1 week, pain interference was higher in USLS (66.3±6.6) and MISC (65.5±5.9) than in SSLF (59.2±9.8), p=0.01. Multiple linear regression showed an association of hysterectomy with increases in pain intensity and pain interference. USLS had a higher proportion of concurrent hysterectomy (100%) than SSLF (0%) and MISC (30.8%), p<0.01. No difference was found based on apical suspension type alone.

CONCLUSIONS

No differences were found in PROMIS pain intensity and pain at 1 week postoperatively after apical suspension procedures.

Pre-Visit Digital Messaging Improves Patient-Reported Outcome Measure Participation Prior to the Orthopaedic Ambulatory Visit: Results from a Double-Blinded, Prospective, Randomized Controlled Trial

BACKGROUND

Patient-reported outcome measures (PROMs) are metrics that assess physical health, mental health, pain, and satisfaction. However, PROM collection in orthopaedic clinics presents numerous logistical and financial challenges. These challenges are reduced when PROMs are completed before clinic encounters, relieving the workflow constraints of in-office PROM collection. The purpose of this study was to determine the efficacy of 3 different methods with respect to pre-visit electronic PROM completion.

METHODS

Consecutive adult orthopaedic patients with no previous PROM participation were enrolled. Patients who registered with the electronic medical record (EMR) patient portal (MyChart) and with active e-mail addresses were randomly assigned to 1 of 3 arms: control (no pre-visit messages), MyChart (EMR patient portal pre-visit messages), and e-mail (e-mail pre-visit messages). The primary outcome measure was pre-visit PROM completion rates in orthopaedic patients, and the secondary outcome measures were time to pre-visit PROM form completion and PROM form completion rates according to patient demographic characteristics. By default, the Patient-Reported Outcomes Measurement Information System (PROMIS) forms were available for completion through the portal by 7 days before scheduled visits. Pre-visit messages were sent 7 days prior to the scheduled visit except in the control group, with reminders sent 3 days prior if still not completed. The patients in each arm who completed all assigned forms were labeled as having total PROM completion, and those who completed at least 1 completed form were considered as having partial PROM completion. Multivariable logistic regression models were used to assess differences in PROM completion rates between study arms. Kruskal-Wallis tests were performed to compare the date of the form completion.

RESULTS

A total of 291 patients were included. The pre-visit total completion rates for assigned PROMs were higher in the MyChart arm (49% of 97 patients; p = 0.005) and the e-mail arm (52% of 100 patients; p = 0.002) in comparison with the control arm (30% of 94 patients). Male patients were more likely than female patients to have partial pre-visit PROM completion (odds ratio [OR], 1.74; p = 0.03), and Caucasian patients were more likely to have partial pre-visit PROM completion than African American patients (OR, 2.28; p = 0.01).

CONCLUSIONS

Orthopaedic patients receiving either e-mail or patient portal messages demonstrated higher pre-visit PROM completion rates. Pre-visit messaging appears to be a useful strategy for increasing PROM completion rates and limiting the clinical workflow strain imposed by in-clinic PROM administration.

LEVEL OF EVIDENCE

Therapeutic Level I. See Instructions for Authors for a complete description of levels of evidence.

What Is the Clinical Benefit of Common Orthopaedic Procedures as Assessed by the PROMIS Versus Other Validated Outcomes Tools?

BACKGROUND

Patient-reported outcome measures (PROMs), including the Patient-reported Outcomes Measurement Information System (PROMIS), are increasingly used to measure healthcare value. The minimum clinically important difference (MCID) is a metric that helps clinicians determine whether a statistically detectable improvement in a PROM after surgical care is likely to be large enough to be important to a patient or to justify an intervention that carries risk and cost. There are two major categories of MCID calculation methods, anchor-based and distribution-based. This variability, coupled with heterogeneous surgical cohorts used for existing MCID values, limits their application to clinical care.

QUESTIONS/PURPOSES

In our study, we sought (1) to determine MCID thresholds and attainment percentages for PROMIS after common orthopaedic procedures using distribution-based methods, (2) to use anchor-based MCID values from published studies as a comparison, and (3) to compare MCID attainment percentages using PROMIS scores to other validated outcomes tools such as the Hip Disability and Osteoarthritis Outcome Score (HOOS) and Knee Disability and Osteoarthritis Outcome Score (KOOS).

METHODS

This was a retrospective study at two academic medical centers and three community hospitals. The inclusion criteria for this study were patients who were age 18 years or older and who underwent elective THA for osteoarthritis, TKA for osteoarthritis, one-level posterior lumbar fusion for lumbar spinal stenosis or spondylolisthesis, anatomic total shoulder arthroplasty or reverse total shoulder arthroplasty for glenohumeral arthritis or rotator cuff arthropathy, arthroscopic anterior cruciate ligament reconstruction, arthroscopic partial meniscectomy, or arthroscopic rotator cuff repair. This yielded 14,003 patients. Patients undergoing revision operations or surgery for nondegenerative pathologies and patients without preoperative PROMs assessments were excluded, leaving 9925 patients who completed preoperative PROMIS assessments and 9478 who completed other preoperative validated outcomes tools (HOOS, KOOS, numerical rating scale for leg pain, numerical rating scale for back pain, and QuickDASH). Approximately 66% (6529 of 9925) of patients had postoperative PROMIS scores (Physical Function, Mental Health, Pain Intensity, Pain Interference, and Upper Extremity) and were included for analysis. PROMIS scores are population normalized with a mean score of 50 ± 10, with most scores falling between 30 to 70. Approximately 74% (7007 of 9478) of patients had postoperative historical assessment scores and were included for analysis. The proportion who reached the MCID was calculated for each procedure cohort at 6 months of follow-up using distribution-based MCID methods, which included a fraction of the SD (1/2 or 1/3 SD) and minimum detectable change (MDC) using statistical significance (such as the MDC 90 from p < 0.1). Previously published anchor-based MCID thresholds from similar procedure cohorts and analogous PROMs were used to calculate the proportion reaching MCID.

RESULTS

Within a given distribution-based method, MCID thresholds for PROMIS assessments were similar across multiple procedures. The MCID threshold ranged between 3.4 and 4.5 points across all procedures using the 1/2 SD method. Except for meniscectomy (3.5 points), the anchor-based PROMIS MCID thresholds (range 4.5 to 8.1 points) were higher than the SD distribution-based MCID values (2.3 to 4.5 points). The difference in MCID thresholds based on the calculation method led to a similar trend in MCID attainment. Using THA as an example, MCID attainment using PROMIS was achieved by 76% of patients using an anchor-based threshold of 7.9 points. However, 82% of THA patients attained MCID using the MDC 95 method (6.1 points), and 88% reached MCID using the 1/2 SD method (3.9 points). Using the HOOS metric (scaled from 0 to 100), 86% of THA patients reached the anchor-based MCID threshold (17.5 points). However, 91% of THA patients attained the MCID using the MDC 90 method (12.5 points), and 93% reached MCID using the 1/2 SD method (8.4 points). In general, the proportion of patients reaching MCID was lower for PROMIS than for other validated outcomes tools; for example, with the 1/2 SD method, 72% of patients who underwent arthroscopic partial meniscectomy reached the MCID on PROMIS Physical Function compared with 86% on KOOS.

CONCLUSION

MCID calculations can provide clinical correlation for PROM scores interpretation. The PROMIS form is increasingly used because of its generalizability across diagnoses. However, we found lower proportions of MCID attainment using PROMIS scores compared with historical PROMs. By using historical proportions of attainment on common orthopaedic procedures and a spectrum of MCID calculation techniques, the PROMIS MCID benchmarks are realizable for common orthopaedic procedures. For clinical practices that routinely collect PROMIS scores in the clinical setting, these results can be used by individual surgeons to evaluate personal practice trends and by healthcare systems to quantify whether clinical care initiatives result in meaningful differences. Furthermore, these MCID thresholds can be used by researchers conducting retrospective outcomes research with PROMIS.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Distribution and Risk Factors of Multidrug-Resistant Bacteria Infection in Orthopedic Patients

Objective

To study the distribution and risk factors of multidrug-resistant bacteria (MDROS) infection in orthopedic patients and to provide reference for clinical prevention and control measures.

Methods

The data of 239 inpatients with orthopedic trauma from June 2019 to December 2020 were selected as the research objects, and the distribution characteristics of MDROS infection were analyzed through the real-time monitoring system of nosocomial infection. Logistic regression analysis was used to screen out the risk factors causing MDROS infection, and the preventive measures were put forward.

Results

178 strains of pathogens were isolated from 239 patients, including 53 strains of MDROS, and the detection rate was 29.78%. The main pathogenic bacteria were ESBLs, MRSA, CRAB, CRE, and MDR/PDRPA. The main infection sites of MDROS in orthopedic patients were the respiratory tract and wound. No CRE1 was detected, and 64.39%, 17.42%, and 14.39% of ESBL-producing bacteria, MRSA, and MDR/PDRPA, respectively, were detected. Logistic multivariate analysis showed that the length of hospital stay, antibiotic use time, open injury, and serum albumin level were independent risk factors of MDROS infection in orthopedic trauma patients.

Conclusion

To prevent MDROS infection in orthopedic patients, we should start from many aspects, focusing on reducing unnecessary hospitalization days, rationally preventing the use of antibacterial drugs, effectively treating basic diseases, etc., timely and effective thorough debridement, strengthening functional training, reducing bed rest, and strengthening targeted monitoring of related infections which are the keys to reduce MDROS infection in orthopedic patients.

A Patient-Reported Outcomes Measurement Information System Short Form for Measuring Physical Function During Geriatric Rehabilitation: Test-Retest Reliability, Construct Validity, Responsiveness, and Interpretability

OBJECTIVES

To study the test-retest reliability and measurement error, construct validity, responsiveness, interpretability, and floor/ceiling effects of a Patient-Reported Outcomes Measurement Information System (PROMIS) short form designed to measure physical function in geriatric rehabilitation patients (PROMIS-PF-GR).

DESIGN

Prospective cohort.

SETTING AND PARTICIPANTS

Inpatient geriatric rehabilitation patients.

METHODS

We evaluated the test-retest reliability by re-administering PROMIS-PF-GR 3 to 5 days after the admission measurement. The intraclass correlation coefficient (ICC) was calculated to determine test-retest reliability; an ICC of ≥0.70 was considered sufficient. Measurement error was established by calculating the standard error of measurement and smallest detectable change. Construct validity and responsiveness were determined by testing a priori formulated hypotheses (criterion: ≥75% hypothesis not rejected). Interpretability was evaluated by calculating the minimal important change using predictive modeling and a global rating as criterion for change. Floor/ceiling effects were established by calculating the percentage patients with the minimum/maximum raw score (criterion: ≤15%) at admission and discharge.

RESULTS

A total of 207 patients participated in the study [mean ± standard deviation age (80 ± 8.3 years), 58% female]. More than one-half of patients (56%) reported to be improved during rehabilitation. The ICC was 0.79 (95% confidence interval 0.70-0.84), the standardized error of measurement was 3.8, and the smallest detectable change 10.6. None of the 4 hypotheses for construct validity were rejected; 2 out of 5 hypotheses for responsiveness were rejected. The minimal important change was 8.0 (95% confidence interval 4.1-12.5). No floor/ceiling effects were found.

CONCLUSIONS AND IMPLICATIONS

The PROMIS-PG-GR showed sufficient test-retest reliability, measurement error, and construct validity. We did not find sufficient evidence for responsiveness, which may be due to the unexplained weak correlation between the PROMIS change score and the Global Rating Scale. We still recommend the use the PROMIS-PG-GR for measuring self-reported physical function in geriatric rehabilitation.

Association Between Baseline PROMIS Scores, Patient-Provider Communication Factors, and Musculoskeletal Health Literacy on Patient and Surgeon Expectations in Foot and Ankle Surgery

BACKGROUND

Various factors may affect differences between patient and surgeon expectations. This study aimed to assess associations between patient-reported physical and mental status, patient-surgeon communication, and musculoskeletal health literacy with differences in patient and surgeon expectations of foot and ankle surgery.

METHODS

Two hundred two patients scheduled to undergo foot or ankle surgery at an academic hospital were enrolled. Preoperatively, patients and surgeons completed the Hospital for Special Surgery Foot & Ankle Surgery Expectations Survey. Patients also completed Patient-Reported Outcomes Measurement Information System (PROMIS) scores in Physical Function, Pain Interference, Pain Intensity, Depression, and Global Health. Patient-surgeon communication and musculoskeletal health literacy were assessed via the modified Patients' Perceived Involvement in Care Scale (PICS) and Literacy in Musculoskeletal Problems (LiMP) questionnaire, respectively.

RESULTS

Greater differences in patient and surgeon overall expectations scores were associated with worse scores in Physical Function (P = .003), Pain Interference (P = .001), Pain Intensity (P = .009), Global Physical Health (P < .001), and Depression (P = .009). A greater difference in the number of expectations between patients and surgeons was associated with all of the above (P ≤ .003) and with worse Global Mental Health (P = .003). Patient perceptions of higher surgeons' partnership building were associated with a greater number of patient than surgeon expectations (P = .017). There were no associations found between musculoskeletal health literacy and differences in expectations.

CONCLUSION

Worse baseline patient physical and mental status and higher patient perceptions of provider partnership building were associated with higher patient than surgeon expectations. It may be beneficial for surgeons to set more realistic expectations with patients who have greater disability and in those whom they have stronger partnerships with. Further studies are warranted to understand how modifications in patient and surgeon interactions and patient health literacy affect agreement in expectations of foot and ankle surgery.

LEVEL OF EVIDENCE

Level II, prospective comparative series.

Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures

We define the minimal important change (MIC) as a threshold for a minimal within-person change over time above which patients perceive themselves importantly changed. There is a lot of confusion about the concept of MIC, particularly about the concepts of minimal important change and minimal detectable change, which questions the validity of published MIC values. The aims of this study were: (1) to clarify the concept of MIC and how to use it; (2) to provide practical guidance for estimating methodologically sound MIC values; and (3) to improve the applicability of PROMIS by summarizing the available evidence on plausible PROMIS MIC values. We discuss the concept of MIC and how to use it and provide practical guidance for estimating MIC values. In addition, we performed a systematic review in PubMed on MIC values of any PROMIS measure from studies using recommended approaches. A total of 50 studies estimated the MIC of a PROMIS measure, of which 19 studies used less appropriate methods. MIC values of the remaining 31 studies ranged from 0.1 to 12.7 T-score points. We recommend to use the predictive modeling method, possibly supplemented with the vignette-based method, in future MIC studies. We consider a MIC value of 2-6 T-score points for PROMIS measures reasonable to assume at this point. For surgical interventions a higher MIC value might be appropriate. We recommend more high-quality studies estimating MIC values for PROMIS.

Patient-Reported Outcomes of Robotic vs Laparoscopic Ventral Hernia Repair With Intraperitoneal Mesh: The PROVE-IT Randomized Clinical Trial

Importance

Despite rapid adoption of the robotic platform for ventral hernia repair with intraperitoneal mesh in the United States, there is no level I evidence comparing it with the traditional laparoscopic approach. This randomized clinical trial sought to demonstrate a clinical benefit to the robotic approach.

Objective

To determine whether robotic approach to ventral hernia repair with intraperitoneal mesh would result in less postoperative pain.

Design, Setting, and Participants

A registry-based, single-blinded, prospective randomized clinical trial at the Cleveland Clinic Center for Abdominal Core Health, Cleveland, Ohio, completed between September 2017 and January 2020, with a minimum follow-up duration of 30 days. Two surgeons at 1 academic tertiary care hospital. Patients with primary or incisional midline ventral hernias of an anticipated width of 7 cm or less presenting in the elective setting and able to tolerate a minimally invasive repair.

Interventions

Patients were randomized to a standardized laparoscopic or robotic ventral hernia repair with fascial closure and intraperitoneal mesh.

Main Outcomes and Measures

The trial was powered to detect a 30% difference in the Numerical Rating Scale (NRS-11) on the first postoperative day. Secondary end points included the Patient-Reported Outcomes Measurement Information System Pain Intensity short form (3a), hernia-specific quality of life, operative time, wound morbidity, recurrence, length of stay, and cost.

Results

Seventy-five patients completed their minimally invasive hernia repair: 36 laparoscopic and 39 robotic. Baseline demographics and hernia characteristics were comparable. Robotic operations had a longer median operative time (146 vs 94 minutes; P < .001). There were 2 visceral injuries in each cohort but no full-thickness enterotomies or unplanned reoperations. There were no significant differences in NRS-11 scores preoperatively or on postoperative days 0, 1, 7, or 30. Specifically, median NRS-11 scores on the first postoperative day were the same (5 vs 5; P = .61). Likewise, postoperative Patient-Reported Outcomes Measurement Information System 3a and hernia-specific quality-of-life scores, as well as length of stay and complication rates, were similar. The robotic platform adds cost (total cost ratio, 1.13 vs 0.97; P = .03), driven by the cost of additional operating room time (1.25 vs 0.85; P < .001).

Conclusions and Relevance

Laparoscopic and robotic ventral hernia repair with intraperitoneal mesh have comparable outcomes. The increased operative time and proportional cost of the robotic approach are not offset by a measurable clinical benefit.

Trial Registration

ClinicalTrials.gov Identifier: NCT03283982.

Athlete-Reported Outcome Measures for Monitoring Training Responses: A Systematic Review of Risk of Bias and Measurement Property Quality According to the COSMIN Guidelines

BACKGROUND

Athlete-reported outcome measures (AROMs) are frequently used in research and practice but no studies have examined their psychometric properties.

OBJECTIVES

Part 1-identify the most commonly used AROMs in sport for monitoring training responses; part 2-assess risk of bias, measurement properties, and level of evidence, based on the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines.

STUDY APPRAISAL AND SYNTHESIS METHODS

Methodological quality of the studies, quality of measurement properties, and level of evidence were determined using the COSMIN checklist and criteria.

RESULTS

Part 1-from 9446 articles screened for title and abstract, 310 out of 334 full texts were included; 53.9% of the AROMs contained multiple items, while 46.1% contained single items. Part 2-from 1895 articles screened for title and abstract, 71 were selected. Most measurement properties of multiple-item AROMs were adequate, but content validity and measurement error were inadequate. With the exclusion of 2 studies examining reliability and responsiveness, no validity studies were found for single items.

CONCLUSIONS

The measurement properties of multiple-item AROMs derived from psychometrics were acceptable (with the exclusion of content validity and measurement error). The single-item AROMs most frequently used in sport science have not been validated. Additionally, nonvalidated modified versions of the originally nonvalidated items are common. Until proper validation studies are completed, all conclusions based on these AROMs are questionable. Established reference methods, such as those of clinimetrics, should be used to develop and assess the validity of AROMs.