Surgeon-level variance in achieving clinical improvement after lumbar decompression: the importance of adequate risk adjustment

Intersurgeon Variability of Minimal Clinically Important Difference for Worsening Achievement Rates After Total Joint Arthroplasty

BACKGROUND

Using the Patient-Reported Outcome Measurement Information System, we sought to evaluate surgeon performance variability via minimal clinically important difference for worsening (MCID-W) achievement rates in primary and revision total knee and hip arthroplasty.

METHODS

This retrospective study analyzed 3,496 primary total hip arthroplasty (THA), 4,622 primary total knee arthroplasty (TKA), 592 revision THA, and 569 revision TKA patients. Patient factors collected included demographics, comorbidities, and Patient-Reported Outcome Measurement Information System physical function short form 10a scores. Surgeon factors collected included caseload, years of experience, and fellowship training. The MCID-W rate was calculated as the percent of patients in each surgeon's cohort who achieved MCID-W. Distribution was presented via a histogram with associated average, standard deviation, range, and interquartile range (IQR). Linear regressions were performed to evaluate the potential correlation between surgeon- and patient-level factors with MCID-W rate.

RESULTS

The average MCID-W rates of the surgeons represented in the primary THA and TKA cohorts were 12.7 ± 9.2% (range, 0 to 35.3%; IQR, 6.7 to 15.5%) and 18.0 ± 8.2% (range, 0 to 36%; IQR, 14.3 to 22.0%). The average MCID-W rates among the revision THA and TKA surgeons were 36.0 ± 22.2% (range, 9.1 to 90%; IQR, 25.0 to 41.4%) and 21.2 ± 7.7% (range, 8.1 to 37.0%; IQR, 16.6 to 25.4%). Strong correlations were not found between patient- or surgeon-level factors and MCID-W rate of the surgeon.

CONCLUSION

We demonstrated variance in MCID-W achievement rates across surgeons in both primary and revision joint arthroplasty, independent of patient- or surgeon-level factors.

Benchmarking short-term postoperative mortality across neurosurgery units: is hospital administrative data good enough for risk-adjustment?

BACKGROUND

Surgical mortality indicators should be risk-adjusted when evaluating the performance of organisations. This study evaluated the performance of risk-adjustment models that used English hospital administrative data for 30-day mortality after neurosurgery.

METHODS

This retrospective cohort study used Hospital Episode Statistics (HES) data from 1 April 2013 to 31 March 2018. Organisational-level 30-day mortality was calculated for selected subspecialties (neuro-oncology, neurovascular and trauma neurosurgery) and the overall cohort. Risk adjustment models were developed using multivariable logistic regression and incorporated various patient variables: age, sex, admission method, social deprivation, comorbidity and frailty indices. Performance was assessed in terms of discrimination and calibration.

RESULTS

The cohort included 49,044 patients. Overall, 30-day mortality rate was 4.9%, with unadjusted organisational rates ranging from 3.2 to 9.3%. The variables in the best performing models varied for the subspecialties; for trauma neurosurgery, a model that included deprivation and frailty had the best calibration, while for neuro-oncology a model with these variables plus comorbidity performed best. For neurovascular surgery, a simple model of age, sex and admission method performed best. Levels of discrimination varied for the subspecialties (range: 0.583 for trauma and 0.740 for neurovascular). The models were generally well calibrated. Application of the models to the organisation figures produced an average (median) absolute change in mortality of 0.33% (interquartile range (IQR) 0.15-0.72) for the overall cohort model. Median changes for the subspecialty models were 0.29% (neuro-oncology, IQR 0.15-0.42), 0.40% (neurovascular, IQR 0.24-0.78) and 0.49% (trauma neurosurgery, IQR 0.23-1.68).

CONCLUSIONS

Reasonable risk-adjustment models for 30-day mortality after neurosurgery procedures were possible using variables from HES, although the models for trauma neurosurgery performed less well. Including a measure of frailty often improved model performance.

What Is the Clinical Benefit of Common Orthopaedic Procedures as Assessed by the PROMIS Versus Other Validated Outcomes Tools?

BACKGROUND

Patient-reported outcome measures (PROMs), including the Patient-reported Outcomes Measurement Information System (PROMIS), are increasingly used to measure healthcare value. The minimum clinically important difference (MCID) is a metric that helps clinicians determine whether a statistically detectable improvement in a PROM after surgical care is likely to be large enough to be important to a patient or to justify an intervention that carries risk and cost. There are two major categories of MCID calculation methods, anchor-based and distribution-based. This variability, coupled with heterogeneous surgical cohorts used for existing MCID values, limits their application to clinical care.

QUESTIONS/PURPOSES

In our study, we sought (1) to determine MCID thresholds and attainment percentages for PROMIS after common orthopaedic procedures using distribution-based methods, (2) to use anchor-based MCID values from published studies as a comparison, and (3) to compare MCID attainment percentages using PROMIS scores to other validated outcomes tools such as the Hip Disability and Osteoarthritis Outcome Score (HOOS) and Knee Disability and Osteoarthritis Outcome Score (KOOS).

METHODS

This was a retrospective study at two academic medical centers and three community hospitals. The inclusion criteria for this study were patients who were age 18 years or older and who underwent elective THA for osteoarthritis, TKA for osteoarthritis, one-level posterior lumbar fusion for lumbar spinal stenosis or spondylolisthesis, anatomic total shoulder arthroplasty or reverse total shoulder arthroplasty for glenohumeral arthritis or rotator cuff arthropathy, arthroscopic anterior cruciate ligament reconstruction, arthroscopic partial meniscectomy, or arthroscopic rotator cuff repair. This yielded 14,003 patients. Patients undergoing revision operations or surgery for nondegenerative pathologies and patients without preoperative PROMs assessments were excluded, leaving 9925 patients who completed preoperative PROMIS assessments and 9478 who completed other preoperative validated outcomes tools (HOOS, KOOS, numerical rating scale for leg pain, numerical rating scale for back pain, and QuickDASH). Approximately 66% (6529 of 9925) of patients had postoperative PROMIS scores (Physical Function, Mental Health, Pain Intensity, Pain Interference, and Upper Extremity) and were included for analysis. PROMIS scores are population normalized with a mean score of 50 ± 10, with most scores falling between 30 to 70. Approximately 74% (7007 of 9478) of patients had postoperative historical assessment scores and were included for analysis. The proportion who reached the MCID was calculated for each procedure cohort at 6 months of follow-up using distribution-based MCID methods, which included a fraction of the SD (1/2 or 1/3 SD) and minimum detectable change (MDC) using statistical significance (such as the MDC 90 from p < 0.1). Previously published anchor-based MCID thresholds from similar procedure cohorts and analogous PROMs were used to calculate the proportion reaching MCID.

RESULTS

Within a given distribution-based method, MCID thresholds for PROMIS assessments were similar across multiple procedures. The MCID threshold ranged between 3.4 and 4.5 points across all procedures using the 1/2 SD method. Except for meniscectomy (3.5 points), the anchor-based PROMIS MCID thresholds (range 4.5 to 8.1 points) were higher than the SD distribution-based MCID values (2.3 to 4.5 points). The difference in MCID thresholds based on the calculation method led to a similar trend in MCID attainment. Using THA as an example, MCID attainment using PROMIS was achieved by 76% of patients using an anchor-based threshold of 7.9 points. However, 82% of THA patients attained MCID using the MDC 95 method (6.1 points), and 88% reached MCID using the 1/2 SD method (3.9 points). Using the HOOS metric (scaled from 0 to 100), 86% of THA patients reached the anchor-based MCID threshold (17.5 points). However, 91% of THA patients attained the MCID using the MDC 90 method (12.5 points), and 93% reached MCID using the 1/2 SD method (8.4 points). In general, the proportion of patients reaching MCID was lower for PROMIS than for other validated outcomes tools; for example, with the 1/2 SD method, 72% of patients who underwent arthroscopic partial meniscectomy reached the MCID on PROMIS Physical Function compared with 86% on KOOS.

CONCLUSION

MCID calculations can provide clinical correlation for PROM scores interpretation. The PROMIS form is increasingly used because of its generalizability across diagnoses. However, we found lower proportions of MCID attainment using PROMIS scores compared with historical PROMs. By using historical proportions of attainment on common orthopaedic procedures and a spectrum of MCID calculation techniques, the PROMIS MCID benchmarks are realizable for common orthopaedic procedures. For clinical practices that routinely collect PROMIS scores in the clinical setting, these results can be used by individual surgeons to evaluate personal practice trends and by healthcare systems to quantify whether clinical care initiatives result in meaningful differences. Furthermore, these MCID thresholds can be used by researchers conducting retrospective outcomes research with PROMIS.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Minimum Clinically Important Difference: A Metric That Matters in the Age of Patient-Reported Outcomes

➤

As the Patient-Reported Outcomes Measurement Information System (PROMIS) is increasingly utilized in orthopaedic research and clinical practice, there is not a consensus regarding the minimum clinically important difference (MCID) calculation.

➤

The varied MCID calculation methods can lead to a range of possible values, which limits the translatability of research efforts.

➤

The completion rate and follow-up period also influence MCID values and should be reported alongside study results.

Digital Disparities: Lessons learned from a Patient Reported Outcomes Program During the COVID-19 Pandemic

The collection of patient reported outcomes (PROs) allows us to incorporate the patient's voice within their care in a quantifiable, validated manner. Large scale collection of PROs is facilitated by the electronic health record (EHR) and its portal, though historically patients have eschewed the portal and completed patient reported outcome measures (PROMS) in clinic via tablet. Furthermore, access to and use of the portal is associated with known racial inequities. Our institution oversees the largest clinical PRO program in the world, and has a long history of racially equitable PRO completion rates via tablet. However, when the COVID-19 pandemic forced us to remove tablets from clinics and rely exclusively on portal use for PRO completion, profound racial disparities resulted immediately. Our experience quantifiably demonstrates the magnitude of inequity that the portal, in its current configuration, generates and serves as a cautionary tale to other health care systems and EHRs.

Physician-Related Variability in the Outcomes of an Invasive Treatment for Neck and Back Pain: A Multi-Level Analysis of Data Gathered in Routine Clinical Practice

Neuro-reflexotherapy (NRT) is a proven effective, invasive treatment for neck and back pain. To assess physician-related variability in results, data from post-implementation surveillance of 9023 patients treated within the Spanish National Health Service by 12 physicians were analyzed. Separate multi-level logistic regression models were developed for spinal pain (SP), referred pain (RP), and disability. The models included all patient-related variables predicting response to NRT and physician-related variables. The Intraclass Correlation Coefficient (ICC) and the Median Odds Ratio (MOR) were calculated. Adjusted MOR (95% CI) was 1.70 (1.47; 2.09) for SP, 1.60 (1.38; 1.99) for RP, and 1.65 (1.42; 2.03) for disability. Adjusted ICC (95%CI) values were 0.08 (0.05; 0.15) for SP, 0.07 (0.03; 0.14) for RP, and 0.08 (0.04; 0.14) for disability. In the sensitivity analysis, in which the 6920 patients treated during the physicians’ training period were excluded, adjusted MOR was 1.38 (1.17; 1.98) for SP, 1.37 (1.12; 2.31) for RP, and 1.25 (1.09; 1.79) for disability, while ICCs were 0.03 (0.01; 0.14) for SP, 0.03 (0.00; 0.19) for RP, and 0.02 (0.00; 0.10) for disability. In conclusion, the variability in results obtained by different NRT-certified specialists is reasonable. This suggests that current training standards are appropriate.