Do Proposed Quality Measures for Carpal Tunnel Release Reveal Important Quality Gaps and Are They Reliable?

Split-sample reliability estimation in health care quality measurement: Once is not enough

OBJECTIVE

To examine the sensitivity of split-sample reliability estimates to the random split of the data and propose alternative methods for improving the stability of the split-sample method.

DATA SOURCES AND STUDY SETTING

Data were simulated to reflect a variety of real-world quality measure distributions and scenarios. There is no date range to report as the data are simulated.

STUDY DESIGN

Simulation studies of split-sample reliability estimation were conducted under varying practical scenarios.

DATA COLLECTION/EXTRACTION METHODS

All data were simulated using functions in R.

PRINCIPAL FINDINGS

Single split-sample reliability estimates can be very dependent on the random split of the data, especially in low sample size and low variability settings. Averaging split-sample estimates over many splits of the data can yield a more stable reliability estimate.

CONCLUSIONS

Measure developers and evaluators using the split-sample reliability method should average a series of reliability estimates calculated from many resamples of the data without replacement to obtain a more stable reliability estimate.

Has the Use of Electrodiagnostic Studies for Carpal Tunnel Syndrome Changed After the 2016 American Academy of Orthopaedic Surgeons Clinical Practice Guideline?

PURPOSE

A 2016 American Academy of Orthopaedic Surgeons (AAOS) clinical practice guideline (CPG) de-emphasized the need for electrodiagnostic studies (EDS) for carpal tunnel syndrome (CTS). We tested the hypothesis that use of EDS decreased after the AAOS CPG.

METHODS

Using a national administrative claims database, we measured the proportion of patients with a diagnosis of CTS who underwent EDS within 1 year after diagnosis between 2011 and 2019. Using an interrupted time series design, we defined 2 time periods (pre-CPG and post-CPG) and compared EDS usage between the periods using segmented regression analysis. We conducted a subgroup analysis of preoperative EDS usage in patients who underwent carpal tunnel release.

RESULTS

Of 2,081,829 patients with CTS, 315,449 (15.2%) underwent EDS within 1 year after diagnosis. The segmented regression analysis showed a decrease in the level of EDS usage after publication of the AAOS CPG (-11.50 per 1,000 patients [95% CI, -1.47 to -0.95 per 1,000 patients]); however, the rate of EDS usage increased in the post-CPG period (+1.75 per 1,000 patients per quarter [95% CI, 0.97-2.54 per 1,000 patients per quarter]). Of 473,753 eligible patients who underwent carpal tunnel release, 139,186 (29.4%) underwent EDS within 6 months before surgery. After publication of the AAOS CPG, preoperative EDS usage decreased by -23.57 per 1,000 patients (95% CI, -37.72 to -9.42 per 1,000 patients). However, these decreasing trends in EDS usage predated the 2016 AAOS CPG.

CONCLUSIONS

The overall and preoperative EDS usage for CTS has been decreasing since at least 2014, predating the 2016 AAOS CPG, reflecting the rapid implementation of evidence into practice. However, EDS usage has increased in the post-CPG period, and a considerable proportion of patients who underwent carpal tunnel release still received EDS.

CLINICAL RELEVANCE

Given its high costs and disputed value, routine EDS usage should be considered for further deimplementation initiatives.