Anchored Minimal Clinically Important Difference Metrics: Considerations for Bias and Regression to the Mean

Calculation of the minimum clinically important difference (MCID) using different methodologies: case study and practical guide

INTRODUCTION

Establishing thresholds of change that are actually meaningful for the patient in an outcome measurement instrument is paramount. This concept is called the minimum clinically important difference (MCID). We summarize available MCID calculation methods relevant to spine surgery, and outline key considerations, followed by a step-by-step working example of how MCID can be calculated, using publicly available data, to enable the readers to follow the calculations themselves.

METHODS

Thirteen MCID calculations methods were summarized, including anchor-based methods, distribution-based methods, Reliable Change Index, 30% Reduction from Baseline, Social Comparison Approach and the Delphi method. All methods, except the latter two, were used to calculate MCID for improvement of Zurich Claudication Questionnaire (ZCQ) Symptom Severity of patients with lumbar spinal stenosis. Numeric Rating Scale for Leg Pain and Japanese Orthopaedic Association Back Pain Evaluation Questionnaire Walking Ability domain were used as anchors.

RESULTS

The MCID for improvement of ZCQ Symptom Severity ranged from 0.8 to 5.1. On average, distribution-based methods yielded lower MCID values, than anchor-based methods. The percentage of patients who achieved the calculated MCID threshold ranged from 9.5% to 61.9%.

CONCLUSIONS

MCID calculations are encouraged in spinal research to evaluate treatment success. Anchor-based methods, relying on scales assessing patient preferences, continue to be the "gold-standard" with receiver operating characteristic curve approach being optimal. In their absence, the minimum detectable change approach is acceptable. The provided explanation and step-by-step example of MCID calculations with statistical code and publicly available data can act as guidance in planning future MCID calculation studies.

Minimal important clinical difference values are not uniformly valid in the active duty military population recovering from shoulder surgery

BACKGROUND

There are multiple methods for calculating the minimal clinically important difference (MCID) threshold, and previous reports highlight heterogeneity and limitations of anchor-based and distribution-based analyses. The Warfighter Readiness Survey assesses the perception of a military population's fitness to deploy and may be used as a functional index in anchor-based MCID calculations. The purpose of the current study in a physically demanding population undergoing shoulder surgery was to compare the yields of 2 different anchor-based methods of calculating MCID for a battery of PROMs, a standard receiver operating characteristic (ROC) curve-based MCIDs and baseline-adjusted ROC curve MCIDs.

METHODS

All service members enrolled prospectively in a multicenter database with prior shoulder surgery that completed pre- and postoperative PROMs at a minimum of 12 months were included. The PROM battery included Single Assessment Numeric Evaluation (SANE), American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASES), Patient Reported Outcome Management Information System (PROMIS) physical function (PF), PROMIS pain interference (PI), and the Warfighter Readiness Survey. Standard anchor-based and baseline-adjusted ROC curve MCIDs were employed to determine if the calculated MCIDs were both statistically and theoretically valid (95% confidence interval [CI] either completely negative or positive).

RESULTS

A total of 117 patients (136 operations) were identified, comprising 83% males with a mean age of 35.7 ± 10.4 years and 47% arthroscopic labral repair/capsulorrhaphy. Using the standard, anchor-based ROC curve MCID calculation, the area under the curve (AUC) for SANE, ASES, PROMIS PF, and PROMIS PI were greater than 0.5 (statistically valid). For ASES, PROMIS PF, and PROMIS PI, the calculated MCID 95% CI all crossed 0 (theoretically invalid). Using the baseline-adjusted ROC curve MCID calculation, the MCID estimates for SANE, ASES, and PROMIS PI were both statistically and theoretically valid if the baseline score was less than 70.5, 69, and 65.7.

CONCLUSION

When MCIDs were calculated and anchored to the results of standard, anchor-based MCID, a standard ROC curve analysis did not yield statistically or theoretically valid results across a battery of PROMs commonly used to assess outcomes after shoulder surgery in the active duty military population. Conversely, a baseline-adjusted ROC curve method was more effective at discerning changes across a battery of PROMs among the same cohort.

Editorial Commentary: The Minimal Clinically Important Difference Is Less Important Than It Sounds: Patients Seek to Achieve Substantial Clinical Benefits and Not Minimally Perceptible Improvements When They Undergo Arthroscopic Surgery

The minimal clinically important difference (MCID) is a frequently reported metric for describing within-patient improvement in patient-reported outcome measures (PROMs). It was originally defined by Jaeschke et al. as "the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient's management." The latter part of this statement is often omitted, and this results in a loss of the originally intended value through lack of sufficient clinical importance to change management. Other pitfalls in the use of the MCID include that they are population- and condition-specific. As such, MCIDs lack external validity and cannot easily be extrapolated from one study to another. Furthermore, broadly different values can be obtained depending on the calculation method used. This makes the MCID an unhelpful metric when seeking to understand the true efficacy of a given intervention. The Food and Drug Administration recommends anchor-based methodologies (which take into account patient perception), over distribution-based methods (which are purely statistical and do not account for clinical meaningfulness to patients). Regardless, it should be noted that even anchor-based methodologies are susceptible to statistical bias, and measures are apt to be influenced by the regression to mean phenomena, where the value of the preintervention scores and their relationship to postintervention scores can bias estimates of the MCID. Finally, when using MCIDs, one must consider that they are a low bar. This means that patients do not undergo treatment to achieve minimally perceptible clinical improvements; instead, they undergo treatment with the hope of achieving substantial clinical benefit or a patient acceptable symptom state, and so these are more appropriate individual-level metrics to consider when evaluating clinically meaningful outcomes of treatment.

Patient-reported outcome measures as an outcome variable in sports medicine research

Injury prevention and rehabilitation research often address variables that would be considered clinician-oriented outcomes, such as strength, range of motion, laxity, and return-to-sport. While clinician-oriented variables are helpful in describing the physiological recovery from injury, they neglect the patient perspective and aspects of patient-centered care. Variables that capture patient perspective are essential when considering the impact of injury and recovery on the lives of patients. The inclusion of patient-reported outcome measures (PROMs) as dependent variables in sports medicine research, including injury prevention and rehabilitation research, provides a unique perspective regarding the patient's perception of their health status, the effectiveness of treatments, and other information that the patient deems important to their care. Over the last 20 years, there has been a significant increase in the use of PROMs in sports medicine research. The growing body of work gives opportunity to reflect on what has been done and to provide some ideas of how to strengthen the evidence moving forward. This mini-review will discuss ideas for the inclusion of PROMs in sports medicine research, with a focus on critical factors, gaps, and future directions in this area of research. Important elements of research with PROMs, including instrument selection, administration, and interpretation, will be discussed and areas for improvement, consideration, and standardization will be provided.

Interindividual Differences in Trainability and Moderators of Cardiorespiratory Fitness, Waist Circumference, and Body Mass Responses: A Large-Scale Individual Participant Data Meta-analysis

Although many studies have assumed variability reflects variance caused by exercise training, few studies have examined whether interindividual differences in trainability are present following exercise training. The present individual participant data (IPD) meta-analysis sought to: (1) investigate the presence of interindividual differences in trainability for cardiorespiratory fitness (CRF), waist circumference, and body mass; and (2) examine the influence of exercise training and potential moderators on the probability that an individual will experience clinically important differences. The IPD meta-analysis combined data from 1879 participants from eight previously published randomized controlled trials. We implemented a Bayesian framework to: (1) test the hypothesis of interindividual differences in trainability by comparing variability in change scores between exercise and control using Bayes factors; and (2) compare posterior predictions of control and exercise across a range of moderators (baseline body mass index (BMI) and exercise duration, intensity, amount, mode, and adherence) to estimate the proportions of participants expected to exceed minimum clinically important differences (MCIDs) for all three outcomes. Bayes factors demonstrated a lack of evidence supporting a high degree of variance attributable to interindividual differences in trainability across all three outcomes. These findings indicate that interindividual variability in observed changes are likely due to measurement error and external behavioural factors, not interindividual differences in trainability. Additionally, we found that a larger proportion of exercise participants were expected to exceed MCIDs compared with controls for all three outcomes. Moderator analyses identified that larger proportions were associated with a range of factors consistent with standard exercise theory and were driven by mean changes. Practitioners should prescribe exercise interventions known to elicit large mean changes to increase the probability that individuals will experience beneficial changes in CRF, waist circumference and body mass.

A High-Sensitivity International Knee Documentation Committee Survey Index From the PROMIS System: The Next-Generation Patient-Reported Outcome for a Knee Injury Population

BACKGROUND

Patient-reported outcomes (PROs) measure progression and quality of care. While legacy PROs such as the International Knee Documentation Committee (IKDC) survey are well-validated, a lengthy PRO creates a time burden on patients, decreasing adherence. In recent years, PROs such as the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function and Pain Interference surveys were developed as computer adaptive tests, reducing time to completion. Previous studies have examined correlation between legacy PROs and PROMIS; however, no studies have developed effective prediction models utilizing PROMIS to create an IKDC index. While the IKDC is the standard knee PRO, computer adaptive PROs offer numerous practical advantages.

PURPOSE

To develop a nonlinear predictive model utilizing PROMIS Physical Function and Pain Interference to estimate IKDC survey scores and examine algorithm sensitivity and validity.

STUDY DESIGN

Cohort study (diagnosis); Level of evidence, 3.

METHODS

The MOTION (Military Orthopaedics Tracking Injuries and Outcomes Network) database is a prospectively collected repository of PROs and intraoperative variables. Patients undergoing knee surgery completed the IKDC and PROMIS surveys at varying time points. Nonlinear multivariable predictive models using Gaussian and beta distributions were created to establish an IKDC index score, which was then validated using leave-one-out techniques and minimal clinically important difference analysis.

RESULTS

A total of 1011 patients completed the IKDC and PROMIS Physical Function and Pain Interference, providing 1618 complete observations. The algorithms for the Gaussian and beta distribution were validated to predict the IKDC (Pearson = 0.84-0.86; R² = 0.71-0.74; root mean square error = 9.3-10.0).

CONCLUSION

The publicly available predictive models can approximate the IKDC score. The results can be used to compare PROMIS Physical Function and Pain Interference against historical IKDC scores by creating an IKDC index score. Serial use of the IKDC index allows for a lower minimal clinically important difference than the conventional IKDC. PROMIS can be substituted to reduce patient burden, increase completion rates, and produce orthopaedic-specific survey analogs.