Present state bias in transition ratings was accurately estimated in simulated and real data

Minimal important change thresholds change over time after knee and hip arthroplasty

OBJECTIVES

The minimal important change (MIC) reflects what patients, on average, consider the smallest improvement in a score that is important to them. MIC thresholds may vary across patient populations, interventions used, posttreatment time points and derivation methods. We determine and compare MIC thresholds for the Oxford Knee Score and Oxford Hip Score (OKS/OHS) at 3 months postoperatively to 12- and 24-month thresholds in patients undergoing knee or hip arthroplasty.

STUDY DESIGN AND SETTING

This cohort study used data from patients undergoing total knee arthroplasty (TKA), unicompartmental knee arthroplasty (UKA), or total hip arthroplasty (THA) at a public hospital between February 2016 and February 2023. At 3, 12, and 24 months postoperatively, patients responded to the OKS/OHS and a 7-point anchor question determining experienced changes in knee or hip pain and functional limitations. We used the adjusted predictive modeling method that accounts for the proportion improved and the reliability of the anchor question to determine MIC thresholds and their mean differences between time points.

RESULTS

Complete data were obtained from 695/957 (73%), 1179/1703 (69%), and 1080/1607 (67%) patients undergoing TKA, 474/610 (78%), 438/603 (73%), and 355/507 (70%) patients undergoing UKA, and 965/1315 (73%), 978/1409 (69%), and 1059/1536 (69%) patients undergoing THA at 3, 12, and 24 months, respectively. The median age ranged from 68 to 70 years and 55% to 60% were females. The proportions improved ranged between 83% and 95%. The OKS/OHS MIC thresholds were 0.1, 4.2, and 5.1 for TKA, 1.8, 5.6, and 3.4 for UKA, and 1.3, 6.1, and 6.0 for THA at 3, 12, and 24 months postoperatively, respectively. The reliability ranged between 0.64 and 0.82, and the MIC values increased between three and 12 months but not between 12 and 24 months.

CONCLUSION

Any absence of deterioration in pain and function is considered important at 3 months after knee or hip arthroplasty. Increasing thresholds over time suggest patients raise their standards for what constitutes a minimal important improvement over the first postoperative year. Besides improving our understanding of patients' views on postoperative outcomes, these clinical thresholds may aid in interpreting registry-based treatment outcome evaluations.

Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis

PURPOSE

The minimal important change (MIC) is defined as the smallest within-individual change in a patient-reported outcome measure (PROM) that patients on average perceive as important. We describe a method to estimate this value based on longitudinal confirmatory factor analysis (LCFA). The method is evaluated and compared with a recently published method based on longitudinal item response theory (LIRT) in simulated and real data. We also examined the effect of sample size on bias and precision of the estimate.

METHODS

We simulated 108 samples with various characteristics in which the true MIC was simulated as the mean of individual MICs, and estimated MICs based on LCFA and LIRT. Additionally, both MICs were estimated in existing PROMIS Pain Behavior data from 909 patients. In another set of 3888 simulated samples with sample sizes of 125, 250, 500, and 1000, we estimated LCFA-based MICs.

RESULTS

The MIC was equally well recovered with the LCFA-method as using the LIRT-method, but the LCFA analyses were more than 50 times faster. In the Pain Behavior data (with higher scores indicating more pain behavior), an LCFA-based MIC for improvement was estimated to be 2.85 points (on a simple sum scale ranging 14-42), whereas the LIRT-based MIC was estimated to be 2.60. The sample size simulations showed that smaller sample sizes decreased the precision of the LCFA-based MIC and increased the risk of model non-convergence.

CONCLUSION

The MIC can accurately be estimated using LCFA, but sample sizes need to be preferably greater than 125.

The impact of different data handling strategies in exploratory and confirmatory factor analysis of diary measures: an evaluation using simulated and real-world asthma nighttime symptoms diary data

BACKGROUND

Daily diaries are an important modality for patient-reported outcome assessment. They typically comprise multiple questions, so understanding their underlying structure is key to appropriate analysis and interpretation. Structural evaluation of such measures poses challenges due to the high volume of repeated measurements. Potential strategies include selecting a single day, averaging item-level observations over time, or using all data while accounting for its multilevel structure.

METHOD

The above strategies were evaluated in a simulated dataset via exploratory and confirmatory factor modelling by comparing their impact on various estimates (i.e., inter-item correlations, factor loadings, model fit). Each strategy was additionally explored using real-world data from an observational study (the Asthma Nighttime Symptoms Diary).

RESULTS

Both single day and item average strategies resulted in biased factor loadings. The former displayed lower overall bias (single day: 0.064; item average: 0.121) and mean square error (single day: 0.007; item average: 0.016) but greater frequency of incorrect factor number identification compared with the latter (single day: 46.4%; item average: 0%). Increased estimated inter-item correlations were apparent in the item-average method. Non-trivial between- and within-person variance highlighted the utility of a multilevel approach. However, convergence issues and Heywood cases were more common under the multilevel approach (90.2% and 100.0%, respectively).

CONCLUSIONS

Our findings suggest that a multilevel approach can enhance our insight when evaluating the structural properties of daily diary data; however, implementation challenges still remain. Our work offers guidance on the impact of data handling decisions in diary assessment.

Meaningful Change Thresholds for the Psoriasis Symptoms and Signs Diary: A Secondary Analysis of a Randomized Clinical Trial

Importance

Change from baseline score on the validated Psoriasis Symptoms and Signs Diary (PSSD) is a widely used, patient-reported end point in clinical trials for psoriasis. Meaningful score change thresholds anchored to patient-reported assessments have not been established in a clinical trial setting.

Objective

To evaluate meaningful within-patient score change thresholds for the PSSD using data from the phase 3 Program to Evaluate the Efficacy and Safety of Deucravacitinib, a Selective TYK2 Inhibitor (POETYK), PSO-1 clinical trial, which compared the efficacy and safety of deucravacitinib vs placebo and apremilast among adults with moderate to severe plaque psoriasis.

Design, Setting, and Participants

In this predefined analysis using data from the POETYK PSO-1 multicenter, randomized, double-blind, placebo-controlled phase 3 clinical trial, conducted from August 7, 2018, to September 2, 2020, 666 adults with moderate to severe plaque psoriasis completed the PSSD daily throughout the trial. Meaningful change thresholds were derived by anchoring mean PSSD score change from baseline to week 16 to category improvements on the Patient Global Impression of Change (PGI-C) and the Patient Global Impression of Severity (PGI-S).

Interventions

Deucravacitinib, 6 mg, once daily; placebo; or apremilast, 30 mg, twice daily.

Main Outcome and Measures

The main outcome was score change from baseline to week 16 on the PSSD, anchored to the PGI-C and PGI-S.

Results

The trial included 666 patients (mean [SD] age, 46.1 [13.4] years; 453 men [68.0%]). Three thresholds were identified using an analysis set of 609 patients. Score improvement of at least 15 points from baseline reflected meaningful within-patient change anchored to the PGI-C. Score improvements of 25 points were supported by both the PGI-C and the PGI-S, while a 30-point score change identified patients with greater improvements in their psoriasis symptoms and signs.

Conclusions and Relevance

This analysis suggests that PSSD score improvements of 15, 25, or 30 points represent increasing improvements in disease burden that are meaningful to patients with psoriasis.

The smallest worthwhile effect is superior to the MCID for estimating acceptable benefits of knee arthroplasty

BACKGROUND AND OBJECTIVES

Traditionally, the minimal clinically important difference (MCID) is used to judge the meaningfulness of outcomes in total knee arthroplasty (TKA). However, MCID estimates do not consider patient costs, potential side effects, and inconveniences. MCIDs vary substantially across TKA studies and have several conceptual and psychometric problems. A more scientifically sound alternative for estimating benefits patients expect TKA is the smallest worthwhile effect (SWE), measured with the benefit-harm trade-off method.

METHODS

We recruited 121 participants and followed them for 6 months after surgery. All participants completed Knee Injury and Osteoarthritis Score (KOOS) Pain and Function, and underwent an interview using the benefit-harm trade-off method.

RESULTS

The absolute SWE at the 50th percentile (approximating the average patient) was 31 points KOOS Pain improvement and 28 points for KOOS Function, daily living. Construct validity was supported with strong associations between meeting SWE (yes or no) and satisfaction with 6-month outcome (yes or no) using Pearson Chi Square (24.5, P < 0.001).

CONCLUSION

Current MCID-based methods for interpreting KOOS change scores have flaws while the SWE method is conceptually superior. We determined the SWE of TKA that would justify the costs, risks, and inconveniences of surgery. SWE estimates suggest that patients expect a substantially greater change in KOOS scores than would be expected using MCID estimates. Clinicians can use SWE estimates when discussing likely outcomes and potential TKA benefits and risks with their patients.

Establishing thresholds for meaningful within-individual change using longitudinal item response theory

Purpose

Thresholds for meaningful within-individual change (MWIC) are useful for interpreting patient-reported outcome measures (PROM). Transition ratings (TR) have been recommended as anchors to establish MWIC. Traditional statistical methods for analyzing MWIC such as mean change analysis, receiver operating characteristic (ROC) analysis, and predictive modeling ignore problems of floor/ceiling effects and measurement error in the PROM scores and the TR item. We present a novel approach to MWIC estimation for multi-item scales using longitudinal item response theory (LIRT).

Methods

A Graded Response LIRT model for baseline and follow-up PROM data was expanded to include a TR item measuring latent change. The LIRT threshold parameter for the TR established the MWIC threshold on the latent metric, from which the observed PROM score MWIC threshold was estimated. We compared the LIRT approach and traditional methods using an example data set with baseline and three follow-up assessments differing by magnitude of score improvement, variance of score improvement, and baseline-follow-up score correlation.

Results

The LIRT model provided good fit to the data. LIRT estimates of observed PROM MWIC varied between 3 and 4 points score improvement. In contrast, results from traditional methods varied from 2 to 10 points—strongly associated with proportion of self-rated improvement. Best agreement between methods was seen when approximately 50% rated their health as improved.

Conclusion

Results from traditional analyses of anchor-based MWIC are impacted by study conditions. LIRT constitutes a promising and more robust analytic approach to identifying thresholds for MWIC.