Estimating meaningful thresholds for multi-item questionnaires using item response theory

Using item response theory to estimate interpretation threshold values for the Frailty Index in community dwelling older adults

BACKGROUND

Although the frailty index (FI) is designed as a continuous measure of frailty, thresholds are often needed to guide its interpretation. This study aimed to introduce and demonstrate the utility of an item response theory (IRT) method in estimating FI interpretation thresholds in community-dwelling adults and to compare them with cutoffs estimated using the receiver operating characteristics (ROC) method.

METHODS

A sample of 1,149 community-dwelling adults (mean[SD], 68[7] years) participated in this cross-sectional study. Participants completed a multi-domain geriatric screen from which the 40-item FI and 3 clinical anchors were computed - namely, (i)self-reported mobility limitations (SRML), (ii)"fair" or "poor" self-rated health (SRH), and (iii) restricted life-space mobility (RLSM). Participants were classified as having SRML-1 if they responded "Yes" to either of the 2 questions regarding walking and stair climbing difficulty and SRML-2 if they reported having walking and stair climbing difficulty. Participants with a Life Space Assessment score <60 points were classified as having RLSM. Threshold values for all anchor questions were estimated using the IRT method and ROC analysis with Youden criterion.

RESULTS

The proportions of participants with SRML-1, SRML-2, Fair/Poor SRH, and RLSM were 21 %, 8 %, 22 %, and 9 %, respectively. The IRT-based thresholds for SRML-2 (0.26), fair/poor SRH (0.29), and RLSM (0.32) were significantly higher than those for SRML-1 (0.18). ROC-based FI cutoffs were significantly lower than IRT-based values for SRML-2, SRH, and RLSM (0.12 to 0.17), and they varied minimally and non-systematically across the anchors.

CONCLUSIONS

The IRT method identifies biologically plausible FI thresholds that could meaningfully complement and contextualize existing thresholds for defining frailty.

Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis

PURPOSE

The minimal important change (MIC) is defined as the smallest within-individual change in a patient-reported outcome measure (PROM) that patients on average perceive as important. We describe a method to estimate this value based on longitudinal confirmatory factor analysis (LCFA). The method is evaluated and compared with a recently published method based on longitudinal item response theory (LIRT) in simulated and real data. We also examined the effect of sample size on bias and precision of the estimate.

METHODS

We simulated 108 samples with various characteristics in which the true MIC was simulated as the mean of individual MICs, and estimated MICs based on LCFA and LIRT. Additionally, both MICs were estimated in existing PROMIS Pain Behavior data from 909 patients. In another set of 3888 simulated samples with sample sizes of 125, 250, 500, and 1000, we estimated LCFA-based MICs.

RESULTS

The MIC was equally well recovered with the LCFA-method as using the LIRT-method, but the LCFA analyses were more than 50 times faster. In the Pain Behavior data (with higher scores indicating more pain behavior), an LCFA-based MIC for improvement was estimated to be 2.85 points (on a simple sum scale ranging 14-42), whereas the LIRT-based MIC was estimated to be 2.60. The sample size simulations showed that smaller sample sizes decreased the precision of the LCFA-based MIC and increased the risk of model non-convergence.

CONCLUSION

The MIC can accurately be estimated using LCFA, but sample sizes need to be preferably greater than 125.

Estimating cutpoints of gait speed and sit-to-stand test values for self-reported mobility limitations in a cohort of community-dwelling older adults from Singapore: comparing receiver operating characteristic (ROC) analysis with adjusted predictive modelling

OBJECTIVES

Clinical interpretability of the gait speed and 5-times sit-to-stand (5-STS) tests is commonly established by comparing older adults with and without self-reported mobility limitations (SRML) on gait speed and 5-STS performance, and estimating clinical cutpoints for SRML using the receiver operating characteristics (ROC) method. Accumulating evidence, however, suggests that the adjusted predictive modeling (APM) method may be more appropriate to estimate these interpretational cutpoints. Thus, we aimed to compare, in community-dwelling older adults, gait speed and 5-STS cutpoints estimated using the ROC and APM methods.

DESIGN

Cross-sectional study.

SETTING AND PARTICIPANTS

This study analyzed data from 955 community-dwelling independently walking older adults (73%women) aged ≥60 years (mean, 68; range, 60-88).

METHODS

Participants completed the 10-metre gait speed and 5-STS tests. Participants were classified as having SRML if they responded "Yes" to either of the 2 questions regarding walking and stair climbing difficulty. Cutpoints for SRML and its component questions were estimated using ROC analysis with Youden criterion and the APM method.

RESULTS

The proportions of participants with self-reported walking difficulty, self-reported stair climbing difficulty, and SRML were 10%, 19%, and 22%, respectively. Gait speed and 5-STS time were moderately correlated with each other (r=-0.56) and with the self-reported measures (absolute r-values, 0.39-0.44). ROC-based gait speed cutpoints were 0.14 to 0.16 m/s greater than APM-based cutpoints (P < 0.05) whilst ROC-based 5-STS time cutpoints were 0.8 to 3.3 s lower than APM-based cutpoints (P < 0.05 for walking difficulty). Compared with ROC-based cutpoints, APM-based cutptoints were more precise and they varied monotonically with self-reported walking difficulty, self-reported stair climbing difficulty, and SRML.

CONCLUSIONS AND IMPLICATIONS

In a sample of 955 older adults, our findings of precise and biologically plausible gait speed and 5-STS cutpoints for SRML estimated using the APM method indicate that this promising method could potentially complement or even replace traditional ROC methods.