Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling

The challenges inherent with anchor-based approaches to the interpretation of important change in clinical outcome assessments

PURPOSE

Anchor-based methods are group-level approaches used to derive clinical outcome assessment (COA) interpretation thresholds of meaningful within-patient change over time for understanding impacts of disease and treatment. The methods explore the associations between change in the targeted concept of the COA measure and the concept measured by the external anchor(s), typically a global rating, chosen as easier to interpret than the COA measure. While they are valued for providing plausible interpretation thresholds, group-level anchor-based methods pose a number of inherent theoretical and methodological conundrums for interpreting individual-level change.

METHODS

This investigation provides a critical appraisal of anchor-based methods for COA interpretation thresholds and details key biases in anchor-based methods that directly influences the magnitude of the interpretation threshold.

RESULTS

Five important research issues inherent with the use of anchor-based methods deserve attention: (1) global estimates of change are consistently biased toward the present state; (2) the use of static current state global measures, while not subject to artifacts of recall, may exacerbate the problem of estimating clinically meaningful change; (3) the specific anchor assessment response(s) that identify the meaningful change group usually involves an arbitrary judgment; (4) the calculated interpretation thresholds are sensitive to the proportion of patients who have improved; and (5) examination of anchor-based regression methods reveals that the correlation between the COA change scores and the anchor has a direct linear relationship to the magnitude of the interpretation threshold derived using an anchor-based approach; stronger correlations yielding larger interpretation thresholds.

CONCLUSIONS

While anchor-based methods are recognized for their utility in deriving interpretation thresholds for COAs, attention to the biases associated with estimation of the threshold using these methods is needed to progress in the development of standard-setting methodologies for COAs.

Present state bias in transition ratings was accurately estimated in simulated and real data

OBJECTIVE

Patient-reported transition ratings are supposed to reflect the change between a previous baseline health state and a present follow-up state, but may reflect the present state to a greater extent. This so-called 'present state bias' (PSB) potentially threatens the validity of transition ratings. Several criteria have been proposed to assess PSB. We examined how well these criteria perform and to which extent confirmatory factor analysis (CFA) for categorical data provides an accurate assessment of the degree of PSB.

STUDY DESIGN AND SETTING

We simulated a multiple samples with baseline and follow-up item responses to a hypothetical questionnaire, and transition ratings. The samples varied with respect to various distributional characteristics and the degree of PSB. The performance of criteria proposed in the literature, and a new CFA-based criterion, were evaluated by the proportion of explained variance in PSB. In addition, four real datasets were analyzed.

RESULTS

The known criteria explained 36-74% of the variance in PSB. A new CFA-based criterion, namely the ratio of the factor loadings of the transition ratings plus one, explained 81-98% of the variance in PSB across the samples.

CONCLUSION

Present state bias in transition ratings can be estimated accurately using CFA.

The minimal perceived change: a formal model of the responder definition according to the patient's meaning of change for patient-reported outcome data analysis and interpretation

Background

Patient-Reported Outcomes (PROs) are standardized questionnaires used to measure subjective outcomes such as quality of life in healthcare. They are considered paramount to assess the results of therapeutic interventions. However, because their calibration is relative to internal standards in people’s mind, changes in PRO scores are difficult to interpret.

Knowing the smallest value in the score that the patient perceives as change can help. An estimator linking the answers to a Patient Global Rating of Change (PGRC: a question measuring the overall feeling of change) with change in PRO scores is frequently used to obtain this value. In the last 30 years, a plethora of methods have been used to obtain these estimates, but there is no consensus on the appropriate method and no formal definition of this value.

Methods

We propose a model to explain changes in PRO scores and PGRC answers.

Results

A PGRC measures a construct called the Perceived Change (PC), whose determinants are elicited. Answering a PGRC requires discretizing a continuous PC into a category using threshold values that are random variables. Therefore, the populational value of the Minimal Perceived Change (MPC) is the location parameter value of the threshold on the PC continuum defining the switch from the absence of change to change.

Conclusions

We show how this model can help to hypothesize what are the appropriate methods to estimate the MPC and its potential to be a rigorous theoretical basis for future work on the interpretation of change in PRO scores.

Jobs, Careers, and Callings: Exploring Work Orientation at Mid-Career

Mid-career is a time when work orientation (i.e., viewing ones’ work as a job, a career, or a calling) comes into sharper focus. Using Wrzeniewski et al.’s tripartite model, we conducted a discriminant function analysis to determine the combination of variables that best discriminates among people who are aligned with a job, a career, or a calling orientation in a sample of 251 full-time, North American mid-career employees. Compared to those who approach work as a job, those with a calling orientation were more engaged in work. The career-oriented stood apart from the others as a function of shorter job tenure, greater turnover intentions, work engagement, career satisfaction, and a tendency to engage in career self-comparisons. Work-orientation groups did not differ significantly in terms of family centrality, work–life balance, life satisfaction, or well-being. The results suggest that the work orientations represent distinct and equally valid ways to approach work.

Protocol for a randomized controlled trial of low-intensity physical activity for frail older adults: Promoting seniors' health with home care aides (Pro-Home)

Regular participation in physical activity benefits older adults physically and mentally. However, the availability and assessment of physical activity programs that are safe and appropriate for homebound older adults at risk for nursing home admission are limited. Here we describe the protocol for a randomized controlled trial that examines the effectiveness of a gentle physical activity program. Delivered by home care aides who regularly help hard-to-reach older home care clients with housekeeping and routine personal care services in the home, this program is implemented in a real-world context of caregiver-client dyads in a Medicaid-funded home care program. The trial uses a two-group repeated measures design (baseline, Month 4, and Month 8) with 300 pairs of eligible home care clients and their home care aides. The results from this trial could provide evidence and guidelines for a new model of home care, which would facilitate the working together of older home care clients and their home care aides to maintain or improve the functional status of nursing home-eligible older adults.

Establishing Minimal Important Differences for the VR-12 and SANE Scores in Patients Following Treatment of Rotator Cuff Tears

Background:

Minimal important differences (MIDs) for the Veterans RAND 12-Item Health Survey (VR-12) and the Single Assessment Numeric Evaluation (SANE) have not been reported in patients following treatment for rotator cuff tears (RCTs).

Purpose:

To determine the MIDs for the VR-12 and SANE among patients with RCT after treatment.

Study Design:

Cohort study (diagnosis); Level of evidence, 2.

Method:

A total of 222 patients diagnosed with RCT completed the VR-12 and SANE at baseline and then received surgical or nonsurgical treatment. After 64 weeks, 160 patients completed the VR-12, the SANE, and a global change questionnaire. We applied a distribution-based approach to estimate the MIDs for the SANE and for the physical component score (PCS) and mental component score (MCS) of the VR-12. We then used the global rating score as an anchor for 20 patients who perceived a minimal improvement, and we applied an anchor-based approach. One-half standard deviation of the baseline score was used in the distribution-based approach. Linear regression analyses and backward model selection were conducted to evaluate the associations between patients’ characteristics and the anchor-based MIDs.

Results:

The MIDs derived from distribution-based method estimates for the VR-12 PCS, MCS, and SANE scores were 4.94, 5.99, and 11.80, respectively. The MIDs estimated using the anchor-based method for the PCS, MCS, and SANE scores were 2.57 (90% CI, –1.62 to 6.76), 1.87 (90% CI, –2.07 to 5.80), and 27.25 (90% CI, 16.17 to 38.33), respectively. The final regression model for significant predictors of the MID on the PCS included baseline PCS (P < .001), body mass index (P = .014), symptom duration (P = .011), diabetes (P = .009), and surgery (P = .089). The final model for the MID on the MCS included baseline MCS (P < .001), patient sex (P = .027), and diabetes (P = .083). The final model for the MID on SANE included baseline SANE score (P = .059) and diabetes (P = .050).

Conclusion:

This is the first study to assess the MIDs for the VR-12 and SANE scores in patients with rotator cuff disease. The estimates of MID will facilitate the interpretation and application of these outcome measures in clinical practice and research.

High-dose versus standard-dose amoxicillin/clavulanate for clinically-diagnosed acute bacterial sinusitis: A randomized clinical trial

Background

The recommended treatment for acute bacterial sinusitis in adults, amoxicillin with clavulanate, provides only modest benefit.

Objective

To see if a higher dose of amoxicillin will lead to more rapid improvement.

Design, setting, and participants

Double-blind randomized trial in which, from November 2014 through February 2017, we enrolled 315 adult outpatients diagnosed with acute sinusitis in accordance with Infectious Disease Society of America guidelines.

Interventions

Standard-dose (SD) immediate-release (IR) amoxicillin/clavulanate 875 /125 mg (n = 159) vs. high-dose (HD) (n = 156). The original HD formulation, 2000 mg of extended-release (ER) amoxicillin with 125 mg of IR clavulanate twice a day, became unavailable half way through the study. The IRB then approved a revised protocol after patient 180 to provide 1750 mg of IR amoxicillin twice a day in the HD formulation and to compare Time Period 1 (ER) with Time Period 2 (IR).

Main measure

The primary outcome was the percentage in each group reporting a major improvement—defined as a global assessment of sinusitis symptoms as “a lot better” or “no symptoms”—after 3 days of treatment.

Key results

Major improvement after 3 days was reported during Period 1 by 38.8% of ER HD versus 37.9% of SD patients (P = 0.91) and during Period 2 by 52.4% of IR HD versus 34.4% of SD patients, an effect size of 18% (95% CI 0.75 to 35%, P = 0.04). No significant differences in efficacy were seen at Day 10. The major side effect, severe diarrhea at Day 3, was reported during Period 1 by 7.4% of HD and 5.7% of SD patients (P = 0.66) and during Period 2 by 15.8% of HD and 4.8% of SD patients (P = 0.048).

Conclusions

Adults with clinically diagnosed acute bacterial sinusitis were more likely to improve rapidly when treated with IR HD than with SD but not when treated with ER HD. They were also more likely to suffer severe diarrhea. Further study is needed to confirm these findings.

Trial registration

ClinicalTrials.gov Identifier: NCT02340000.

Testing the construct validity of a health transition question using vignette-guided patient ratings of health

Background

A single-item transition question is often used to assess improvement or worsening in health, but its validity has not been tested extensively. The purpose of this study was to test the construct validity of a transition question by relating it to qualitative changes in patient’s self-rating of health guided by clinical vignettes.

Methods

We studied 169 patients with active rheumatoid arthritis (RA) before and after treatment escalation. At both assessments, patients scored their current health on a rating scale after first rating three vignettes describing mild, moderate, or severe RA. We classified patients into one of these three RA categories using a nearest-neighbor match. We then related the change in these self-rated categories between visits to responses to a transition question on visit 2.

Results

Sixty patients improved their RA vignette category after treatment, 86 remained in the same vignette category, and 23 worsened categories. On the transition question, 101 patients reported improvement, 48 reported no change, and 20 reported worsening, representing a modest association with changes in RA vignette categories (polychoric correlation r = 0.19). The association was stronger if patients who were in the mild RA category at both visits were also classified as improved if their self-rating changed from below to above their mild vignette rating (r = 0.23) and when incorporating the importance of changes on the transition question (r = 0.26).

Conclusion

Changes in health states, guided by clinical vignettes, support the construct validity of the transition question.

Domain-specific transition questions demonstrated higher validity than global transition questions as anchors for clinically important improvement

OBJECTIVES

Estimates of minimal clinically important differences in health measures may be affected by the anchor used. We examined if domain-specific transition questions had higher construct validity than global health transition questions as anchors for measures in a given domain.

STUDY DESIGN AND SETTING

In a prospective study of 249 patients with rheumatoid arthritis, we examined changes in pain, physical function, joint swelling, stiffness, fatigue, and depression with treatment. We related these changes to a domain-specific transition question, global arthritis transition question, and the Short Form-36 (SF-36) health transition item.

RESULTS

Changes in all six clinical measures were more highly correlated with the domain-specific transition questions than with the global arthritis question and SF-36 transition question. Discrimination between patients who improved or not was also better using domain-specific questions. Estimates of minimal clinically important improvement (MCII) differed with the anchor when these were based on mean changes. MCII estimates from receiver operating characteristic curve analysis were not influenced by the choice of anchor when anchors had high agreement.

CONCLUSION

Domain-specific transition questions had higher construct validity as anchors for determining clinically important differences in health measures focused on a single domain than either global disease or general health transition questions.

Guidelines in Emergency Medicine Network (GEMNet): guideline for the use of thromboprophylaxis in ambulatory trauma patients requiring temporary limb immobilisation

The Guidelines in Emergency Medicine Network (GEMNet) has been created to promote best medical practice in a range of conditions presenting to emergency departments (EDs) in the UK. This guideline presents a summary of the best available evidence to guide the use of thromboprophylaxis in adult ambulatory outpatients who present to the ED following acute limb trauma and require temporary immobilisation. The document has been developed following discussion among emergency physicians and collegiate fellows to decide which topics would benefit from the development of clinical guidelines. The document is intended as a guideline for use in the ED by emergency physicians and is based on the review of the best existing evidence for treatments used in this setting. The document is summarised as a Clinical Decision Support Guideline that has been presented as an easy to follow algorithm. The intention is for each guideline to be updated and reviewed as further evidence becomes available. The formal revision date has been set at 5 years from publication, though the guideline is subject to continuous informal review.

The minimum important difference for the International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form in women with stress urinary incontinence

INTRODUCTION

Minimum important difference (MID) estimates the minimum degree of change in an instrument's score that correlates with a patient's subjective sense of improvement. We aimed to determine the MID for the International Consultation on Incontinence Questionnaire-Urinary Incontinence Short Form (ICIQ-UI SF) using both anchor based and distribution based methods derived using data from the Trial of Midurethral Slings (TOMUS).

MATERIALS AND METHODS

Instruments for the anchor-based analyses included the urogenital distress inventory (UDI), incontinence impact questionnaire (IIQ), patient global impression of improvement (PGI-I), incontinence episodes (IE) on 7-day bladder diary, and satisfaction with surgical results. After confirming moderate correlation (r ≥ 0.3) of ICIQ-UI SF and each anchor, MIDs were determined by calculating the difference between the mean instrument scores for individuals with the smallest amount of improvement and with no change. The distribution-based method of MID assessment was applied using effect sizes of 0.2 and 0.5 SD (small to medium effects). Triangulation was used to examine these multiple MID values in order to converge on a small range of values.

RESULTS

Anchor-based MIDs range from -4.5 to -5.7 at 12 months and from -3.1 to 4.3 at 24 months. Distribution-based MID values were lower. Triangulation analysis supports a MID of -5 at 12 months and -4 at 24 months.

CONCLUSION

The recommended MIDs for ICIQ-UI SF are -5 at 12 months and -4 at 24 months. In surgical patients, ICIQ-UI SF score changes that meet these thresholds can be considered clinically meaningful.

Methods for interpreting change over time in patient-reported outcome measures

PURPOSE

Interpretation guidelines are needed for patient-reported outcome (PRO) measures' change scores to evaluate efficacy of an intervention and to communicate PRO results to regulators, patients, physicians, and providers. The 2009 Food and Drug Administration (FDA) Guidance for Industry Patient-Reported Outcomes (PRO) Measures: Use in Medical Product Development to Support Labeling Claims (hereafter referred to as the final FDA PRO Guidance) provides some recommendations for the interpretation of change in PRO scores as evidence of treatment efficacy.

METHODS

This article reviews the evolution of the methods and the terminology used to describe and aid in the communication of meaningful PRO change score thresholds.

RESULTS

Anchor- and distribution-based methods have played important roles, and the FDA has recently stressed the importance of cross-sectional patient global assessments of concept as anchor-based methods for estimation of the responder definition, which describes an individual-level treatment benefit. The final FDA PRO Guidance proposes the cumulative distribution function (CDF) of responses as a useful method to depict the effect of treatments across the study population.

CONCLUSIONS

While CDFs serve an important role, they should not be a replacement for the careful investigation of a PRO's relevant responder definition using anchor-based methods and providing stakeholders with a relevant threshold for the interpretation of change over time.

Clinical significance as it relates to evidence-based practice

PURPOSE

This paper analyzes the concept of clinical significance (CS) in relation to evidence-based practice (EBP). The purpose is to show that CS terminology is inconsistent in the nursing literature. It is argued that nursing outcomes and interventions that include findings of CS are difficult to interpret due to lack of an operational definition. It is further argued that the absence of a consistent operational definition is incompatible with EBPs which require standardization of terminology.

DATA SOURCES

The current literature and research studies, particularly from the electronic databases Cumulative Index to Nursing and Allied Health Literature, ISI Web of Knowledge, PubMed, and Cochrane Database of Systematic Reviews.

DATA SYNTHESIS

The disparate uses of CS negatively impact standardizing and quantifying research outcomes to discern EBPs. The authors propose a definition of CS inclusive of the multifarious uses that were revealed in the literature, and conclude that there is a need for professional nursing consensus to define the term. A standard operational definition of CS would enable consistency as clinicians interpret research findings and facilitate translating research to practice.

CONCLUSIONS

Given the centrality of CS to interpreting research findings and applying them to practice, there is a need to solidify the terminology of and measurements for CS in nursing. National nursing agencies, including The National Institute for Nursing Research and Sigma Theta Tau International, should make standardizing CS a high priority for targeted funding. One method of doing so would be to support a consensus convergence to review and select the optimal measures of CS for nursing research. Research to increase knowledge about what constitutes measurement and change, or CS from the patient perspective is needed. Editors and peer reviewers should encourage authors to include a discussion of CS. Discussions of CS should receive greater emphasis in research journals. It is hoped that the preliminary findings from the concept analysis presented in this article will facilitate the work of such a consensus forum.

IMPLICATIONS FOR NURSING PRACTICE

The most immediate and tangible advantages to a common conceptual definition and meaning of CS terminologies by nursing, regardless of which definition is selected, are less confusion and more clarity. Ultimately, the most enduring benefit of a common conceptual definition and measurement for CS is the bridge it provides between research and practice, and the facility with which it promotes the integration of research into EBP.

Cerebral blood flow after surgery for unruptured cerebral aneurysms: effects of surgical manipulation and irrigation fluid

BACKGROUND

Cerebral blood flow (CBF) is important in the management of cerebrovascular diseases. Surgical manipulation may compromise the appropriate interpretation of postoperative CBF changes, but the effects are not well understood.

OBJECTIVE

To investigate the effect of surgical manipulation on postoperative CBF in a setting of prospective randomized comparison of 2 irrigation fluids during surgery.

METHODS

Twenty patients undergoing the clipping of unruptured cerebral aneurysms through the pterional approach were randomly assigned to use of Artcereb, an artificial cerebrospinal fluid, or physiological saline as irrigation fluid. Postoperative CBF and clinical conditions were evaluated 3 times in the first 7 to 10 postoperative days.

RESULTS

Postoperative CBF decreased by 10 to 15% on the first postoperative day in the ipsilateral inferior frontal gyrus, where surgical manipulation may be greatest. CBF reduction was less in regions remote from the surgical site and later in the follow-up periods. Selection of irrigation fluid did not influence postoperative CBF significantly, although postoperative clinical conditions may be better using Artcereb.

CONCLUSION

Postoperative CBF changes due to surgical manipulation should be considered in patients whose hemodynamic conditions are important for appropriate management.

A new approach to combining clinical relevance and statistical significance for evaluation of quality of life changes in the individual patient

OBJECTIVE

Empirical investigation of formerly proposed criteria for relevant changes of health-related quality of life (QOL) regarding their use for monitoring changes in the individual patient. Suggestion of a new criterion trying to overcome the drawbacks of former criteria.

STUDY DESIGN AND SETTING

QOL data were collected longitudinally in 160 cancer patients receiving chemotherapy at an oncological outpatient unit, giving rise to a total of 975 QOL assessments. QOL was measured using the European Organization on Research and Treatment of Cancer Quality of Life Core Questionnaire. Several formerly suggested criteria of relevant change (distribution based, anchor based) were compared in terms of both prevalence and statistical significance of the resulting relevant changes.

RESULTS

When considering criteria of relevant change suggested in the literature, high proportions (average: 42.3-48.3%) of reputedly relevant changes were found. The majority of these changes (average: 55.8-62.2%) were statistically insignificant. Combination of an increased threshold for clinical relevance with the concept of statistical significance resulted in a more meaningful change criterion.

CONCLUSION

Formerly recommended thresholds of relevant change in QOL appear to be unduly low when focusing on the individual patient. A modified criterion is therefore suggested for this case. However, more research is needed for validation and refinement of the proposed criterion.

The relationship between disease control, symptom distress, functioning, and quality of life in adults with asthma

PURPOSE

The purpose of this study was to evaluate quality of life (QOL) from a multidimensional perspective in relation to asthma control, symptom distress, and functioning in adults with asthma.

METHODS

A cross-sectional, mailing survey design was used. The convenience sample of 172 people diagnosed with asthma participated in this study. QOL, conceptualized as subjective satisfaction with life, was measured by the Quality of Life Index-Pulmonary Version III (QLI-PV III). Functioning was measured with the Living with Asthma Questionnaire (LWAQ). Instruments measuring asthma control and symptom experience have been developed for this study. Multiple regression and path analysis were used to examine the relationships.

RESULTS

The QOL was affected directly by functioning (beta = -0.70). Asthma control and symptom distress were indirectly influencing QOL. Among the predictors, direct paths were found between asthma control and functioning (gamma = 0.20), and between symptom distress and functioning (gamma = 0.57); 51% of the total variation in functioning was explained by symptom distress and asthma control; 48% of the total variation in QOL was explained by functioning.

CONCLUSION

The findings highlight the importance of symptom distress and control of asthma symptoms with respect to functioning and QOL in people with asthma.