Identifying response shift statistically at the individual level

Implications of the syntheses on definition, theory, and methods conducted by the Response Shift - in Sync Working Group

PURPOSE

Our aim is to advance response shift research by explicating the implications of published syntheses by the Response Shift - in Sync Working Group in an integrative way and suggesting ways for improving the quality of future response shift studies.

METHODS

Members of the Working Group further discussed the syntheses of the literature on definitions, theoretical underpinnings, operationalizations, and response shift methods. They outlined areas in need of further explication and refinement, and delineated additional implications for future research.

RESULTS

First, the proposed response shift definition was further specified and its implications for the interpretation of results explicated in relation to former, published definitions. Second, the proposed theoretical model was further explained in relation to previous theoretical models and its implications for formulating research objectives highlighted. Third, ways to explore alternative explanations per response shift method and their implications for response shift detection and explanation were delineated. The implications of the diversity of the response shift methods for response shift research were presented. Fourth, the implications of the need to enhance the quality and reporting of the response shift studies for future research were sketched.

CONCLUSION

With our work, we intend to contribute to a common language regarding response shift definitions, theory, and methods. By elucidating some of the major implications of earlier work, we hope to advance response shift research.

If it's information, it's not "bias": a scoping review and proposed nomenclature for future response-shift research

BACKGROUND

The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as "bias," "noise," "nuisance," or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings.

METHODS

A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles).

RESULTS

Of 10 methods used, papers using four of them utilized terms like "bias" and aimed to remove response-shift effects to reveal "true change." Yet, the investigators' reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods.

CONCLUSIONS

It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.

Critical examination of current response shift methods and proposal for advancing new methods

Purpose

This work is part of an international, interdisciplinary initiative to synthesize research on response shift in results of patient-reported outcome measures. The objective is to critically examine current response shift methods. We additionally propose advancing new methods that address the limitations of extant methods.

Methods

Based on literature reviews, this critical examination comprises design-based, qualitative, individualized, and preference-based methods, latent variable models, and other statistical methods. We critically appraised their definition, operationalization, the type of response shift they can detect, whether they can adjust for and explain response shift, their assumptions, and alternative explanations. Overall limitations requiring new methods were identified.

Results

We examined 11 methods that aim to operationalize response shift, by assessing change in the meaning of one’s self-evaluation. Six of these methods distinguish between change in observed measurements (observed change) and change in the construct that was intended to be measured (target change). The methods use either (sub)group-based or individual-level analysis, or a combination. All methods have underlying assumptions to be met and alternative explanations for the inferred response shift effects. We highlighted the need to address the interpretation of the results as response shift and proposed advancing new methods handling individual variation in change over time and multiple time points.

Conclusion

No single response shift method is optimal; each method has strengths and limitations. Additionally, extra steps need to be taken to correctly interpret the results. Advancing new methods and conducting computer simulation studies that compare methods are recommended to move response shift research forward.

Supplementary Information

The online version of this article (10.1007/s11136-020-02755-4) contains supplementary material, which is available to authorized users.

Response shift in results of patient-reported outcome measures: a commentary to The Response Shift-in Sync Working Group initiative

PURPOSE

The Working Group undertook a critical, comprehensive synthesis of the response shift work to date. We aimed to (1) describe the rationale for this initiative; (2) outline how the Working Group operated; (3) summarize the papers that comprise this initiative; and (4) discuss the way forward.

METHODS

Four interdisciplinary teams, consisting of response shift experts, external experts, and new investigators, prepared papers on (1) definitions and theoretical underpinnings, (2) operationalizations and response shift methods, (3) implications for healthcare decision-making, and (4) on the published magnitudes of response shift effects. Draft documents were discussed during a two-day meeting. Papers were reviewed by all members.

RESULTS

Vanier and colleagues revised the formal definition and theory of response shift, and applied these in an amended, explanatory model of response shift. Sébille and colleagues conducted a critical examination of eleven response shift methods and concluded that for each method extra steps are required to make the response shift interpretation plausible. Sawatzky and colleagues created a framework for considering the impact of response shift on healthcare decision-making at the level of the individual patient (micro), the organization (meso), and policy (macro). Sajobi and colleagues are conducting a meta-analysis of published response shift effects. Preliminary findings indicate that the mean effect sizes are often small and variable across studies that measure different outcomes and use different methods.

CONCLUSION

Future response shift research will benefit from collaboration among diverse people, formulating alternative hypotheses of response shift, and conducting the most conclusive studies aimed at testing these (falsification).

Response shift effects in the assessment of urologic cancer patients' quality of life

OBJECTIVE

Urologic cancer patients often assess their quality of life (QoL) as being relatively good. Response shift (change in internal standards) may be one reason for that effect. The aim of this study was to test such response shift effects.

METHODS

A sample of 197 male urologic cancer patients was tested while hospitalised (t1) and three months later (t2). The participants had to assess their current health state and the health state of two anchoring vignettes. They also completed the EORTC QLQ-C30 and PHQ-4 questionnaires, including a retrospective thentest. The control sample was comprised of members of the general population.

RESULTS

The patients rated their general health as being worse than people from the general population did (effect size: d = 0.73, p < 0.001). Moreover, the patients assessed the vignette presenting physical problems as being significantly healthier than people from the general population did (d = 0.42, p < 0.001). Under the retrospective thentest condition, the patients rated their QoL and their mental burden as being significantly worse than in the pre-test condition. Both methods showed response shift effects.

CONCLUSION

Urologic cancer patients' assessments of their QoL should be handled with caution because of possible shifts in their internal standards of judgement.

Gait Training after Stroke on a Self-Paced Treadmill with and without Virtual Environment Scenarios: A Proof-of-Principle Study

Purpose: The purpose of this proof-of-principle study was to show that virtual reality (VR) technology could be coupled with a self-paced treadmill to further improve walking competency in individuals with chronic stroke. Method: A 62-year-old man with a chronic right hemispheric stroke participated in a treadmill walking programme involving first a control (CTL) protocol, then VR training. In CTL training, he walked without time constraints while viewing still pictures and reacting to treadmill movements similar to those that he would have experienced later in VR training. In VR training, he experienced treadmill movements programmed to simulate changes encountered in five virtual environments rear-projected onto a large screen. Training difficulty in nine sessions over 3 weeks was increased by varying the time constraints, terrain surface changes, and obstacles to avoid. Effects on walking competency were assessed using clinical measures (5 m walk test, 6 min walk test, Berg Balance Scale, Activities-specific Balance Confidence scale) and questionnaires (Assessment of Life Habits Scale and personal appraisal). Results: CTL and VR training resulted in a similar progression through the training sessions of total time walked on the treadmill. The VR training led to an additional increase in speed as measured by walking 5 metres as fast as possible and distance walked in 6 minutes, as well as improved balance self-efficacy and anticipatory locomotor adjustments. As reported by the participant, these improved outcomes transferred to real-life situations. Conclusions: Despite the limited potential for functional recovery from chronic stroke, an individual can achieve improvements in mobility and self-efficacy after participating in VR-coupled treadmill training, compared with treadmill training with the same intensity and surface perturbations but without VR immersion. A larger scale, randomized controlled trial is warranted to determine the efficacy of VR-coupled treadmill training for mobility intervention post-stroke.

A systematic review of the quality of reporting of simulation studies about methods for the analysis of complex longitudinal patient-reported outcomes data

PURPOSE

This study describes the characteristics and quality of reporting for published computer simulation studies about statistical methods to analyze complex longitudinal (i.e., repeated measures) patient-reported outcomes (PROs); we included methods for longitudinal latent variable measurement and growth models and response shift.

METHODS

Scopus, PsycINFO, PubMed, EMBASE, and Social Science Citation Index were searched for English-language studies published between 1999 and 2016 using selected keywords. Extracted information included characteristics of the study purpose/objectives, simulation design, software, execution, performance, and results. The quality of reporting was evaluated using published best-practice guidelines.

SYNTHESIS

A total of 1470 articles were reviewed and 42 articles met the inclusion criteria. The majority of the included studies (73.8%) investigated an existing statistical method, primarily a latent variable model (95.2%). Most studies specified the population model, including variable distributions, mean parameters, and correlation/covariances. The number of time points and sample size(s) were reported by all studies, but justification for the selected values was rarely provided. The majority of the studies (52.4%) did not report on model non-convergence. Bias, accuracy, and model fit were commonly reported performance metrics. All studies reported results descriptively, and 26.2% also used an inferential method.

CONCLUSIONS

While methodological research on statistical analyses of complex longitudinal PRO data is informed by computer simulation studies, current reporting practices of these studies have not been consistent with best-practice guidelines. Comprehensive reporting of simulation methods and results ensures that the strengths and limitations of the investigated statistical methods are thoroughly explored.

Scoping review of response shift methods: current reporting practices and recommendations

BACKGROUND

Response shift (RS) has been defined as a change in the meaning of an individual's self-evaluation of his/her health status and quality of life. Several statistical model- and design-based methods have been developed to test for RS in longitudinal data. We reviewed the uptake of these methods in patient-reported outcomes (PRO) literature.

METHODS

CINHAHL, EMBASE, Medline, ProQuest, PsycINFO, and Web of Science were searched to identify English-language articles about RS published until 2016. Data on year and country of publication, PRO measure adopted, RS detection method, type of RS detected, and testing of underlying model assumptions were extracted from the included articles.

RESULTS

Of the 1032 articles identified, 101 (9.8%) articles were included in the study. While 54.5 of the articles reported on the Then-test, 30.7% of the articles reported on Oort's or Schmitt's structural equation modeling (SEM) procedure. Newer RS detection methods, such as relative importance analysis and random forest regression, have been used less frequently. Less than 25% reported on testing the assumptions underlying the adopted RS detection method(s).

CONCLUSIONS

Despite rapid methodological advancements in RS research, this review highlights the need for further research about RS detection methods for complex longitudinal data and standardized reporting guidelines.

Identifying patterns of adaptation in breast cancer patients with cancer-related fatigue using response shift analyses at subgroup level

Fatigue is the most prevalent symptom in breast cancer. It might be perceived differently among patients over time as a consequence of the differing patients’ adaptation and psychological adjustment to their cancer experience which can be related to response shift (RS). RS analyses can provide important insights on patients’ adaptation to cancer but it is usually assumed that RS occurs in the same way in all individuals which is unrealistic. This study aimed to identify patients’ subgroups in which different RS effects on self‐reported fatigue could occur over time using a combination of methods for manifest and latent variables. The FATSEIN study comprised 466 breast cancer patients followed over a 2‐year period. Fatigue was measured with the Multidimensional Fatigue Inventory questionnaire (MFI‐20) during 10 visits. A novel combination of Mixed Models, Growth Mixture Modeling, and Structural Equation Modeling was used to assess the occurrence of RS in fatigue changes to identify subgroups displaying different RS patterns over time. An increase in fatigue was evidenced over the 8‐month follow‐up, followed by a decrease between the 8‐ and 24‐month. Four latent classes of patients were identified. Different RS patterns were detected in all latent classes between the inclusion and 8 months (last cycle of chemotherapy). No RS was evidenced between 8‐ and 24‐month. Several RS effects were evidenced in different groups of patients. Women seemed to adapt differently to their treatment and breast cancer experience possibly indicating differing needs for medical/psychological support.

The Guttman errors as a tool for response shift detection at subgroup and item levels

PURPOSE

Statistical methods for identifying response shift (RS) at the individual level could be of great practical value in interpreting change in PRO data. Guttman errors (GE) may help to identify discrepancies in respondent's answers to items compared to an expected response pattern and to identify subgroups of patients that are more likely to present response shift. This study explores the benefits of using a GE-based method for RS detection at the subgroup and item levels.

METHODS

The analysis was performed on the SatisQoL study. The number of GE was determined for each individual at each time of measurement (at baseline T0 and 6 months after discharge M6). Individuals showing discrepancies (with many GE) were suspected to interpret the items differently from the majority of the sample. Patients having a large number of GE at M6 only and not at T0 were assumed to present RS. Patients having a small number of GE at T0 and M6 were assumed to present no RS. The RespOnse Shift ALgorithm in Item response theory (ROSALI) was then applied on the whole sample and on both groups.

RESULTS

Different types of RS (non-uniform recalibration, reprioritization) were more prevalent in the group composed of patients assumed to present RS based on GE. On the opposite, no RS was detected on patients having few GE.

CONCLUSIONS

Guttman errors and item response theory models seem to be relevant tools to discriminate individuals affected by RS from the others at the item level.

Understanding and optimizing brain health in HIV now: protocol for a longitudinal cohort study with multiple randomized controlled trials

Background

Chronic HIV infection commonly affects both cognition and mental health, even with excellent systemic viral control. The causes of compromised brain health are likely to be a multi-factorial combination of HIV-related biological factors, co-morbidities such as aging and cerebrovascular disease, and the erosion of coping skills, physical health, and social supports resulting from the strains of living with a chronic illness.

Methods/design

This study aims to provide a better understanding of the relationship between cognitive complaints, depression, and objectively measured cognitive impairment in HIV, and of the key factors, whether biological or personal, which relate to these presentations and to their evolution over time. Characterization of this heterogeneity will permit more focused pathophysiological studies, and allow more targeted interventions. The project makes extensive use of Web-based research and health care delivery tools, aiming to provide cost-effective, “clinic ready” tools to improve brain health in HIV. This project has two overarching aims, reflecting our dual goals of understanding and improving brain health in HIV, focusing on cognitive impairment, its contributors and consequences.

The objectives are to contribute evidence for the validity of a brief brain health assessment, to estimate the extent to which HIV-related cognition-relevant clinical factors and patient-centered outcomes inter-relate and evolve over time, allowing identification of the mechanisms underpinning longitudinal change in brain health and to contribute evidence for the feasibility, effectiveness potential, acceptability, and underlying mechanisms of promising interventions for optimizing brain health. We adopt a cohort multiple randomized control trials design. A total of 900 participants will be characterized prospectively over a 27-month period to answer questions about the evolution of outcomes of interest. All participants will be offered basic brain health self-management information. Sub-groups will participate in pilot studies of specific, more intensive interventions to provide pragmatic evidence for feasibility, effectiveness, and comparative effectiveness.

Discussion

This work will provide needed estimates of the burden, heterogeneity, evolution, and mechanisms underlying compromised brain health in HIV, and test a range of promising non-pharmacological interventions. This is an on-going study; the trials nested within this cohort that are currently recruiting participants were registered on 7 October 2015 (Clinicaltrials.gov NCT02571504 and NCT02571595).

Response shift and disease activity in inflammatory bowel disease

PURPOSE

Response shift (RS) may mask true change in health-related quality of life in longitudinal studies. People with chronic conditions may experience RS as they adapt to their disease, but it is unknown whether fluctuations in disease activity will influence the presence of RS. The study purpose was to test for RS in individuals with inflammatory bowel disease (IBD), a condition characterized by periods of symptom flares and remission.

METHODS

Data were from the Manitoba IBD Cohort Study (N = 388). Multi-group confirmatory factor analysis (MG-CFA) and a RS detection method based on structural equation modeling were used to test for reconceptualization, reprioritization, and recalibration RS in participants with consistent active, consistent inactive, and inconsistent disease activity over a 6-month period on the SF-36.

RESULTS

The MG-CFA revealed that a weak invariance model with equal factor loadings across groups was the best fit to the baseline SF-36 data. Reconceptualization, uniform recalibration, and non-uniform recalibration RS was detected in the consistent active group, but effect sizes were small. For the consistent inactive group, recalibration RS was observed and effect sizes were small to moderate. For the inconsistent disease activity group, small-to-moderate recalibration RS effects were observed. There was no evidence of reprioritization.

CONCLUSIONS

Individuals with a chronic disease may exhibit RS even if they are not actively experiencing symptoms on a consistent basis. Heterogeneity in the type and magnitude of RS effects may be observed in chronic disease patients who experience changes in disease symptoms.

How are you? Do people with inflammatory bowel disease experience response shift on this question?

BACKGROUND

As individuals experience changes in their health, they may alter the way they evaluate health and quality of life. The purpose of this study is to estimate the extent to which individuals with IBD change their rating of health over time because of response shift (RS).

METHODS

This is a reanalysis of a population-based longitudinal study of IBD in Manitoba, Canada (n = 388). RS was examined using trajectories of the difference between observed and predicted health. Logistic regression and dual trajectories were used to identify predictors of RS.

RESULTS

Disease activity, vitality, pain, somatization, and physical and social function explained 51% of the variation in general health over two years with no evidence of RS in 82% of the sample. Negative RS was found for 8%, who initially rated health better than predicted; positive RS was found for 6%. The positive RS group was younger and had better baseline scores on measures of general health, hostility, pain, mental health and social and role function; less pain and better social function scores at baseline were predictors of negative RS.

CONCLUSIONS

In conclusion, the majority of people with IBD did not demonstrate a RS indicating that the health rating over time was stable in relation to that predicted by known time varying clinical variables. This adds to the evidence that the single question on self-rated health is useful for monitoring individuals over time.

Exploring the Response Shift Effect on the Quality of Life of Patients with Schizophrenia

Background. Interpretation of quality of life (QoL) scores over time can be difficult because of possible changes in internal standards, values, and conceptualization of QoL by individuals. This effect is called a response shift (RS). The purpose of this study was to examine whether an RS effect occurred over a 24-mo period in patients who were suffering from schizophrenia. Methods. The random forest method was applied to detect any RS reprioritization in a multicenter cohort study. QoL was recorded using a generic questionnaire (SF36) at baseline (T0), 12 mo (T12), and 24 mo (T24). Patients were categorized into 3 groups based on psychotic symptoms and relapse (stable, improved, and worsened groups) from their clinical profiles. The random forest method was performed to predict the General Health score of the SF36 from the other QoL domain scores of the SF36. We estimated the average variable importance of the QoL domain for each of the 3 groups. Results. A total of 124 (53.2%) patients were defined as stable, 59 (25.3%) as improved, and 50 (21.5%) as worsened. Among the stable group, the Social Functioning domain became more important over time. Of those classified as improved, the Mental Health domain became more important over time, while the Vitality domain became less important. Among those in the group who worsened, the Mental Health domain became less important while the Vitality and Bodily Pain domains became more important. Conclusions. Our study identified differential RS reprioritization among patients with different clinical profiles. Further work is needed to determine whether RS should be interpreted as a measurement bias or as an effect integrated in a true change.

Identifying reprioritization response shift in a stroke caregiver population: a comparison of missing data methods

PURPOSE

Response shift (RS) is an important phenomenon that influences the assessment of longitudinal changes in health-related quality of life (HRQOL) studies. Given that RS effects are often small, missing data due to attrition or item non-response can contribute to failure to detect RS effects. Since missing data are often encountered in longitudinal HRQOL data, effective strategies to deal with missing data are important to consider. This study aims to compare different imputation methods on the detection of reprioritization RS in the HRQOL of caregivers of stroke survivors.

METHODS

Data were from a Canadian multi-center longitudinal study of caregivers of stroke survivors over a one-year period. The Stroke Impact Scale physical function score at baseline, with a cutoff of 75, was used to measure patient stroke severity for the reprioritization RS analysis. Mean imputation, likelihood-based expectation-maximization imputation, and multiple imputation methods were compared in test procedures based on changes in relative importance weights to detect RS in SF-36 domains over a 6-month period. Monte Carlo simulation methods were used to compare the statistical powers of relative importance test procedures for detecting RS in incomplete longitudinal data under different missing data mechanisms and imputation methods.

RESULTS

Of the 409 caregivers, 15.9 and 31.3 % of them had missing data at baseline and 6 months, respectively. There were no statistically significant changes in relative importance weights on any of the domains when complete-case analysis was adopted. But statistical significant changes were detected on physical functioning and/or vitality domains when mean imputation or EM imputation was adopted. There were also statistically significant changes in relative importance weights for physical functioning, mental health, and vitality domains when multiple imputation method was adopted. Our simulations revealed that relative importance test procedures were least powerful under complete-case analysis method and most powerful when a mean imputation or multiple imputation method was adopted for missing data, regardless of the missing data mechanism and proportion of missing data.

CONCLUSIONS

Test procedures based on relative importance measures are sensitive to the type and amount of missing data and imputation method. Relative importance test procedures based on mean imputation and multiple imputation are recommended for detecting RS in incomplete data.

Emotional vitality in caregivers: application of Rasch Measurement Theory with secondary data to development and test a new measure

OBJECTIVE

To describe the practical steps in identifying items and evaluating scoring strategies for a new measure of emotional vitality in informal caregivers of individuals who have experienced a significant health event.

DESIGN

The psychometric properties of responses to selected items from validated health-related quality of life and other psychosocial questionnaires administered four times over a one-year period were evaluated using Rasch Measurement Theory.

SETTING

Community.

SUBJECTS

A total of 409 individuals providing informal care at home to older adults who had experienced a recent stroke.

MAIN MEASURES

Rasch Measurement Theory was used to test the ordering of response option thresholds, fit, spread of the item locations, residual correlations, person separation index, and stability across time.

RESULTS

Based on a theoretical framework developed in earlier work, we identified 22 candidate items from a pool of relevant psychosocial measures available. Of these, additional evaluation resulted in 19 items that could be used to assess the five core domains. The overall model fit was reasonable (χ(2) = 202.26, DF = 117, p = 0.06), stable across time, with borderline evidence of multidimensionality (10%). Items and people covered a continuum ranging from -3.7 to +2.7 logits, reflecting coverage of the measurement continuum, with a person separation index of 0.85. Mean fit of caregivers was lower than expected (-1.31 ±1.10 logits).

CONCLUSION

Established methods from the Rasch Measurement Theory were applied to develop a prototype measure of emotional vitality that is acceptable, reliable, and can be used to obtain an interval level score for use in future research and clinical settings.

Changes in quality of life after epilepsy surgery: the role of reprioritization response shift

OBJECTIVES

Health-related quality of life (HRQOL) is an important indicator of well-being in patients with epilepsy. When assessing changes in HRQOL over time, some of the changes observed may be due to the patients' change in the internal standards, value, or meaning that they attribute to the HRQOL domain being measured, rather than actual change, a phenomenon known as response shift. Response shifts are increasingly recognized as an important explanation for the seemingly paradoxical or counterintuitive HRQOL results often observed in chronic conditions. We investigated the presence of changing values (reprioritization response shift) in data from a surgical randomized controlled trial of patients with temporal lobe epilepsy (TLE).

METHODS

Eighty patients with TLE, who were randomized to surgical and medical treatment, provided data on the epilepsy-specific 31-item Quality of life in Epilepsy (QOLIE-31) questionnaire at baseline and 1 year after randomization. Reprioritization response shift among the seven QOLIE-31 domains was assessed using changes in the relative importance weights derived from logistic regression and discriminant analysis.

RESULTS

The relative importance analysis showed a statistically significant increase over time in the importance of social function, but a significant decrease in the relative importance of seizure worry. There were no significant changes in the relative importance of the remaining five domains in distinguishing between surgical and medical group over time.

SIGNIFICANCE

Patients that receive surgical treatment are more likely to experience a decrease in the valuation of seizure worry and an increase in the value of social function as compared to patients who are medically treated. Changes in expectation about seizure freedom and social function may confound the assessment of longitudinal change on these outcomes, and highlight the importance of assessing response shift and the limitations of assessing HRQOL at a single point in time or without a control group. A PowerPoint slide summarizing this article is available for download in the Supporting Information section here.

Response shift recalibration and reprioritization in health-related quality of life was identified prospectively in older men with and without stroke

OBJECTIVES

To prospectively consider evidence for response shift (RS) in health-related quality of life (HRQL) in older men who experienced stroke or remained stroke free. RS is a change in the meaning of self-evaluation; this includes recalibration, reprioritization, and reconceptualization.

STUDY DESIGN

A cohort of 3,983 male World War II Royal Canadian Air Force recruits has been followed since 1948. There were three prospectively determined groups: stroke survivors (n = 168; mean age, 80.1 years); older stroke-free group (n = 254; mean age, 82.8 years); and younger stroke-free group (n = 323; mean age, 74.7 years). The Short Form-36 (SF-36) was used to evaluate HRQL. Longitudinal structural equation models were developed using SF-36 subscales and three latent variables. Measurement invariance over two time points for each of the three groups was evaluated to identify RS.

RESULTS

All RS models had reasonable fit: stroke [root mean square error of approximation (RMSEA), 0.069; 90% confidence interval (CI): 0.052, 0.086], older (RMSEA, 0.055; 90% CI: 0.041, 0.068), and younger (RMSEA, 0.062; 90% CI: 0.051, 0.074). Recalibration of physical function occurred in all three groups. Reprioritization of role limitations due to physical health happened in both stroke-free groups.

CONCLUSION

This study is unique in our ability to prospectively identify RS recalibration and reprioritization in HRQL in aging men with stroke and remaining free of stroke. Changes in the meaning of self-evaluation of HRQL occur not only with stroke but also in men who remain free of stroke.

Assessment of response shift using two structural equation modeling techniques

OBJECTIVE

To identify response shift using two structural equation modeling (SEM) techniques.

STUDY DESIGN AND SETTING

Hypertensive patients (n = 909) with coronary artery disease (CAD) completed SF-36 surveys at both baseline and 1-year follow-up. Response shift was identified using Oort and Schmitt SEM techniques. The type of response shift linked to changes in various parameters of the SEM measurement model is defined differently for both SEM approaches. Effect sizes were calculated for the impact of response shift on the change of SF-36 domain scores when using the Oort approach.

RESULTS

Both Oort and Schmitt SEM approaches identified response shift only in the SF-36 physical functioning (PF) scale. The effect size of recalibration on the change of PF domain scores when using the Oort approach was -0.12.

CONCLUSION

This study showed that hypertensive patients with CAD experienced a response shift over a 1-year period. Both the SEM approaches identified response shift (uniform recalibration using the Oort approach and recalibration using the Schmitt approach); however, both approaches use different parameters to define and test response shift. We found that either the variation in analytic methods or the sample used may influence the identification and type of response shift.

Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study

Background

Multiple sclerosis (MS), a common neurodegenerative disease, has well-described associations with quality of life (QoL) impairment. QoL changes found in longitudinal studies are difficult to interpret due to the potential response shift (RS) corresponding to respondents’ changing standards, values, and conceptualization of QoL. This study proposes to test the capacity of Random Forest (RF) for detecting RS reprioritization as the relative importance of QoL domains’ changes over time.

Methods

This was a longitudinal observational study. The main inclusion criteria were patients 18 years old or more with relapsing-remitting multiple sclerosis. Every 6 months up to month 24, QoL was recorded using generic and MS-specific questionnaires (MusiQoL and SF-36). At 24 months, individuals were divided into two ‘disability change’ groups: worsened and not-worsened patients. The RF method was performed based on Breiman’s description. Analyses were performed to determine which QoL scores of SF-36 predicted the MusiQoL index. The average variable importance (AVI) was estimated.

Results

A total of 417 (79.6%) patients were defined as not-worsened and 107 (20.4%) as worsened. A clear RS was identified in worsened patients. While the mental score AVI was almost one third higher than the physical score AVI at 12 months, it was 1.5 times lower at 24 months.

Conclusion

This work confirms that the RF method offers a useful statistical approach for RS detection. How to integrate the RS in the interpretation of QoL scores remains a challenge for future research.

Trial registration

ClinicalTrials.gov identifier: NCT00702065

Relative importance measures for reprioritization response shift

PURPOSE

Response shift (RS), a change in the meaning of an individual's self-evaluation of a target construct, such as health-related quality of life (HRQOL), can affect the interpretation of change in measures of the construct collected over time. This study proposes new statistical methods to test for reprioritization RS, in which the relative importance of HRQOL domains changes over time.

METHODS

The methods use descriptive discriminant analysis or logistic regression models and bootstrap inference to test for change in relative importance weights (method 1) or ranks (method 2) for discriminating between patient groups at two occasions. The methods are demonstrated using data from the Manitoba Inflammatory Bowel Disease (IBD) Cohort Study (n = 388). Reprioritization of domains from the IBD Questionnaire (IBDQ) and SF-36 was investigated for groups with active and inactive disease symptoms.

RESULTS

The IBDQ bowel symptoms and SF-36 bodily pain domains had the highest ranks for group discrimination. Using Method 1, there was evidence of reprioritization RS in the IBDQ social functioning domain and the SF-36 bodily pain and social functioning domains. Method 2 did not detect change for any of the domains.

CONCLUSIONS

Compared to IBD patients without active disease symptoms, those with active symptoms were likely to change the meaning of their self-evaluations of pain and social interactions. Further research is needed to compare these new RS detection methods under a variety of data analytic conditions before recommendations about the optimal method can be made.

Building on transformative learning and response shift theory to investigate health-related quality of life changes over time in individuals with chronic health conditions and disability

A major goal of treatment for people living with chronic illness or disability is self-management leading to optimized health-related quality of life. This change process has been described in the adult education literature as transformative learning, while in health-related quality of life research, response shift has emerged as a key concept. Response shift and transformative learning literature were reviewed, and the theoretical frameworks of the 2 concepts were compared and contrasted. Response shift is described as a change in internal standards, values, or definition of a construct (eg, health-related quality of life) over time, commonly seen in individuals with chronic illness. In the context of chronic illness, transformative learning is described as a complex process of personal change including beliefs, feelings, knowledge, and values. Transformative learning is often triggered by the diagnosis of a chronic illness. This results in a critical reflection of taken-for-granted assumptions and leads to new ways of thinking, influencing personal changes in daily living. Comparing the models of response shift and transformative learning in chronic illness, the catalyst in response shift appears comparable with the trigger in transformational learning; mechanisms to process of changing; and perceived quality of life to outcomes. Both transformative learning and response shift have much to offer health care providers in understanding the learning process for the person living with chronic illness or disability to optimize their quality of life. Suggestions for future research in response shift and transformative learning in individuals with chronic health conditions and disability are proposed.

Changes in life satisfaction in persons with spinal cord injury during and after inpatient rehabilitation: adaptation or measurement bias?

Purpose

To analyze changes in life satisfaction (LS) scores over time in persons with spinal cord injury (SCI) and to interpret what these changes mean.

Methods

Multicenter, prospective cohort study of persons with SCI (n = 96) classified into 3 life satisfaction trajectories identified earlier. Assessment took place 6 times from the start of active rehabilitation up to 5 years after discharge. Three LS scores were compared: (1) LS ‘now’ score, (2) ‘comparison’ score between LS ‘now’ and LS ‘before the SCI’, and (3) retrospective score of LS ‘before the SCI’.

Results

Persons in the low LS trajectory showed increase in the LS ‘now’ score, but not in the LS ‘comparison’ score and retrospective score. Persons in the recovery trajectory showed increase in the LS ‘now’ and LS ‘comparison’ scores, but not in the retrospective score. Persons in the high LS trajectory showed increase in the ‘comparison’ LS score and decrease in the retrospective score, but no change in the LS ‘now’ score.

Conclusions

Diverging patterns of change were found and that were interpreted as adaptation or scale recalibration. Recalibration could also be considered healthy rebalancing after SCI. Being able to compare different LS ratings can facilitate the interpretation of change in and stability of LS.

Response shift effect on gastrointestinal quality of life index after laparoscopic cholecystectomy

PURPOSE

Traditional pre- and post-surgery quality of life assessments are inadequate for assessing change in health-related quality of life (HRQoL) after laparoscopic cholecystectomy (LC). This study examined whether a response shift, a change in the internal standards of a patient, occurs in patients who have received LC.

METHODS

Self-administered gastrointestinal quality of life index (GIQLI) was used to evaluate preoperative, postoperative, and retrospective postoperative HRQoL. Response shifts, unadjusted treatment effects, adjusted treatment effects, and their effect sizes were calculated.

RESULTS

In all GIQLI domains, a significant response shift was indicated by the significantly higher pre-test scores compared to then-test scores (P < 0.05). The effect size of the response shift ranged from 0.19 for the physical impairment domain of the GIQLI to 0.49 for the total GIQLI score. It was observed the treatment effect was greater after adjusting for the presence of response shift.

CONCLUSION

Patients who have received LC undergo a response shift that affects their outcome measurement at 6 months postoperative. Response shift is a potentially confounding factor and should be considered when designing clinical studies that employ self-administered HRQoL measures. This evidence of confounding effects warrants further study of response shift at longer intervals after LC, after other health care interventions, and in patients with varying preoperative health status.

Reference bias: presentation of extreme health states prior to EQ-VAS improves health-related quality of life scores. a randomised cross-over trial

BACKGROUND

Clinical practice and clinical research has made a concerted effort to move beyond the use of clinical indicators alone and embrace patient focused care through the use of patient reported outcomes such as health-related quality of life. However, unless patients give consistent consideration to the health states that give meaning to measurement scales used to evaluate these constructs, longitudinal comparison of these measures may be invalid. This study aimed to investigate whether patients give consideration to a standard health state rating scale (EQ-VAS) and whether consideration of good and poor health state descriptors immediately changes their self-report.

METHODS

A randomised crossover trial was implemented amongst hospitalised older adults (n = 151). Patients were asked to consider descriptions of extremely good (Description-A) and poor (Description-B) health states. The EQ-VAS was administered as a self-report at baseline, after the first descriptors (A or B), then again after the remaining descriptors (B or A respectively). At baseline patients were also asked if they had considered either EQ-VAS anchors.

RESULTS

Overall 106/151 (70%) participants changed their self-evaluation by ≥ 5 points on the 100 point VAS, with a mean (SD) change of +4.5 (12) points (p < 0.001). A total of 74/151 (49%) participants did not consider the best health VAS anchor, of the 77 who did 59 (77%) thought the good health descriptors were more extreme (better) then they had previously considered. Similarly 85/151 (66%) participants did not consider the worst health anchor of the 66 who did 63 (95%) thought the poor health descriptors were more extreme (worse) then they had previously considered.

CONCLUSIONS

Health state self-reports may not be well considered. An immediate significant shift in response can be elicited by exposure to a mere description of an extreme health state despite no actual change in underlying health state occurring. Caution should be exercised in research and clinical settings when interpreting subjective patient reported outcomes that are dependent on brief anchors for meaning.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry (#ACTRN12607000606482) http://www.anzctr.org.au.

Case management poststroke did not induce response shift: the value of residuals

OBJECTIVES

To estimate the extent to which different methods of assessing response shift lead to different conclusions about its presence.

STUDY DESIGN AND SETTING

A reanalysis of a randomized clinical trial involving 190 persons poststroke to evaluate the effectiveness of a case management intervention aimed at assisting persons with stroke to make the transition from acute care to home.

RESULTS

Response shift was found to be nondifferential across groups and was therefore ruled out as an explanation for why the case management intervention showed no impact on patients' perceptions of their health. The results of a then-test did not show differential recalibration overtime. Factor analysis did not support reconceptualization response shift in either group but did find weak evidence for reprioritization response shift. An analysis of residuals indicated that approximately 50% of study subjects in each group experienced response shift that is probably close to reconceptualization.

CONCLUSION

A framework for assessing response shift was proposed and investigators planning trials of interventions targeting patient-reported outcomes should build into the trial methods for response shift investigation. In trials of interventions likely to induce response shift, before concluding about intervention-induced change, response shift should be ruled out by using a combination of design and statistical approaches.

Response shift in self-reported functional scores after knee microfracture for full thickness cartilage lesions

OBJECTIVE

Our aim was to determine whether response shift (RS), a change in the internal standards of a patient, occurs in patients treated for full thickness knee cartilage defects. We have also evaluated the effect of functional scores on patient satisfaction after surgery.

DESIGN

Self-administered questionnaires were used to evaluate pre- and post-operative and retrospective post-operative scores of 53 patients following knee microfracture. Patient satisfaction, Lysholm, Visual Analogue Scale (VAS) for pain and modified International Knee Documentation Committee (IKDC) scores were evaluated. RS (pre-test-then-test), unadjusted and adjusted treatment effects (UTE and ATE) and their effect sizes were calculated.

RESULTS

All four functional outcome measures had a positive RS. The effect size of the RS ranged from around 0.35 for the Lysholm and IKDC2 score to over 0.9 for the VAS pain score. Gender, age, smoking status and time since follow-up did not significantly affect the RS. RS did not differ significantly between the three patient satisfaction groups (P>0.05). Post-operative Lysholm and IKDC1 scores differed most significantly between the satisfaction groups.

CONCLUSIONS

All four scores had a significant shift, implying that patients thought they felt worse before the operation in retrospect than they did at the time. The traditional way of assessing treatment effect, difference between post-intervention and pre-intervention functional scores, may be confounded by change in the internal standards of the patient and should take this into account. RS did not affect the clinical interpretation in this case series. Patient-reported satisfaction after surgery is only related to post-operative scores.