The Guttman errors as a tool for response shift detection at subgroup and item levels

Response shift in parent-reported psychopathology in children with chronic physical illness

PURPOSE

Because physical-mental comorbidity in children is relatively common, this study tested for response shift (RS) in children with chronic physical illness using a parent-reported measure of child psychopathology.

METHODS

Data come from Multimorbidity in Children and Youth across Life-course (MY LIFE), a prospective study of n = 263 children aged 2-16 years with physical illness in Canada. Parents provided information on child psychopathology using the Ontario Child Health Study Emotional Behavioral Scales (OCHS-EBS) at baseline and 24 months. Oort's structural equation modeling was used to test for different forms of RS in parent-reported assessments between baseline and 24 months. Model fit was evaluated using root mean square error of approximation (RMSEA), comparative fit index (CFI), and standardized root mean residual (SRMR).

RESULTS

There were n = 215 (81.7%) children with complete data and were included in this analysis. Of these, n = 105 (48.8%) were female and the mean (SD) age was 9.4 (4.2) years. A two-factor measurement model provided good fit to the data [RMSEA (90% CI) = 0.05 (0.01, 0.10); CFI = 0.99; SRMR = 0.03]. Non-uniform recalibration RS was detected on the conduct disorder subscale of the OCHS-EBS. This RS effect had negligible impact on the longitudinal change in externalizing and internalizing disorders construct over time.

CONCLUSIONS

Response shift detected on the conduct disorder subscale of the OCHS-EBS, indicated that parents of children with physical illness may recalibrate their responses on child psychopathology over 24 months. Researchers and health professionals should be aware of RS when using the OCHS-EBS to assess child psychopathology over time.

Detection of response shift in health-related quality of life studies: a systematic review

Background

Response Shift (RS) refers to the idea that an individual may undergo changes in its health-related quality of life (HRQOL). If internal standard, values, or reconceptualization of HRQOL change over time, then answer to the same items by the same individuals may not be comparable over time. Traditional measures to evaluate RS is prone to bias and strong methodologies to study the existence of this phenomenon is required. The objective is to systematically identify, analyze, and synthesize the existing and recent evidence of statistical methods used for RS detection in HRQOL studies.

Methods

The analysis of selected studies between January 2010 and July 2020 was performed through a systematic review in MEDLINE/PubMed, Scopus, Web of Science, PsycINFO and Google Scholar databases. The search strategy used the terms “Health-Related Quality of Life” and “Response Shift” using the filters “Humans”, “Journal Article”, “English” and “2010/01/01–2020/07/31”. The search was made in August 2020.

Results

After considering the inclusion and exclusion criteria, from the total selected articles (675), 107 (15.9%) of the publications were included in the analysis. From these, 79 (71.0%) detected the existence of RS and 86 (80.4%) only used one detection method. The most used methods were Then Test (n = 41) and Oort’s Structural Equation Models (SEM) (n = 35). Other method used were Multiple Lineal Regression (n = 7), Mixed-Effect Regression (n = 6), Latent Trajectory Analysis (n = 6), Item Response Theory (n = 6), Logistics Regression (n = 5), Regression and Classification Trees (n = 4) and Relative Importance Method (n = 4). Most of these detected recalibration, including Then Test (n = 27), followed by Oort’s SEM that detected the higher combination of RS types: recalibration (n = 24), reprioritization (n = 13) and reconceptualization (n = 7).

Conclusions

There is a continuous interest of studying RS detection. Oort’s SEM becomes the most versatile method in its capability for detecting RS in all different types. Despite results from previous systematic reviews, same methods have been used during the last years. We observed the need to explore other alternative methods allowing same detection capacity with robust and highly precise methodology. The investigation on RS detection and types requires more study, therefore new opportunity grows to continue attending this phenomenon through a multidisciplinary perspective.

Evaluation of the link between the Guttman errors and response shift at the individual level

PURPOSE

Methods for response shift (RS) detection at the individual level could be of great interest when analyzing changes in patient-reported outcome data. Guttman errors (GEs), which measure discrepancies in respondents' answers compared to the average sample responses, might be useful for detecting RS at the individual level between two time points, as RS may induce an increase in the number of discrepancies over time. This study aims to establish the link between recalibration RS and the change in the number of GEs over time (denoted index [Formula: see text]) via simulations and explores the discriminating ability of this index.

METHODS

We simulated the responses of individuals affected or not affected by recalibration RS (defined as changes in the patients' standard of measurement) to determine whether simulated individuals with recalibration had a greater change in the number of GEs over time than individuals without recalibration. The effects of factors related to the sample, the questionnaire structure and recalibration were investigated. As an illustrative example, the change in the number of GEs was computed in patients suffering from eating disorders.

RESULTS

Within simulations, simulated individuals affected by recalibration had, on average, a greater change in the number of GEs over time than did individuals without RS. Some of the parameters related to the questionnaire structure and recalibration magnitude appeared to have substantial effects on the values of [Formula: see text]. Discriminating abilities appeared, however, globally low.

CONCLUSION

Some evidence of the link between recalibration and the change in GEs was found in this study. GEs could be a valuable nonparametric tool for RS detection at a more individual level, but further investigation is needed.

Comparison of structural equation modelling, item response theory and Rasch measurement theory-based methods for response shift detection at item level: A simulation study

When assessing change in patient-reported outcomes, the meaning in patients’ self-evaluations of the target construct is likely to change over time. Therefore, methods evaluating longitudinal measurement non-invariance or response shift at item-level were proposed, based on structural equation modelling or on item response theory. Methods coming from Rasch measurement theory could also be valuable. The lack of evaluation of these approaches prevents determining the best strategy to adopt. A simulation study was performed to compare and evaluate the performance of structural equation modelling, item response theory and Rasch measurement theory approaches for item-level response shift detection. Performances of these three methods in different situations were evaluated with the rate of false detection of response shift (when response shift was not simulated) and the rate of correct response shift detection (when response shift was simulated). The Rasch measurement theory-based method performs better than the structural equation modelling and item response theory-based methods when recalibration was simulated. Consequently, the Rasch measurement theory-based approach should be preferred for studies investigating only recalibration response shift at item-level. For structural equation modelling and item response theory, the low rates of reprioritization detection raise issues on the potential different meaning and interpretation of reprioritization at item-level.

Scoping review of response shift methods: current reporting practices and recommendations

BACKGROUND

Response shift (RS) has been defined as a change in the meaning of an individual's self-evaluation of his/her health status and quality of life. Several statistical model- and design-based methods have been developed to test for RS in longitudinal data. We reviewed the uptake of these methods in patient-reported outcomes (PRO) literature.

METHODS

CINHAHL, EMBASE, Medline, ProQuest, PsycINFO, and Web of Science were searched to identify English-language articles about RS published until 2016. Data on year and country of publication, PRO measure adopted, RS detection method, type of RS detected, and testing of underlying model assumptions were extracted from the included articles.

RESULTS

Of the 1032 articles identified, 101 (9.8%) articles were included in the study. While 54.5 of the articles reported on the Then-test, 30.7% of the articles reported on Oort's or Schmitt's structural equation modeling (SEM) procedure. Newer RS detection methods, such as relative importance analysis and random forest regression, have been used less frequently. Less than 25% reported on testing the assumptions underlying the adopted RS detection method(s).

CONCLUSIONS

Despite rapid methodological advancements in RS research, this review highlights the need for further research about RS detection methods for complex longitudinal data and standardized reporting guidelines.

Identifying patterns of adaptation in breast cancer patients with cancer-related fatigue using response shift analyses at subgroup level

Fatigue is the most prevalent symptom in breast cancer. It might be perceived differently among patients over time as a consequence of the differing patients’ adaptation and psychological adjustment to their cancer experience which can be related to response shift (RS). RS analyses can provide important insights on patients’ adaptation to cancer but it is usually assumed that RS occurs in the same way in all individuals which is unrealistic. This study aimed to identify patients’ subgroups in which different RS effects on self‐reported fatigue could occur over time using a combination of methods for manifest and latent variables. The FATSEIN study comprised 466 breast cancer patients followed over a 2‐year period. Fatigue was measured with the Multidimensional Fatigue Inventory questionnaire (MFI‐20) during 10 visits. A novel combination of Mixed Models, Growth Mixture Modeling, and Structural Equation Modeling was used to assess the occurrence of RS in fatigue changes to identify subgroups displaying different RS patterns over time. An increase in fatigue was evidenced over the 8‐month follow‐up, followed by a decrease between the 8‐ and 24‐month. Four latent classes of patients were identified. Different RS patterns were detected in all latent classes between the inclusion and 8 months (last cycle of chemotherapy). No RS was evidenced between 8‐ and 24‐month. Several RS effects were evidenced in different groups of patients. Women seemed to adapt differently to their treatment and breast cancer experience possibly indicating differing needs for medical/psychological support.