Power analyses for measurement model misspecification and response shift detection with structural equation modeling

PURPOSE

Statistical power for response shift detection with structural equation modeling (SEM) is currently underreported. The present paper addresses this issue by providing worked-out examples and syntaxes of power calculations relevant for the statistical tests associated with the SEM approach for response shift detection.

METHODS

Power calculations and related sample-size requirements are illustrated for two modelling goals: (1) to detect misspecification in the measurement model, and (2) to detect response shift. Power analyses for hypotheses regarding (exact) overall model fit and the presence of response shift are demonstrated in a step-by-step manner. The freely available and user-friendly R-package lavaan and shiny-app 'power4SEM' are used for the calculations.

RESULTS

Using the SF-36 as an example, we illustrate the specification of null-hypothesis (H0) and alternative hypothesis (H1) models to calculate chi-square based power for the test on overall model fit, the omnibus test on response shift, and the specific test on response shift. For example, we show that a sample size of 506 is needed to reject an incorrectly specified measurement model, when the actual model has two-medium sized cross loadings. We also illustrate power calculation based on the RMSEA index for approximate fit, where H0 and H1 are defined in terms of RMSEA-values.

CONCLUSION

By providing accessible resources to perform power analyses and emphasizing the different power analyses associated with different modeling goals, we hope to facilitate the uptake of power analyses for response shift detection with SEM and thereby enhance the stringency of response shift research.

Re-evaluating randomized clinical trials of psychological interventions: Impact of response shift on the interpretation of trial results

Background

Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. However, comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called ‘response shift’. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients’ frames of reference.

Aims

The overall aim is to gain insight into the general relevance of response shift for psychological health intervention research. Specifically, the aim is to re-analyse data of published randomized controlled trials (RCTs) investigating the effectiveness of psychological interventions targeting different health aspects, to assess (1) the occurrence of response shift, (2) the impact of response shift on interpretation of treatment effectiveness, and (3) the predictive role of clinical and background variables for detected response shift.

Method

We re-analysed data from RCTs on guided internet delivered cognitive behavioural treatment (CBT) for insomnia in the general population with and without elevated depressive symptoms, an RCT on meaning-centred group psychotherapy targeting personal meaning for cancer survivors, and an RCT on internet-based CBT treatment for persons with diabetes with elevated depressive symptoms. Structural equation modelling was used to test the three objectives.

Results

We found indications of response shift in the intervention groups of all analysed datasets. However, results were mixed, as response shift was also indicated in some of the control groups, albeit to a lesser extent or in opposite direction. Overall, the detected response shifts only marginally impacted trial results. Relations with selected clinical and background variables helped the interpretation of detected effects and their possible mechanisms.

Conclusion

This study showed that response shift effects can occur as a result of psychological health interventions. Response shift did not influence the overall interpretation of trial results, but provide insight into differential treatment effectiveness for specific symptoms and/or domains that can be clinically meaningful.

Using structural equation modeling to investigate change and response shift in patient-reported outcomes: practical considerations and recommendations

BACKGROUND

Patient-reported outcomes (PROs) are of increasing importance for health-care evaluations. However, the interpretation of change in PROs may be obfuscated due to changes in the meaning of the self-evaluation, i.e., response shift. Structural equation modeling (SEM) is the most widely used statistical approach for the investigation of response shift. Yet, non-technical descriptions of SEM for response shift investigation are lacking. Moreover, application of SEM is not straightforward and requires sequential decision-making practices that have not received much attention in the literature.

AIMS

To stimulate appropriate applications and interpretations of SEM for the investigation of response shift, the current paper aims to (1) provide an accessible description of the SEM operationalizations of change that are relevant for response shift investigation; (2) discuss practical considerations in applying SEM; and (3) provide guidelines and recommendations for researchers who want to use SEM for the investigation and interpretation of change and response shift in PROs.

CONCLUSION

Appropriate applications and interpretations of SEM for the detection of response shift will help to improve our understanding of response shift phenomena and thus change in PROs. Better understanding of patients' perceived health trajectories will ultimately help to adopt more effective treatments and thus enhance patients' wellbeing.

Quality of Life During and After Completion of Neoadjuvant Chemoradiotherapy for Esophageal and Junctional Cancer

Background

The course of health-related quality of life (HRQOL) during and after completion of neoadjuvant chemoradiotherapy (nCRT) for esophageal or junctional carcinoma is unknown.

Methods

This study was a multicenter prospective cohort investigation. Patients with esophageal or cancer to be treated with nCRT plus esophagectomy were eligible for inclusion in the study. The HRQOL of the patients was measured with European Organization for Research and Treatment of Cancer QLQ-C30, QLQ-OG25, and QLQ-CIPN20 questionnaires before and during nCRT, then 2, 4, 6, 8, 10, 12, 14, and 16 weeks after nCRT and before surgery. Predefined end points were based on the hypothesized impact of nCRT. The primary end points were physical functioning, odynophagia, and sensory symptoms. The secondary end points were global quality of life, fatigue, weight loss, and motor symptoms. Mixed modeling analysis was used to evaluate changes over time.

Results

Of 106 eligible patients, 96 (91%) were included in the study. The rate of questionnaires returned ranged from 94% to 99% until week 12, then dropped to 78% in week 16 after nCRT. A negative impact of nCRT on all HRQOL end points was observed during the last cycle of nCRT (all p < 0.001) and 2 weeks after nCRT (all p < 0.001). Physical functioning, odynophagia, and sensory symptoms were restored to pretreatment levels respectively 8, 4, and 6 weeks after nCRT. The secondary end points were restored to baseline levels 4–6 weeks after nCRT. Odynophagia, fatigue, and weight loss improved after nCRT compared with baseline levels at respectively 6 (p < 0.001), 16 (p = 0.001), and 12 weeks (p < 0.001).

Conclusion

After completion of nCRT for esophageal cancer, HRQOL decreases significantly, but all HRQOL end points are restored to baseline levels within 8 weeks. Odynophagia, fatigue, and weight loss improved 6–16 weeks after nCRT compared with baseline levels.

Electronic supplementary material

The online version of this article (10.1245/s10434-019-07779-w) contains supplementary material, which is available to authorized users.

The Analysis of Multivariate Longitudinal Data: An Instructive Application of the Longitudinal Three-Mode Model

Structural equation modeling is a common technique to assess change in longitudinal designs. However, these models can become of unmanageable size with many measurement occasions. One solution is the imposition of Kronecker product restrictions to model the multivariate longitudinal structure of the data. The resulting longitudinal three-mode models (L3MMs) are very parsimonious and have attractive interpretation. This paper provides an instructive description of L3MMs. The models are applied to health-related quality of life (HRQL) data obtained from 682 patients with painful bone metastasis, with eight measurements at 13 occasions; before and every week after treatment with radiotherapy. We explain (1) how the imposition of Kronecker product restrictions can be used to model the multivariate longitudinal structure of the data, (2) how to interpret the Kronecker product restrictions and the resulting L3MM parameters, and (3) how to test substantive hypotheses in L3MMs. In addition, we discuss the challenges for the evaluation of (differences in) fit of these complex and parsimonious models. The L3MM restrictions lead to parsimonious models and provide insight in the change patterns of relationships between variables in addition to the general patterns of change. The L3MM thus provides a convenient model for multivariate longitudinal data, as it not only facilitates the analysis of complex longitudinal data but also the substantive interpretation of the dynamics of change.