Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients

The clinical effectiveness of a self-management intervention for patients with persistent depressive disorder and their partners/caregivers: results from a multicenter, pragmatic randomized controlled trial

BACKGROUND

Persistent depressive disorder (PDD) is prevalent and debilitating. For patients with PDD, psychiatric rehabilitation using self-management interventions is advised as the next therapeutic step after multiple unsuccessful treatment attempts. The "Patient and Partner Education Program for All Chronic Diseases" (PPEP4All) is a brief, structured self-management program that focuses on functional recovery for patients and their partners/caregivers. In chronic somatic disorder populations, PPEP4All has already been shown to be clinically effective. We examined whether PPEP4All adapted for PDD (PPEP4All-PDD, nine weekly group or individual sessions) is also clinically effective for adults/elderly with PDD and their partners/caregivers compared to care-as-usual (CAU) in specialized mental healthcare.

METHODS

In this mixed-method multicenter pragmatic randomized controlled trial, 70 patients with PDD and 14 partners/caregivers were allocated to either PPEP4All-PDD (patients, n = 37; partners/caregivers, n = 14) or CAU (patients, n = 33; partners/caregivers, not included) and completed questionnaires at 0, 3, 6, and 12 months regarding depressive symptoms, psychopathology, psychosocial burden, mental resilience, and happiness/well-being. Qualitative data were collected regarding treatment satisfaction. Data were analyzed using mixed model analyses and an intention-to-treat (ITT) approach.

RESULTS

There was no statistically significant difference in any outcome regarding clinical effectiveness between PPEP4All-PDD and CAU. Subgroup analysis for depressive symptoms did not show any interaction effect for any subgroup. Although 78% of participants recommended PPEP4All-PDD, there was no difference in treatment satisfaction between PPEP4All-PDD (score = 6.6; SD = 1.7) and CAU (score = 7.6; SD = 1.2), p = 0.06.

CONCLUSION

Although depressive symptoms did not improve relative to CAU, this only confirmed that treatment for patients with treatment-resistant PDD should move from symptom reduction to functional recovery. Also, functional recovery may be reflected in other outcomes than psychosocial burden, such as self-empowerment, in patients with treatment-resistant PDD. Future research on PPEP4All-PDD could focus on a longer-term program and/or online program that may also be offered earlier in the treatment process as an empowerment intervention.  TRIAL REGISTRATION: Netherlands Trial Register Identifier NL5818. Registered on 20 July 2016 https://clinicaltrialregister.nl/nl/trial/20302.

Development and Validation of Oral Health-Related Quality of Life Scale for Patients Undergoing Endodontic Treatment (OHQE) for Irreversible Pulpitis

An oral health-related quality of life measure specific to patients undergoing endodontic treatment has not been developed. This study aimed to validate the oral health-related quality of life scale for patients undergoing endodontic treatment (OHQE) for irreversible pulpitis, comprised of 42 questions. Sixty-two patients with irreversible pulpitis, comprising 23 (37.1%) males and 39 (62.9%) females, were enrolled between August 2022 and February 2023. Data were collected at three time points: pretreatment, post-treatment, and at the second week post-treatment. Factor analysis revealed physical, psychological, and expectations as subscales of OHQE. Cronbach's alpha coefficients ranged from 0.87 to 0.95 for each subscale. Each subscale of the General Oral Health Assessment Index (GOHAI) was moderately correlated with the OHQE subscales. Good-poor analysis revealed a significant difference between the high-scoring and low-scoring groups for each OHQE subscale. The intraclass correlation coefficients of the OHQE subscales ranged from 0.89 to 0.95. Multivariate linear regression analysis revealed a significant correlation between the pretreatment and post-treatment psychological factors (p < 0.05). Thus, OHQE will help researchers and policymakers understand the impact of oral health on the quality of life of patients with irreversible pulpitis undergoing endodontic treatment. OHQE could contribute to the appropriate planning, treatment decisions, and management of dental treatment.

Translation and cross-cultural validation of the Turkish, Moroccan Arabic and Moroccan Berber versions of the 48-item Symptom Questionnaire (SQ-48)

We aimed to validate cross-culturally the Turkish, Moroccan Arabic and Moroccan Berber versions of the 48-item Symptom Questionnaire (SQ-48). Its psychometric properties were assessed in four samples: patients (n = 150) and controls (n = 103) with Turkish or Moroccan origins (n = 103) and patients (n = 189) and controls (n = 463) with native Dutch origins. Internal consistency and discriminatory power of SQ-48 subscales across groups were adequate to high. However, immigrant groups scored on average higher than Dutch native groups, but there was full configural, metric and partial scalar invariance in the immigrant groups. Although the SQ-48 is a valid measure of psychopathology in immigrant groups of Turkish and Moroccan origins, their cut-off values should likely be higher compared to natives.

A comparison of the multilevel MIMIC model to the multilevel regression and mixed ANOVA model for the estimation and testing of a cross-level interaction effect: A simulation study

When observing data on a patient-reported outcome measure in, for example, clinical trials, the variables observed are often correlated and intended to measure a latent variable. In addition, such data are also often characterized by a hierarchical structure, meaning that the outcome is repeatedly measured within patients. To analyze such data, it is important to use an appropriate statistical model, such as structural equation modeling (SEM). However, researchers may rely on simpler statistical models that are applied to an aggregated data structure. For example, correlated variables are combined into one sum score that approximates a latent variable. This may have implications when, for example, the sum score consists of indicators that relate differently to the latent variable being measured. This study compares three models that can be applied to analyze such data: the multilevel multiple indicators multiple causes (ML-MIMIC) model, a univariate multilevel model, and a mixed analysis of variance (ANOVA) model. The focus is on the estimation of a cross-level interaction effect that presents the difference over time on the patient-reported outcome between two treatment groups. The ML-MIMIC model is an SEM-type model that considers the relationship between the indicators and the latent variable in a multilevel setting, whereas the univariate multilevel and mixed ANOVA model rely on sum scores to approximate the latent variable. In addition, the mixed ANOVA model uses aggregated second-level means as outcome. This study showed that the ML-MIMIC model produced unbiased cross-level interaction effect estimates when the relationships between the indicators and the latent variable being measured varied across indicators. In contrast, under similar conditions, the univariate multilevel and mixed ANOVA model underestimated the cross-level interaction effect.

Impact of response shift effects in the assessment of self-reported depression during treatment: Insights from a rTMS versus Venlafaxine randomized controlled trial

PURPOSE

Patient-Reported Outcomes are essential to properly assess treatment effectiveness in randomized clinical trial (RCT) for Major Depressive Disorder (MDD). MDD self-assessment may vary over time depending on change in the meaning of patients' self-evaluation of depression, i.e. Response Shift (RS). Our aim was to investigate RS and its impact on different depression domains in a clinical trial comparing rTMS versus Venlafaxine.

METHODS

The occurrence and type of RS was determined using Structural Equation Modeling applied to change over time in 3 domains (Sad Mood, Performance Impairment, Negative Self-Reference) of the short-form Beck Depression Inventory (BDI-13) in a secondary analysis of a RCT on 170 patients with MDD treated by rTMS, venlafaxine or both.

RESULTS

RS was evidenced in the venlafaxine group in the Negative Self-Reference and Sad Mood domains.

CONCLUSION

RS effects differed between treatment arms in self-reported depression domains in patients with MDD. Ignoring RS would have led to a slight underestimation of depression improvement, depending on treatment group. Further investigations of RS and advancing new methods are needed to better inform decision making based on Patient-Reported Outcomes.

The Effect of Noninvariance on the Estimation of the Mediated Effect in the Two-Wave Mediation Model

The two-wave mediation model is the most suitable model for examining mediation effects in a randomized intervention and includes measures taken at pretest and posttest. When using self-report measures, the meaning of responses may change for the treatment group over the course of the intervention and result in noninvariance across groups at posttest, a phenomenon referred to as response shift. We investigate how the mediated effect would be impacted by noninvariance when using sum scores (i.e., assuming invariance). In a Monte Carlo simulation study, the magnitude and proportion of items that had noninvariant intercepts, the direction of noninvariance, number of items, effect size of the mediated effect and sample size were varied. Results showed increased Type I and Type II errors due to a biased estimate of the intervention effect on the mediator resulting from noninvariance. Thus, measurement noninvariance could lead to erroneous conclusions about the process underlying the intervention.

If it's information, it's not "bias": a scoping review and proposed nomenclature for future response-shift research

BACKGROUND

The growth in response-shift methods has enabled a stronger empirical foundation to investigate response-shift phenomena in quality-of-life (QOL) research; but many of these methods utilize certain language in framing the research question(s) and interpreting results that treats response-shift effects as "bias," "noise," "nuisance," or otherwise warranting removal from the results rather than as information that matters. The present project will describe the various ways in which researchers have framed the questions for investigating response-shift issues and interpreted the findings, and will develop a nomenclature for such that highlights the important information about resilience reflected by response-shift findings.

METHODS

A scoping review was done of the QOL and response-shift literature (n = 1100 articles) from 1963 to 2020. After culling only empirical response-shift articles, raters characterized how investigators framed and interpreted study research questions (n = 164 articles).

RESULTS

Of 10 methods used, papers using four of them utilized terms like "bias" and aimed to remove response-shift effects to reveal "true change." Yet, the investigators' reflections on their own conclusions suggested that they do not truly believe that response shift is error to be removed. A structured nomenclature is proposed for discussing response-shift results in a range of research contexts and response-shift detection methods.

CONCLUSIONS

It is time for a concerted and focused effort to change the nomenclature of those methods that demonstrated this misinterpretation. Only by framing and interpreting response shift as information, not bias, can we improve our understanding and methods to help to distill outcomes with and without response-shift effects.

The clinical and cost-effectiveness of a self-management intervention for patients with persistent depressive disorder and their partners/caregivers: study protocol of a multicenter pragmatic randomized controlled trial

BACKGROUND

After regular treatment, patients with persistent depressive disorder (PDD) may remain in specialized psychiatric outpatient care without achieving remission. Lacking other options, these patients often receive long-term, non-protocolized care as usual (CAU) that does not involve the partner/caregiver of the patient. Although the revised depression treatment guidelines suggest focusing on psychiatric rehabilitation and self-management as the next treatment step for PDD, an evidence-based cost-effective self-management protocol for PDD is lacking. This study investigates the "Patient and Partner Education Program for All Chronic Illnesses" (PPEP4All) as a brief self-management protocol that could lead to lower costs, higher quality of life, and less disease burden in PDD patients and their partners/caregivers.

METHODS

Presented is the rationale and methods of a multicenter pragmatic randomized controlled trial to evaluate the clinical efficacy and cost-effectiveness of PPEP4All for patients with PDD and their partners/caregivers. In accordance with current recommendations, a mixed methods research approach is used with both quantitative and qualitative data. A total of 178 eligible outpatients with PDD and their partners/caregivers are recruited and randomized to either PPEP4All or CAU. Those assigned to PPEP4All receive nine weekly self-management sessions with a trained PPEP4All therapist. Primary and secondary outcome measurements are at 0, 3, 6, and 12 months.

DISCUSSION

This project will result in the implementation of a self-management intervention for patients with PDD, meeting an urgent need in mental healthcare. Using PPEP4All can optimize the quality and efficiency of care for both patients with PDD and their partners/caregivers.

TRIAL REGISTRATION

Netherlands Trial Register Identifier NTR5973 . Registered on 20 July 2016.

Re-evaluating randomized clinical trials of psychological interventions: Impact of response shift on the interpretation of trial results

Background

Effectiveness of psychological treatment is often assessed using patient-reported health evaluations. However, comparison of such scores over time can be hampered due to a change in the meaning of self-evaluations, called ‘response shift’. Insight into the occurrence of response shift seems especially relevant in the context of psychological interventions, as they often purposefully intend to change patients’ frames of reference.

Aims

The overall aim is to gain insight into the general relevance of response shift for psychological health intervention research. Specifically, the aim is to re-analyse data of published randomized controlled trials (RCTs) investigating the effectiveness of psychological interventions targeting different health aspects, to assess (1) the occurrence of response shift, (2) the impact of response shift on interpretation of treatment effectiveness, and (3) the predictive role of clinical and background variables for detected response shift.

Method

We re-analysed data from RCTs on guided internet delivered cognitive behavioural treatment (CBT) for insomnia in the general population with and without elevated depressive symptoms, an RCT on meaning-centred group psychotherapy targeting personal meaning for cancer survivors, and an RCT on internet-based CBT treatment for persons with diabetes with elevated depressive symptoms. Structural equation modelling was used to test the three objectives.

Results

We found indications of response shift in the intervention groups of all analysed datasets. However, results were mixed, as response shift was also indicated in some of the control groups, albeit to a lesser extent or in opposite direction. Overall, the detected response shifts only marginally impacted trial results. Relations with selected clinical and background variables helped the interpretation of detected effects and their possible mechanisms.

Conclusion

This study showed that response shift effects can occur as a result of psychological health interventions. Response shift did not influence the overall interpretation of trial results, but provide insight into differential treatment effectiveness for specific symptoms and/or domains that can be clinically meaningful.

The Spanish Version of the Sexual Opinion Survey (SOS-6): Evidence of Validity of a Short Version

The aim of this work was to provide validity evidences of the short Spanish version of the Sexual Opinion Survey (SOS-6). First, a psychometric study was performed to examine the factorial invariance of the scale across different groups. A sample of 1,500 heterosexual adults (50% men, 50% women) from the general Spanish population, aged 18 to 80, was examined. Results showed that the unidimensional model of the SOS-6 is invariant across sex, age range, relationship status, and educational level. Secondly, a laboratory study was performed to determine the relationship between erotophilia and objective and subjective sexual arousal in response to sexual stimuli. The sample consisted of 130 young Spanish university students (46.92% men, 53.08% women) with heterosexual orientation. In men, erotophilia was related to subjective sexual arousal; in women, erotophilia was related to subjective sexual arousal and estimation of genital sensations. In conclusion, this work provides validity evidences of the short version of the SOS-6 and describes the factorial equivalence of the scale across groups and its concurrent validity.

Testing for response shift in treatment evaluation of change in self-reported psychopathology amongst secondary psychiatric care outpatients

OBJECTIVES

If patients change their perspective due to treatment, this may alter the way they conceptualize, prioritize, or calibrate questionnaire items. These psychological changes, also called "response shifts," may pose a threat to the measurement of therapeutic change in patients. Therefore, it is important to test the occurrence of response shift in patients across their treatment.

METHODS

This study focused on self-reported psychological distress/psychopathology in a naturalistic sample of 206 psychiatric outpatients. Longitudinal measurement invariance tests were computed across treatment in order to detect response shifts.

RESULTS

Compared with before treatment, post-treatment psychopathology scores showed an increase in model fit and factor loading, suggesting that symptoms became more coherently interrelated within their psychopathology domains. Reconceptualization (depression/mood) and reprioritization (somatic and cognitive problems) response shift types were found in several items. We found no recalibration response shift.

CONCLUSION

This study provides further evidence that response shift can occur in adult psychiatric patients across their mental health treatment. Future research is needed to determine whether response shift implies an unwanted potential bias in treatment evaluation or a desired cognitive change intended by treatment.