What too strict a method obscures about the validity of outcome measures

How do we know whether treatment has failed? Paradoxical outcomes in counseling with young people

Background

In both routine practice contexts and research studies, evidence from standardized self-report symptom measures, administered pre- and post-treatment, is predominantly used to determine whether psychotherapy has been successful. Understanding the nature of unsuccessful psychotherapy requires an ability to evaluate the credibility of outcome data generated by such techniques. An important body of research has identified discrepancies between outcomes assessed through symptom measures and those obtained from other sources. However, not enough is known about the extent to which such paradoxical outcomes exist.

Objective

This study analyzes the relationship between outcomes, as assessed by a standardized self-report measure, and as assessed by ratings of young people's descriptions of change at post-counseling interviews.

Methods

Participants were 50 young people (13-16 years old) who had taken part in a trial of up to 10 weeks of school-based humanistic counseling. Our primary standardized measure was the Young Person's CORE (YP-CORE). To assess young people's experiences of counseling change, three independent raters scrutinized transcripts of post-counseling interviews, and scored levels of helpfulness on a 1 (Not at all helpful) to 10 (Extremely helpful) scale. Inter-rater reliabilities were 0.94 (Cronbach's Alpha) and 0.96 (McDonald's Omega). Sensitivity analyses were conducted to explore relationships between helpfulness ratings and other outcome measures, i.e., satisfaction with counseling (ESQ) and the Goal-Based-Outcome Tool (GBO), and process measures, i.e., the Working Alliance Inventory (WAI-S) and the Barret Lennard Relationship Inventory (BLRI).

Results

Multilevel analysis indicated that helpfulness ratings were not significantly associated with changes in YP-CORE scores. Analyzed categorically, 38% of those showing reliable improvement on the standardized measure were below the median for self-described helpfulness, and 47% of those not showing reliable change were at or above the median for self-described helpfulness. Sensitivity analyses demonstrated closer correlations between helpfulness ratings and other outcome measures (ESQ and GBO), and between helpfulness ratings and process measures (WAI-S and BLRI).

Discussion

Our results raise questions about reliance on symptom change outcome measures for defining treatment success and failure, given their disparity with clients' own descriptions of the helpfulness of therapy. Implications for practice and research are discussed.

Life, complexity, and psychotherapy: A systemic perspective on the elusive construct of extra-therapeutic factors

STUDY AIMS

Despite being mentioned in well-known models of psychotherapeutic change, the concept of extra-therapeutic factors seems to have left the scene across time, eaten away by the progressive refinement of the construct of common factors. Aim of the present study is to better understanding the historical evolution of the concept of extra-therapeutic factors and its importance for psychotherapy today.

METHODS

This is a position paper based on a literature review on extra-therapeutic factors and psychotherapy outcome and process.

DISCUSSION

There is growing evidence of the decisive role of clients' and therapists' characteristics, as well as of the importance of therapeutic alliances and relationships in promoting change. Within this context, the concept of extra-therapeutic factors still deserves proper attention, rather than being relegated to a residual and vanishing category.

CONCLUSION

A renewed interest on extra-therapeutic factors could reinvigorate the debate over the relationship between psychotherapy and life contexts and conditions. In terms of complexity and systems thinking nothing, a priori, really falls outside the realm of psychotherapy and, in turn, psychotherapy is not a practice that could be abstracted from the context in which it is practiced or from clients' and therapists' lives.

It turned into something else: patients' long-term experiences of transitions to or from telepsychotherapy during the COVID-19 pandemic

Introduction

The shift from in-person therapy to telepsychotherapy during the COVID-19 pandemic was unprepared for, sudden, and inevitable. This study explored patients' long-term experiences of transitions to telepsychotherapy and back to the office.

Methods

Data were collected approximately two years after the declaration of COVID-19 as a pandemic. Eleven patients were interviewed (nine women and two men, aged 28 to 56, six in psychodynamic psychotherapy, five in CBT). Treatments switched between in-person and video/telephone sessions. Interview transcripts were analyzed applying the qualitative methodology of inductive thematic analysis.

Results

(1) The patients experienced the process in telepsychotherapy as impeded. Interventions were difficult to understand and lost impact. Routines surrounding the therapy sessions were lost. Conversations were less serious and lost direction. (2) Understanding was made more difficult when the nuances of non-verbal communication were lost. (3) The emotional relationship was altered. Remote therapy was perceived as something different from regular therapy, and once back in the therapy room, the patients felt that therapy started anew. The emotional presence was experienced as weakened, but some of the patients found expressing their feelings easier in the absence of bodily co-presence. According to the patients, in-person presence contributed to their security and trust, whereas they felt that the therapists were different when working remotely, behaving in a more easygoing and familiar way, as well as more solution-focused, supportive and unprofessional, less understanding and less therapeutic. Despite this, (4) telepsychotherapy also gave the patients an opportunity to take therapy with them into their everyday lives.

Discussion

The results suggest that in the long run, remote psychotherapy was seen as a good enough alternative when needed. The present study indicates that format alternations have an impact on which interventions can be implemented, which can have important implications for psychotherapy training and supervision in an era when telepsychotherapy is becoming increasingly common.

The Lübeck Medication Satisfaction Questionnaire—A Novel Measurement Tool for Therapy Satisfaction

Background: Therapy satisfaction is widely considered an important aspect of clinical care. Still, there are currently no freely available questionnaires for its measurement. We developed the Lübeck Medication Satisfaction Questionnaire (LMSQ) for that purpose. Here, we present its content and psychometric properties. Methods: The LMSQ was validated on 86 patients in a single center study. The Kaiser-Meyer-Olkin test, confirmatory factor analysis, covariance analysis, and a test of exact fit were performed. Reliability was tested using Cronbach’s α and McDonald’s ω. The relationship to other patient-reported outcomes was tested using Pearson’s correlation. Results: Confirmatory factors analysis yielded moderate factor loadings with p < 0.001 in all subscales. Reliability was adequate (α = 0.857 and ω = 0.872). Model fitness was excellent in all tests. The LMSQ was positively correlated with medication adherence (r = 0.603, p < 0.001) and most dimensions of health literacy. Conclusions: The LMSQ possesses adequate psychometric properties for its purpose. We recommend further validation in a more diverse patient collective.

Rigorous idiography: Exploring subjective and idiographic data with rigorous methods-The method of derangements

Psychological research often seeks general rules applying across individuals, an aim that is in tension with examining that which is unique to any individual. There are general statistical regularities across individuals' subjective self-report which enable much psychology and psychotherapy research to combine data from self-report questionnaire responses with statistical and psychometric methods to create a fundamental part of Cronbach and Meehl's foundational nomological networks of validity. However, these methods only apply when most participants answer the same questions on measures creating nomothetic data and this has led to a neglect of idiographic data. This paper reviews a method of analysis of idiographic data, of "rigorous idiography": the method of derangements. This is a remarkably simple statistical test of whether purely idiographic data convey reliable information. We show how the method appeared to become stuck in a bibliometric backwater but we expand on its potential for research and practise and hope it will be taken up and used correctly and more widely.

Loss of Rituals, Boundaries, and Relationship: Patient Experiences of Transition to Telepsychotherapy Following the Onset of COVID-19 Pandemic

Telepsychotherapy is an increasingly common way of conducting psychotherapy. Previous research has shown that patients usually have positive experiences of online therapy, however, with large individual differences. The aim of this study was to explore patients’ experiences of transition from in-person psychotherapy sessions to telepsychotherapy during the COVID-19 pandemic, as well as variation in the experiences with regard to the patients’ personality orientation. Seven psychotherapy patients in Sweden were interviewed and the transcripts were analyzed using thematic analysis. Additionally, the participants were asked to rate their dissatisfaction/satisfaction with the transition, how hindering/helpful the transition was, and how unsafe/safe they felt after the transition in comparison to before. Personality orientation on relatedness or self-definition was assessed applying a self-assessment instrument (Prototype Matching of Anaclitic-Introjective Personality Configuration; PMAI). The participants experienced telepsychotherapy as qualitatively different from in-person psychotherapy. They reported several essential losses: the rituals surrounding therapy sessions were lost, including the transitional time and space between their every-day life and the therapy sessions, less therapeutic work was done, the therapists could lose their therapeutic stance, the sense of rapport was impaired, and the patients felt less open and emotionally available. On the other hand, some patients could feel freer online. As six of the participants had an anaclitic personality orientation, the present study could especially contribute to the understanding of how patients with strong affiliative needs and fear of abandonment experience the transition to meeting their therapists via communication technology. The participants’ self-ratings showed that they were only marginally dissatisfied with the transition and experienced the transition as slightly hindering, whereas they felt rather safe after the transition, indicating low concordance between qualitative and quantitative evaluations. New studies are needed to explore the introjective patients’ experiences of the transition. An essential topic is also to collect evidence and to test how the impaired sense of rapport when using communication technology can be remedied by adequate, patient-tailored interventions, a topic that has to be included in psychotherapy education and training.