Using clinical significance in psychotherapy outcome research: The need for a common procedure and validity data

Cognitive change before sudden gains in cognitive behavioural therapy for panic disorder

BACKGROUND

Sudden gains occur in a range of disorders and treatments and are of clinical and theoretical significance if they can shed light on therapeutic change processes. This study investigated the relationship between sudden gains in panic symptoms and preceding cognitive change during cognitive behavioural therapy (CBT) for panic disorder.

METHOD

Participants with panic disorder completed in session measures of panic symptoms and catastrophic cognitions. Independent samples t-tests were used to compare the post-treatment score of those who met criteria for one or more sudden gain during treatment with those who did not, and to compare within-session cognitive change between pre-sudden gain sessions and the previous (control) session.

RESULTS

Twenty-two (42%) of 53 participants experienced a sudden gain during treatment. Participants demonstrating a sudden gain showed more improvement in panic symptoms from pre- to post-treatment than those without a sudden gain. The within-session cognitive change score in the pre-gain session was significantly greater than in the control session.

CONCLUSIONS

Sudden gains occurred in individual CBT for panic disorder and within-session cognitive change was associated with sudden gains. This is consistent with the cognitive model of panic disorder and highlights how sudden gains can help to identify key change processes.

Intolerance-of-uncertainty therapy versus metacognitive therapy for generalized anxiety disorder in primary health care: A randomized controlled pilot trial

OBJECTIVE

This randomized controlled pilot study investigated the feasibility of a future full-scale RCT to compare the effects of intolerance-of-uncertainty therapy (IUT) and metacognitive therapy (MCT) in primary health care patients with generalized anxiety disorder (GAD). Preliminary treatment effects were also evaluated.

MATERIALS AND METHODS

64 patients with GAD at a large primary health care center in Stockholm, Sweden, were randomized to IUT or MCT. Feasibility outcomes included participant recruitment and retention, willingness to receive psychological treatment, and therapists' competence in and adherence to treatment protocols. Self-reported scales were used to assess treatment outcomes, including worry, depression, functional impairment, and quality of life.

RESULTS

Recruitment was satisfactory, and dropout was low. On a scale from 0 to 6, participants were satisfied with participating in the study (M = 5.17, SD = 1.09). Following brief training, therapists' competence was rated as moderate, and adherence was rated as weak to moderate. From pre- to post-treatment, reductions on the primary treatment outcome measure of worry were of a large effect size and statistically significant in both the IUT and MCT conditions (Cohen's d for IUT = -2.69, 95% confidence interval [-3.63, -1.76] and d for MCT = -3.78 [-4.68, -2.90]). The between-group effect size from pre- to post-treatment was large and statistically significant (d = -2.03 [-3.31, -0.75]), in favor of the MCT condition.

CONCLUSION

It is feasible to carry out a full-scale RCT to compare the effects of IUT to MCT for patients with GAD in primary health care. Both protocols seem effective, and MCT seems superior to IUT, but a full-scale RCT is needed to confirm these conclusions.

TRIAL REGISTRATION

ClinicalTrials.gov (no. NCT03621371).

Pain education and pain management skills in virtual reality in the treatment of chronic low back pain: A multiple baseline single-case experimental design

Chronic lower back pain is a major health problem and current treatments do not always lead to adequate pain control. Virtual reality (VR) is an upcoming technology that has shown to be effective in reducing acute pain. However, the value of VR in reducing chronic pain is still unknown. Therefore, the current study focuses on the effects of a recently developed VR application 'Reducept' using a multiple baseline single-case experimental design in 8 patients (N = 8). Reducept is a VR-training program aiming to improve pain management skills and providing pain education in patients with (chronic lower) back pain. Results based on visual and statistical analyses indicated that Reducept has the potential to reduce chronic lower back pain, although its clinical relevance was small. This study is one of the first that focuses on the possible effects of Reducept using sophisticated visual and statistical analyses. Our study shows a detailed overview of individual changes in pain intensity over time. Further research is necessary to investigate the working mechanism of Reducept and its impact on chronic pain conditions.

What are patients completing Cognitive Behavioral Insomnia Therapy telling us with their post-treatment Insomnia Severity Index scores?

STUDY OBJECTIVE

Cognitive Behavioral Insomnia Therapy (CBT-I) is the gold standard insomnia treatment and the Insomnia Severity Index (ISI) is a frequently used treatment outcome measure. The ISI has strong psychometric properties and is purported to measure perceived insomnia symptom severity. However, little is known about the factors that drive insomnia severity perception and the psychometric properties of the ISI post-CBT-I.

METHODS

Participants were treatment-seeking adults meeting DSM-5 Insomnia Disorder criteria (n = 203, ages 18-77, M = 45.95 years). Participants completed sleep and mood questionnaires, including the ISI, pre- and post-CBT-I. They completed daily Consensus Sleep Diaries each morning throughout two pre-treatment weeks, eight weeks of treatment and two weeks post-treatment. A hierarchical regression analysis examined what predicted post-CBT-I ISI scores and Cronbach's alpha was computed to examine post-treatment reliability of the ISI.

RESULTS

The regression analysis revealed that lower post-treatment ISI scores were associated with lower pre-treatment ISI, and greater decreases in fatigue and generalized anxiety symptoms. The model did not significantly improve when pre-treatment sleep effort or changes in sleep diary indices were added. The post-treatment ISI Cronbach's alpha was .88.

CONCLUSIONS

Although the ISI has been shown to have sound psychometric properties, clinicians should consider that post CBT-I ISI scores are not related to their sleep improvements. Instead, they seem to be related to whether patients perceive themselves as poor sleepers pre-treatment and whether they felt less tired and anxious after CBT-I. Researchers should consider the impact of factors other than sleep when using the ISI at post-treatment. Patients are telling us that CBT-I should focus on addressing symptoms of fatigue and general anxiety; perhaps CBT-I could be improved further to address these concerns more effectively.

SUMMARY

This investigation shows that when individuals are rating their symptom severity after CBT-I, they are also integrating how they felt before treatment and whether they experienced a change in their fatigue or anxiety.

The unified protocol as an internet-based intervention for emotional disorders: Randomized controlled trial

The Unified Protocol (UP) as a transdiagnostic intervention has primarily been applied in the treatment of anxiety disorders and in face-to-face-settings. The current study investigated the efficacy of a 10-week Internet-based adaptation of the UP for anxiety, depressive, and somatic symptom disorders. The trial was registered under DRKS00014820 at the German Clinical Trial Registry, DRKS. Participants (n = 129) were randomized to treatment or waitlist control. Significant treatment effects were found for symptom distress, satisfaction with life, positive/negative affect and markers of anxiety, depression, and somatic symptom burden (within-group Hedges’ g = 0.32–1.38 and between-group g = 0.20–1.11). Treatment gains were maintained at 1- and 6-month-follow-up. Subgroup analyses showed comparable effects in participants with anxiety and depressive disorders. 26.6% dropped out of treatment and 35.38% did not provide post-treatment assessments. The results strengthen the application of the UP as an Internet-based treatment for alleviating symptom distress across emotional disorders. More research on the applicability for single disorders is needed and avenues to improve adherence and attrition rates should be explored.

Differences in Treatment Effects of Cognitive-behavioral Therapy for Insomnia Based on Sleep Reactivity: A Preliminary Study

INTRODUCTION

Sleep reactivity is the trait-like degree to which stress disrupts sleep, resulting in difficulty falling and staying asleep. Although previous studies have suggested that individuals who have high sleep reactivity may be resistant to cognitive-behavioral therapy for insomnia (CBT-I) effects, there have been no studies that have investigated this empirically. This study explored differential treatment responses in CBT-I based on sleep reactivity levels.

MATERIAL AND METHOD

Participants for this study were nineteen insomnia patients who met DSM-5 criteria for insomnia disorder. All participants received four weekly sessions of structured cognitive-behavioral therapy for insomnia (CBT-I). Individuals completed the Insomnia Severity Index (ISI), Korean version of Center for Epidemiologic Studies Depression Scale-Revised (K-CESD-R), Ford Insomnia Response to Stress Test (FIRST), Dysfunctional Beliefs and Attitudes about Sleep Scale-16 (DBAS-16), the Daily Inventory of Stressful Events (DISE) and a sleep diary. Participants were classified into two groups based on sleep reactivity level (high and low sleep reactivity).

RESULT

Following treatment, significant changes were found for ISI, K-CESD-R, DBAS-16 and FIRST scores, sleep onset latency, wake after sleep onset, sleep efficiency, number of awakenings, sleep quality and feeling refreshed upon awakening in both groups. Improvements in sleep efficiency was lower in the high sleep reactivity group compared to the low sleep reactivity group. No differences in ISI, K-CESD-R, DBAS-16 scores, and stress event frequency during the treatment duration were found between groups.

CONCLUSION

These findings suggest that sleep reactivity level may be an important factor that affects treatment outcome of CBT-I. Furthermore, the results may suggest that individual response to stress events are more important than the stressor itself.

Evaluating an Aboriginal community controlled residential alcohol and other drug services: Use of benchmarking to examine within treatment changes in wellbeing

INTRODUCTION

Aboriginal Community Controlled Organisations (ACCO) have an important role in the Australian health-care sector. However, there has been a lack of research evaluating ACCOs in the treatment of alcohol and other drug (AOD) use. Using a benchmarking approach, the present study examined within treatment changes on measures of wellbeing for people attending a residential AOD ACCO.

METHODS

The study focused on The Glen, an AOD residential treatment service that is managed by the Ngaimpe Aboriginal Corporation (n = 775). The Glen is a male-only service and provides treatment to both Indigenous and non-Indigenous men. The evaluation focused on measures of wellbeing (i.e. symptom distress and quality of life) collected at intake, 30 and 60 days during the person's stay. Comparative benchmarking was conducted with a cohort of men who were attending non-ACCO residential AOD treatment services (n = 4457).

RESULTS

The Glen participants demonstrated statistically significant improvements on measures of wellbeing. The Glen participants were more likely to complete treatment than participants attending non-ACCO services. Likewise, Indigenous people attending The Glen were more likely to complete treatment (compared to Indigenous people attending non-ACCO services). Rates of reliable and clinically significant change suggested that changes in quality of life were largely equivalent between The Glen and non-ACCO services, while participants attending The Glen tended to demonstrate larger reductions in symptom distress compared to the non-ACCO services.

DISCUSSION AND CONCLUSION

The study provides further support for the important role that ACCOs play in supporting Indigenous people in their recovery.

Reliable change and the reliable change index: still useful after all these years?

In 1984 Jacobson and colleagues introduced the concept of reliable change, viz the amount of change on a measure that an individual needed to show to determine that it exceeded the extent of change likely due to measurement error alone. Establishing reliable change was a pre-requisite for determining clinical significance. This paper summarizes the rationale for determining reliable change as providing an individual-focused, idiographic alternative to the dominant nomothetic approach to clinical outcome research based on group mean data and statistical significance. The conventional computational steps for calculating an individual’s standardized difference (reliable change) score and the minimum raw change score on the measure (a reliable change index) required to classify individuals as reliably positively changed, indeterminate, or reliably deteriorated are described. Two methods for graphically representing reliable change are presented, and a range of possible uses in both research and practice settings are summarized. A number of issues and debates concerning the calculation of reliable change are reviewed. It is concluded that the concept of reliable change remains useful for both cognitive behavioural researchers and practitioners, but that there are options regarding methods of computation. In any use of reliable change, the rationale for selecting among method options and the exact computations used need clear and careful description so that we can continuously judge the utility and appropriateness of the use of reliable change and enhance its value to the field.

Key learning aims

(1)

Individually tailored Internet-delivered cognitive-behavioral therapy for survivors of intimate partner violence: A randomized controlled pilot trial

Intimate partner violence (IPV) is a serious public health concern worldwide and defined as behavior performed by spouses or other intimate partners that causes physical, sexual, or psychological harm. Internet-delivered cognitive-behavioral therapy (ICBT) may be particularly useful for survivors of IPV for several reasons, including barriers pertaining to limited community recourses and treatment availability, safety concerns, and issues of stigma, guilt and shame, which may prevent members of this population from seeking help via face-to-face interactions. However, Internet interventions are lacking. The primary aim of the present randomized controlled pilot trial was to explore the feasibility of ICBT as guided self-help individually tailored to the predominant symptomatology of PTSD or depression in survivors of IPV. A second aim was to conduct a preliminary evaluation exploring the short- and long-term effects of the treatment in comparison to a waitlist control condition. Results showed that the treatment was feasible. Attrition rate was low (9.4%), and participants were satisfied with treatment. However, treatment adherence was moderate in terms of completed modules (62.5%). Results of the preliminary evaluation of treatment effects showed large and statistically significant between-group effect sizes (Cohen's d = 0.86-1.08) on some measures of PTSD and depression at post assessment, favoring the treatment condition. However, there were no effects on other measures. At follow-up assessment, when the control condition had received delayed treatment, there were large and statistically significant within-group effect sizes (d = 0.96-1.48) on measures of PTSD, depression and anxiety, and small effects (d = 0.48) on a measure of quality of life. The results of the present pilot study are promising and warrant further research on ICBT for this population.

Initial outcomes of couple HOPES: A guided online couple intervention for PTSD and relationship enhancement

Couple HOPES (Helping Overcome PTSD and Enhance Satisfaction) is a guided, online couple intervention adapted from Cognitive-Behavioral Conjoint Therapy for posttraumatic stress disorder (PTSD). It was created to overcome a range of barriers to accessing evidence-based treatments for PTSD and the intimate relationship problems associated with it. This manuscript describes initial outcomes of the intervention in a series of 10 couples. Participants were military, veteran and first responders with probable PTSD and their intimate partners. Couples completed the program and measurements of PTSD, relationship satisfaction, and secondary outcomes at pre-, mid-, and post-intervention. Mean satisfaction for the program was high and it was completed by seven of ten couples. Participants with PTSD evidenced significant and large pre- to post-intervention effect size improvements in PTSD symptoms (g = 0.80) and perceived health (g = 1.13). They also exhibited non-significant but medium effect size pre- to post-intervention improvements in quality of life (g = 0.62), and depression (g = 0.53), and small effect size pre- to post-intervention improvements in argumentativeness (g = 0.43), anger (g = 0.31), and anxiety (g = 0.31). Partners reported significant and moderate pre- to post-intervention effect size improvements in relationship satisfaction (g = 0.68), and medium but not significant effect size improvements in accommodation of PTSD (g = 0.56). Results provide initial support for the feasibility, acceptability, and efficacy of Couple HOPES for improving PTSD and relationship satisfaction. However, more testing in larger samples, including with randomized controlled designs, is needed.

Outcomes of the grief and communication family support intervention on parent and child psychological health and communication

Parents and children risk developing psychological health problems following the death of a partner/parent and may need professional support. This study used the reliable change criterion and clinically significant change to examine the outcomes of the Grief and Communication Family Support Intervention, comprising three family meetings with a family therapist, among 10 parents and 14 children, using pre-post outcome scores. The results provided preliminary evidence that the Grief and Communication Family Support Intervention may improve self-esteem and reduce anxiety in some parents and may improve communication and reduce internalizing and externalizing problems in some children.

Routine outcome measurement in specialist non-government alcohol and other drug treatment services: Establishing effectiveness indicators for the NADAbase

INTRODUCTION AND AIMS

The collection of routine outcome measurement (ROM) data provides an opportunity for service providers to conduct benchmarking to inform quality assurance practices. To conduct comparative benchmarking, it is important that services have access to comparative data. This paper aims to establish effectiveness indicators for ROM data collected within the alcohol and other drug (AOD) sector.

DESIGN AND METHODS

ROM data were collected by specialist non-government AOD treatment services within the Network of Alcohol and other Drugs Agencies online database (i.e. NADAbase). All participants were attending treatment within New South Wales, Australia (N = 21 572). Effectiveness indicators were calculated by using effect sizes, standard error of measurement, and rates of reliable and clinically significant change. The study focused on quality of life (EUROHIS Quality of Life Scale), psychological distress (Kessler-10) and substance dependence (Substance Dependence Scale).

RESULTS

Since 2010, 21 572 unique people have completed at least one NADAbase Client Outcome Measure. Amphetamines (36%) and alcohol (32%) were the most commonly reported primary substances of concern. Effectiveness indicators were established for the total sample, as well as for people attending residential rehabilitation (n = 8161) and community-based (n = 10 306) treatment services. Standard error of measurement was the least stringent effectiveness indicator (i.e. a higher proportion of people demonstrated improvement), while the clinically significant change was the most stringent approach.

DISCUSSION AND CONCLUSIONS

The study demonstrated the utility of the NADAbase to establish effectiveness indicators for benchmarking purposes. Recommendations are provided for the use of benchmarking to inform quality assurance activities in the sector.

[Development and Evaluation of an Acceptance and Commitment Therapy (ACT)-Based Group Intervention for Cancer Patients with Psychological Burden]

OBJECTIVE

Cancer is associated with many psychosocial stressors that lead to high psychological distress in over 50% of patients. About one third of patients make use of psycho-oncological services. Acceptance and Commitment Therapy (ACT), a transdiagnostic approach of the third wave of cognitive behavioral therapy, represents a promising approach for psycho-oncological interventions. However, the German-speaking area lacks a manualized ACT-based group intervention that is tested for feasibility and effectiveness.

METHODS

To bridge this gap, we developed and preliminarily tested an ACT group intervention in this one-arm monocentric pilot study. It targets psychological burdened cancer patients in all disease stages. During 8 sessions, the ACT core processes mindfulness, values and acceptance are worked out by means of metaphors and experience-oriented exercises.

RESULTS

Pre-Post-Data of our pilot study with n=25 patients indicate reductions in fear of progression, depression and psychological inflexibility, as well as an increase in psychological health-related quality of life and well-being. Regression analyses showed that changes in psychological flexibility predicted changes in all psychological outcome measures. The very high adherence underlines the feasibility of the intervention for the integrated outpatient clinic setting.

DISCUSSION

Our pilot study provides first evidences for the good feasibility and effectiveness of an ACT-based group intervention in the German-speaking area. These results on effectivness should be interpreted with caution due to the lack of an experimental control group.

CONCLUSION

Future studies should test the effectiveness of the intervention in randomized controlled trials (RCTs) with longer follow-up periods.

Transdiagnostische Behandlung emotionaler Störungen bei Jugendlichen mit dem Unified Protocol: Eine Fallstudie

<b><i>Hintergrund:</i></b> Angst- und depressive Störungen sind die häufigsten psychischen Störungen im Jugendalter und weisen untereinander eine hohe Komorbiditätsrate auf. Während kognitive Verhaltenstherapie (KVT) in der Behandlung von Angststörungen bei Jugendlichen wirksam ist, weist die Behandlung von depressiven Störungen nur mo­derate Wirksamkeit auf. Es liegen jedoch kaum Studien vor, die die Wirksamkeit der KVT in der Behandlung komorbid vorliegender Angst- und depressiver Störungen bei Jugendlichen untersuchen. Der transdiagnostische Ansatz des Unified Protocols (UP) für emotionale Störungen konnte seine Wirksamkeit im englischsprachigen Raum bereits im Jugend- und Erwachsenenalter nachweisen. Eine deutsche Übersetzung und Überprüfung des UP für Jugendliche steht bislang noch aus. <b><i>Falldarstellung:</i></b> In dieser Kasuistik wird eine 17-jährige Patientin vorgestellt, die die Kriterien einer sozialen Angststörung nach ICD-10 und DSM-5 erfüllt und zudem eine depressive Symptomatik aufweist. Nach einer umfassenden Diagnostik mithilfe eines strukturierten klinischen Interviews und Selbstberichtverfahrens wurden 17 Sitzungen KVT basierend auf dem UP für Adoleszente (UP-A) durchgeführt. Ziel der vorliegenden Fallbeschreibung ist die Vorstellung des Therapierationals und der Umsetzung des Manuals. Nach der Therapie erfüllte die Jugendliche die Kriterien der sozialen Angststörung nicht mehr, es zeigte sich zudem eine Abnahme der depressiven Symptomatik. <b><i>Schlussfolgerungen:</i></b> Die Fallvorstellung verweist auf die Umsetzbarkeit der deutschen Übersetzung des UP-A für die transdiagnostische Behandlung von Angst- und depressiven Störungen bei Jugendlichen. Eine randomisiert kontrollierte Studie zur Wirksamkeitsprüfung steht noch aus.

Critical consideration of assessment methods for clinically significant changes of mental distress after psycho-oncological interventions

OBJECTIVES

Considering the heterogeneity of cancer entities and the associated disease progression, personalized care of patients is increasingly emphasized in psycho-oncology. This individualization makes the use of measurements of individual clinically significant change important when studying the efficacy and effectiveness of psycho-oncological care. Two conceptualizations for the measurement of clinical significance are critically contrasted in this study: the Reliable Change Index (RCI) and the Minimal Important Difference (MID) method.

METHODS

In total, 2,121 cancer patients participated in the study and a subsample of 708 patients was reassessed about 4 months later. Psychological distress was measured using the Hospital Anxiety and Depression Scale. We evaluated two measures of clinical significance (RCI, MID) by comparing the respective numbers of improved, unimproved, and deteriorated patients.

RESULTS

Individually significant changes were observed with both methods; however, determined rates of improvement differed substantially: MID (66.67%) and RCI (48.23%). Most importantly, according to MID, 17.93% of patients were identified as being improved, although their respective improvements were not statistically significant and thus unreliable.

CONCLUSIONS

The benefits of RCI outweigh MID, and therefore, the RCI is recommended as a measure to assess change.

'Effective' at What? On Effective Intervention in Serious Mental Illness

The term "effective," on its own, is honorific but vague. Interventions against serious mental illness may be "effective" at goals as diverse as reducing "apparent sadness" or providing housing. Underexamined use of "effective" and other success terms often obfuscates differences and incompatibilities in interventions, degrees of effectiveness, key omissions in effectiveness standards, and values involved in determining what counts as "effective." Yet vague use of such success terms is common in the research, clinical, and policy realms, with consequences that negatively affect the care offered to individuals experiencing serious mental illness. A pragmatist-oriented solution to these problems suggests that when people use success terms, they need to explain and defend the goals and supporting values embedded in the terms, asking and answering the questions, "Effective at what? For whom? How effective? And why that goal?" Practical and epistemic standards for effectiveness will likely remain plural for good reasons, but each standard should be well explained and well justified.

Approaches to denote treatment outcome: Clinical significance and clinical global impression compared

OBJECTIVES

The authors of a previous study proposed a statistically based approach to denote treatment outcome, translating pretest and posttest scores into clinically relevant categories, such as recovery and reliable improvement. We assessed the convergent validity of the Jacobson-Truax (JT) approach, using T-score based cutoff values, with ratings by an independent evaluator.

METHODS

Pretest and retest scores on the Brief Symptom Inventory (BSI) and clinical global impression improvement (CGI-I) ratings were collected repeatedly through routine outcome monitoring from 5,900 outpatients with common mental disorders. Data were collected in everyday practice in a large mental health care provider.

RESULTS

Continuous pretest-to-retest BSI change scores had a stronger association with CGI-I than the categorical variable based on JT. However, JT categorization and improvement according to CGI converged substantially with association indices (Somers' D) ranging from D = .50 to .56. Discordance was predominantly due to a more positive outcome according to JT than on CGI-I ratings.

CONCLUSION

Converting continuous outcome variables into clinically meaningful categories comes at the price of somewhat diminished concurrent validity with CGI-I. Nevertheless, support was found for the proposed threshold values for reliable change and recovery, and the outcome denoted in these terms corresponded with CGI improvement for most patients.

Effects, Adherence, and Therapists' Perceptions of Web- and Mobile-Supported Group Therapy for Depression: Mixed-Methods Study

Background

Blended group therapy (bGT) has been investigated a several times for anxiety and depression, but information on patients’ adherence to and therapists’ perception of the novel format is nonexistent. Furthermore, many studies investigated mainly female and highly educated populations, limiting the validity of previous findings.

Objective

This study aimed to reduce the gaps and limitations of the previous findings by evaluating an integrated internet- and mobile-supported bGT format.

Methods

A total of 27 patients diagnosed with major depression (14/27, 52% female and 7/27, 25.9% compulsory education) participated in a 7-week treatment at a university outpatient clinic. Furthermore, 8 novice therapists participated in semistructured interviews and a subsequent cross-validation survey.

Results

Primary symptom reduction was high (d=1.31 to 1.51) and remained stable for the follow-up period. Therapists identified advantages (eg, patient engagement, treatment intensification, and improved therapeutic relation) and disadvantages (eg, increased workload, data issues, and undesired effects) of bGT. The required online guidance time was 10.3 min per patient and week, including guidance on exercises (67% or 6.9 min) and intimate communication (33% or 3.4 min). Concerning patients’ adherence to bGT, tracked completion of all Web-based and mobile tasks was high and comparable with group attendance.

Conclusions

Results suggest high feasibility of bGT in a gender-balanced, moderately educated sample. bGT provides group therapists with tools for individual care, resulting in an optimization of the therapy process, and high completion rates of the implemented bGT elements. The limited work experience of the involved therapists restricts the study findings, and potential drawbacks need to be regarded in the development of future bGT interventions.

No Change? A Grounded Theory Analysis of Depressed Patients' Perspectives on Non-improvement in Psychotherapy

Aim: Understanding the effects of psychotherapy is a crucial concern for both research and clinical practice, especially when outcome tends to be negative. Yet, while outcome is predominantly evaluated by means of quantitative pre-post outcome questionnaires, it remains unclear what this actually means for patients in their daily lives. To explore this meaning, it is imperative to combine treatment evaluation with quantitative and qualitative outcome measures. This study investigates the phenomenon of non-improvement in psychotherapy, by complementing quantitative pre-post outcome scores that indicate no reliable change in depression symptoms with a qualitative inquiry of patients' perspectives. Methods: The study took place in the context of a Randomised Controlled Trial evaluating time-limited psychodynamic and cognitive behavioral therapy for major depression. A mixed methods study was conducted including patients' pre-post outcome scores on the BDI-II-NL and post treatment Client Change Interviews. Nineteen patients whose data showed no reliable change in depression symptoms were selected. A grounded theory analysis was conducted on the transcripts of patients' interviews. Findings: From the patients' perspective, non-improvement can be understood as being stuck between knowing versus doing, resulting in a stalemate. Positive changes (mental stability, personal strength, and insight) were stimulated by therapy offering moments of self-reflection and guidance, the benevolent therapist approach and the context as important motivations. Remaining issues (ambition to change but inability to do so) were attributed to the therapy hitting its limits, patients' resistance and impossibility and the context as a source of distress. "No change" in outcome scores therefore seems to involve a "partial change" when considering the patients' perspectives. Conclusion: The study shows the value of integrating qualitative first-person analyses into standard quantitative outcome evaluation and particularly for understanding the phenomenon of non-improvement. It argues for more multi-method and multi-perspective research to gain a better understanding of (negative) outcome and treatment effects. Implications for both research and practice are discussed.

Validity Beyond Measurement: Why Psychometric Validity Is Insufficient for Valid Psychotherapy Research

In psychotherapy research, "validity" is canonically understood as the capacity of a test to measure what is purported to measure. However, we argue that this psychometric understanding of validity prohibits working researchers from considering the validity of their research. Psychotherapy researchers often use measures with a different epistemic goal than test developers intended, for example when a depression symptom measure is used to indicate "treatment success" (cf. outcome measurement for evidence-based treatment). However, the validity of a measure does not cover the validity of its use as operationalization of another target concept within a research procedure, nor the validity of its function toward an epistemic goal. In this paper, we discuss the importance of considering validity of the epistemic process beyond the validity of measures per se, based on an empirical case example from our psychotherapy study ("SCS", Cornelis et al., 2017). We discuss why the psychometric understanding of validity is insufficient in covering epistemic validity, and we evaluate to what extent the available terminology regarding validity of research is sufficient for working researchers to accurately consider the validity of their overall epistemic process. As psychotherapy research is meant to offer a sound evidence-base for clinical practice, we argue that it is vital that psychotherapy researchers are able to discuss the validity of the epistemic choices made to serve the clinical goal.

Integrative systematic review of psychodrama psychotherapy research: Trends and methodological implications

BACKGROUND

Psychodrama is an experiential psychotherapy in which guided role-play is used to gain insights and work on personal and interpersonal problems and possible solutions. Despite the wealth of literature describing clinical work, psychodrama intervention research is relatively scarce compared to other psychotherapies and psychological interventions.

OBJECTIVE

For this reason we implemented the integrative approach to systematic review that authorizes the combination of publications with diverse methodologies and all types of participants, interventions, comparisons, and outcomes. Our aim was to produce a comprehensive summary of psychodrama intervention research in the last decade that critically evaluates methodological issues to inform future studies.

METHODS

We searched four major electronic databases (PsycINFO, PubMEd, Scopus by Elsevier, and Web of Science) for peer-reviewed articles on psychodrama interventions published in English between 1 January 2007 and 31 December 2017. The quality of qualitative and mixed methods studies was assessed on the basis of pre-established guidelines, and the risk of bias was assessed for all quantitative randomized control studies, consistent with the PRISMA protocol.

FINDINGS

The database search and a hand search resulted in 31 psychodrama intervention publications. Overall, these studies examined the effects of psychodrama on more than 20 different outcomes and most studies had adult clients. The next largest group was adolescents, whereas only two studies involved children. Thus psychodrama intervention research in the last decade suggests there are promising results in all methodologies, and highlights the need to enhance methodological as well as reporting quality and to theorize and examine modality-specific mechanisms that lead to therapeutic change. Recommendations to improve methodology, transparency, and specificity in reporting future psychodrama and other psychotherapy research are discussed.

Statistics for Evaluating Pre-post Change: Relation Between Change in the Distribution Center and Change in the Individual Scores

In a number of scientific fields, researchers need to assess whether a variable has changed between two time points. Average-based change statistics (ABC) such as Cohen's d or Hays' ω² evaluate the change in the distributions' center, whereas Individual-based change statistics (IBC) such as the Standardized Individual Difference or the Reliable Change Index evaluate whether each case in the sample experienced a reliable change. Through an extensive simulation study we show that, contrary to what previous studies have speculated, ABC and IBC statistics are closely related. The relation can be assumed to be linear, and was found regardless of sample size, pre-post correlation, and shape of the scores' distribution, both in single group designs and in experimental designs with a control group. We encourage other researchers to use IBC statistics to evaluate their effect sizes because: (a) they allow the identification of cases that changed reliably; (b) they facilitate the interpretation and communication of results; and (c) they provide a straightforward evaluation of the magnitude of empirical effects while avoiding the problems of arbitrary general cutoffs.

Experiences of Change in Emotion Regulation Group Therapy. A Mixed-Methods Study of Six Patients

Emotion Regulation Group Therapy (ERGT) is a treatment for non-suicidal self-injury (NSSI) that has recently been implemented in Sweden and evaluated in an open trial with 95 patients in 14 adult outpatient psychiatric clinics (Sahlin et al., 2017). The purpose of the present study was to explore in more detail six of these patients’ experiences of change in ERGT by means of a person-oriented mixed-methods design. Reliable change and clinical significance were calculated for each individual on measures of self-harm, depression, anxiety, stress, and emotion regulation. Semi-structured interviews were carried out, transcribed and analysed using thematic analysis. Both the quantitative and qualitative data on change suggested that the most consistent changes occurred on emotion regulation. The treatment sessions that were most appreciated were those that focused on emotion awareness and emotion regulation. The participants also expressed appreciation of what ERGT afforded in terms of belonging and sharing with others, and the sense of equality in the relationship to the therapist. Critical comments were expressed concerning some parts of the treatment, as for example not having access to an individual therapist. Among the limitations of the study are the small convenience sample, which does not allow for any generalizing conclusions, and that the interviews took place a considerable time (2-3 years) after the participation in ERGT.

Longitudinal analysis of statistical and clinically significant psychosocial change following mental health rehabilitation

Purpose: With appropriate mental health rehabilitation, schizophrenia is increasingly associated with reports of recovery and stability. However, there is little empirical evidence evaluating the efficacy of services delivering this care. This study evaluated the effectiveness of rehabilitation for improving psychosocial function in consumers with schizophrenia.Methods: An electronic database of standardized assessment instruments mandated and maintained by the health service was retrospectively reviewed to extract ratings of psychosocial function, daily living skills, and mood state from consecutive admissions to an inpatient rehabilitation service. Outcomes were compared at admission, discharge, and one-year follow-up to identify statistically significant change. Individual reliable and clinically significant change was also assessed by comparison with a normative group of clients functioning independently in the community.Results: From admission to discharge the rehabilitation group made statistically significant gains in psychosocial function and daily living skills. Improvements were reliable and clinically significant in one-quarter to one-third of individual consumers. Approximately half sustained their improvements at follow-up, although this represented only a small fraction of the overall cohort. Consumers not demonstrating gains exhibited psychometric floor effects at admission.Conclusions: Rehabilitation can produce statistically and clinically significant immediate improvement in psychosocial function for a sub-set of consumers with elevated scores at admission. The durability of any gains is less clear, and strategies promoting longer-term maintenance are encouraged. Furthermore, currently mandated outcome measures are confounded by issues of sensitivity and reporting compliance, and exploration of alternative instruments for assessing recovery is recommended.Implications for RehabilitationRoutinely collected standardized outcome measures can be used to investigate the effectiveness of mental health rehabilitationIn addition to statistical significance, the clinical significance of outcomes should be evaluated to identify change that is individually meaningfulCurrently mandated outcomes instruments do not adequately evaluate many individuals' recovery journeyMental health service evaluation and quality improvement processes would likely benefit from adoption of recovery-oriented measures.

Brief Therapist-Guided Exposure Treatment of Panic Attacks: A Pilot Study

A three-session therapist-guided exposure treatment was tested in a consecutive series of eight primary health care patients suffering from panic attacks who specifically used distraction techniques as their primary safety behavior. The Panic Disorder Severity Scale Self-Report (PDSS-SR) was administered at baseline (1-3 weeks before the first session), and 1, 2, and 3 weeks after treatment. Weekly ratings on the Body Sensations Questionnaire (BSQ) and the Agoraphobic Cognitions Questionnaire (ACQ) during treatment were undertaken to explore when reliable change took place on these measures. The results showed a large within-group effect size on PDSS-SR ( d = 1.63); six of the eight patients were classified as responders, and four of them showed remission. Large effect sizes ( ds between 1.17 and 3.00) were seen also on BSQ and ACQ, as well as on agoraphobic avoidance, general level of anxiety, and depression. The results on BSQ and ACQ suggest that the fear of body sensations in most cases was reduced before a change occurred in agoraphobic cognitions. These results indicate that a brief three-session exposure-based treatment may be sufficient for this subgroup of panic patients. The findings need to be replicated under controlled conditions with larger samples and different therapists before more firm conclusions can be drawn. Future research should also focus on the relevance of dividing patients into subgroups based on type of safety behavior.

Feasibility of a blended group treatment (bGT) for major depression: uncontrolled interventional study in a university setting

OBJECTIVE

This study investigated the feasibility of a novel blended (face-to-face and computer-based) group intervention for the reduction of depressive symptoms in major depression.

DESIGN

Patient-centred uncontrolled interventional study.

SETTING

University setting in a general community sample. A multimodal recruitment strategy (public health centres and public areas) was applied.

PARTICIPANTS

Based on independent interviews, 26 participants, diagnosed with major depressive disorder (81% female; 23% comorbidity >1 and 23% comorbidity >2), entered treatment.

INTERVENTION

Acceptance and mindfulness based, as well as self-management and resource-oriented psychotherapy principles served as the theoretical basis for the low-threshold intervention. The blended format included face-to-face sessions, complemented with multimedia presentations and a platform featuring videos, online work sheets, an unguided group chat and remote therapist-patient communication.

MAIN OUTCOME MEASURES

The Center for Epidemiological Studies-Depression scale and the 12-item General Health Questionnaire.

RESULTS

Large to very large within group effect sizes were found on self-reported depression (F_{(2, 46.37)}=25.69, p<0.001; d=1.80), general health (F_(2,46.73)=11.47, p<0.001; d=1.32), personal resources (F_(2,43.36)=21.17, p<0.001; d=0.90) and mindfulness (F_(2,46.22)=9.40, p<0.001; d=1.12) after a follow-up period of 3 months. Treatment satisfaction was high, and 69% ranked computer and multimedia use as a therapeutic factor. Furthermore, participants described treatment intensification as important advantage of the blended format. Half of the patients (48%) would have preferred more time for personal exchange.

CONCLUSION

The investigated blended group format seems feasible for the reduction of depressive symptoms in major depression. The development of blended interventions can benefit from assuring that highly structured treatments actually meet patients' needs. As a next step, the intervention should be tested in comparative trials in routine care.

TRIAL REGISTRATION NUMBER

DRKS00010894; Pre-results.

Patient baseline interpersonal problems as moderators of outcome in two psychotherapies for bulimia nervosa

Objective: We tested an aptitude by treatment interaction; namely, whether patients' baseline interpersonal problems moderated the comparative efficacy of cognitive-behavioral therapy (CBT) vs. interpersonal psychotherapy (IPT) for bulimia nervosa (BN). Method: Data derived from a randomized-controlled trial. Patients reported on their interpersonal problems at baseline; purge frequency at baseline, midtreatment, and posttreatment; and global eating disorder severity at baseline and posttreatment. We estimated the rate of change in purge frequency across therapy, and the likelihood of attaining clinically meaningful improvement (recovery) in global eating disorder severity by posttreatment. We then tested the interpersonal problem by treatment interactions as predictors of both outcomes. Results: Patients with more baseline overly communal/friendly problems showed steeper reduction in likelihood of purging when treated with CBT vs. IPT. Patients with more problems of being under communal/cold had similar reductions in likelihood of purging across both treatments. Patients with more baseline problems of being overly agentic were more likely to recover when treated with IPT vs. CBT, whereas patients with more problems of being under agentic were more likely to recover when treated with CBT vs. IPT. Conclusions: Interpersonal problems related to communion and agency may inform treatment fit among two empirically supported therapies for BN.

A case study of lessons learnt from implementing a routine outcome monitoring system for psychotherapy in a South African community clinic

Monitoring outcomes in psychotherapy is important for advancing research into empirically supported treatments, collecting data to inform practice-based evidence and for monitoring patient treatment responses during routine care. We implemented two routine outcome monitoring measures in a community psychology clinic in South Africa, namely, the Outcome Rating Scale and Kessler Psychological Distress Scale. In this article, we adopt a case study methodology to describe and critically reflect on the implementation process and evaluate the utility of the outcome data collected during the first year of implementation. Our findings suggest that the measures have some utility but that there were significant problems with implementation. We found that Outcome Rating Scale and Kessler Psychological Distress Scale scores correlated in the expected direction; however, Outcome Rating Scale intake scores for the present population were at odds with the measure’s norms. The findings highlight the care that needs to be taken to change routine clinical practice when outcome measures are implemented in local settings and the need for research to validate the use of these ROM measures in South African clinical settings.

Guided and unguided Acceptance and Commitment Therapy for social anxiety disorder and/or panic disorder provided via the Internet and a smartphone application: A randomized controlled trial

Acceptance and Commitment Therapy (ACT) can be effective in treating anxiety disorders, yet there has been no study on Internet-delivered ACT for social anxiety disorder (SAD) and panic disorder (PD), nor any study investigating whether therapist guidance is superior to unguided self-help when supplemented with a smartphone application. In the current trial, n=152 participants diagnosed with SAD and/or PD were randomized to therapist-guided or unguided treatment, or a waiting-list control group. Both treatment groups used an Internet-delivered ACT-based treatment program and a smartphone application. Outcome measures were self-rated general and social anxiety and panic symptoms. Treatment groups saw reduced general (d=0.39) and social anxiety (d=0.70), but not panic symptoms (d=0.05) compared to the waiting-list group, yet no differences in outcomes were observed between guided and unguided interventions. We conclude that Internet-delivered ACT is appropriate for treating SAD and potentially PD. Smartphone applications may partially compensate for lack of therapist support.

Implementation and Evaluation of Evidence-Based Psychotherapeutic Practices for Youth in a Mental Health Organization

The current initiative and program evaluation study is a demonstration of the research to practice process in youth-focused psychotherapy. We collaborated within a community-university partnership to create practice and research infrastructure in order to develop, implement, and evaluate two new models of service founded on evidence-based psychotherapeutic practice parameters. The two new service models incorporated validated interventions to address behavior problems in elementary age children, and depression in adolescents, which were delivered in separate but similarly run intensive outpatient programs within a mental health setting. We utilized a rigorous training, technical assistance, fidelity monitoring, and outcome measurement strategy to promote the integrity and quality of services provided. The resultant programs were delivered with acceptable to high fidelity and effects on youth and parenting measures collected during program and from pre to post showed a decrease in targeted problems in youth and positive benefits for families. This initiative and program evaluation adds to the accumulating research-to-practice literature in children's mental health.

Initial data on a 5-item measure of OCD symptom severity and change: The obsessive compulsive session change index (OCSCI)

Repeated measurement of symptoms throughout treatment allows providers to assess individual patterns of symptom change. However, outside of a clinical trial, administration of lengthy measures at each session is unlikely. Therefore, in order to bridge the gap between clinical research and actual clinical practice we developed a brief self-report measure of obsessive compulsive symptoms that can be used at each session, the Obsessive Compulsive Session Change Index (OCSCI). The OCSCI assesses time spent on obsessions, time spent on compulsions, interference and distress related to obsessions and compulsions, and relative change in symptoms from the beginning of treatment. This paper reports initial psychometric properties of the OCSCI. Twenty-seven adults receiving exposure and ritual prevention for obsessive compulsive disorder (OCD) completed the OCSCI at each session. Results showed that the OCSCI had good internal consistency, and convergent and divergent validity. Specifically, the OCSCI had moderate to strong correlations with both self-reported and clinician rated OCD. Finally, the OCSCI was not as strongly correlated with depressive symptoms as it was with OCD symptoms. The findings presented herein suggest that the OCSCI can be useful as a measure of client progress throughout treatment.

Validity of clinically significant change classifications yielded by Jacobson-Truax and Hageman-Arrindell methods

BACKGROUND

Reporting of the clinical significance of observed changes is recommended when publishing mental health treatment outcome studies and is increasingly used in routine outcomes monitoring systems. Since recovery rates vary with the method chosen, we investigated the validity of classifications of clinically significant change when the Jacobson-Truax method and the Hageman-Arrindell method were used.

METHODS

Of 718 inpatients who completed the Depression Anxiety Stress Scales (DASS-21) and Quality of Life Enjoyment and Satisfaction Questionnaire at admission and discharge to a psychiatric clinic, 355 were invited (and 119 agreed) to complete the questionnaires and the Recovery Assessment Scale six weeks post discharge.

RESULTS

Both the JT and HA methods showed comparably good validity when referenced against the other indices. Clinically significant change on the DASS-21 was related to a greater consumer-based sense of recovery, greater perceived quality of life, and fewer readmissions to hospital within 28 days of discharge.

CONCLUSIONS

Since there was found to be no advantage to using one method over another when recovery is of interest, the simpler JT method is recommended for routine usage.

Both sides of the story: Exploring how improved and less-improved participants experience mindfulness-based stress reduction for social anxiety disorder

OBJECTIVE

What works for whom in mindfulness-based group interventions for social anxiety disorder (SAD)? The present article compared the experiences of 14 participants in a clinical study of mindfulness-based stress reduction (MBSR) for young adults with SAD.

METHODS

A two-staged mixed methods design was used to identify the participants who reported the highest (n = 7) and lowest (n = 7) levels of symptomatic change on outcome measures after treatment, and analyze qualitative in-depth interviews to explore what they experienced as helpful and unhelpful during the MBSR program. The qualitative interviews were analyzed using a thematic analysis methodology.

RESULTS

We identified the global theme of (i) Discovering agency to change or not feeling empowered through the MBSR program, and four sub-themes: (ii) Forming an active commitment or feeling ambivalence toward learning mindfulness, (iii) Engaging with others or avoiding contact with the group, (iv) Using the mindfulness exercises to approach or resigning when facing unpleasant experiences, and (v) Using the course to break interpersonal patterns or remaining stuck in everyday life.

CONCLUSIONS

MBSR may be helpful for young adults with SAD, although it may be important to match clients to their preferred form of treatment.

Methods for Quantifying the Clinical Significance of Change During Intervention Program Participation

Assessing the practical or clinical significance (CS) of an intervention program's outcomes is useful in determining its effectiveness. The CS approach gives information beyond traditional analyses by quantifying the proportions of people who meaningfully improve and deteriorate. We link latent transition analyses (LTA) to the CS literature and use a case study to contrast it with the long-standing Jacobson and Truax (JT) approach. Data came from 2,717 individuals convicted of a substance-related offense who participated in an indicated prevention program Prime For Life^® (PFL). We selected outcomes describing drinking beliefs and behavior. Both CS approaches categorized a majority of participants as improved (i.e., transitioning from baseline subgroups with risky behaviors and cognitions into posttest subgroups showing lower risk). Results demonstrate how the JT approach allows the assessment of improvements on individual outcomes, while the LTA provides more nuanced information about risk groupings. Selecting a CS approach depends on research goals, availability of normative data, and data considerations. JT is an appropriate method when evaluating single outcomes. In contrast, LTA is better when a multivariate description is desired, advanced missing data handling methods are needed, or outcomes are not normally distributed. Although infrequently done, evaluating CS provides useful information about program effectiveness.

A train of thought: 25 years of Psychotherapy research

This paper serves as an introduction to the 25th anniversary issue of Psychotherapy research. It includes a consideration of the original aims of the journal in light of the most cited articles, various developments in research orientation and methodology, and most recent publications. It demonstrates both diversity and consistency in content over time, as well as the international reach of the journal.

Is interpersonal counselling (IPC) sufficient treatment for depression in primary care patients? A pilot study comparing IPC and interpersonal psychotherapy (IPT)

BACKGROUND

Psychotherapeutic treatment is underused in primary care, where even short-term psychotherapy can be perceived as too lengthy and labour-intensive. We tested here for the first time the preliminary efficacy of seven sessions of interpersonal counselling (IPC) by comparison with sixteen sessions of interpersonal psychotherapy (IPT) in regular clinical settings.

METHODS

Patients seeking treatment for the first time who met the DSM-IV criteria for major depressive disorder (MDD, mild/moderate) were randomized to either IPC (n=20) or IPT (n=20). The efficacy of the treatments was assessed using the 34-item Clinical Outcomes in Routine Evaluation (CORE-OM) scale and the Beck Depression Inventory (BDI) scale.

RESULTS

90% of the patients completed all the treatment sessions. IPC delivered by psychiatric nurses in primary care proved equally as effective as IPT delivered by psychotherapists/psychologists in secondary care. The pre-treatment to 12-month follow-up within-group effect sizes were large: 1.52 (CORE-OM) and 1.41 (BDI) in the IPC group and 1.58 (CORE-OM) and 1.40 (BDI) in the IPT group. At the 12-month follow-up 59% of the patients in the IPC group and 63% in the IPT group were classified as recovered on the CORE-OM scale, with corresponding remission rates of 61% for both groups on the BDI scale.

LIMITATIONS

The small sample size limited the power to detect differences between the groups and the naturalistic settings may have confounded the results.

CONCLUSIONS

This clinical trial suggests that IPC is an appropriate and even sufficient first-phase intervention for handling previously untreated mild to moderate depression in primary health care.

Response and Remission After Cognitive and Exposure Therapy for Hypochondriasis

Cognitive behavioral therapy has demonstrated large effect sizes for the treatment of hypochondriasis. However, response and remission rates, which provide important additional information about clinically significant improvements, have seldom been reported. In the current study, rates of response and remission after cognitive therapy and exposure therapy were evaluated. The study was based on a randomized controlled trial that treated patients with hypochondriasis (N = 75). The primary outcome measure was a clinician-administered structured interview for hypochondriasis. At posttreatment, response was found for 72.0% and remission for 45.3% of the patients. At 12-month follow-up, the response rate was 68.0%, and the remission rate was 54.7%. No significant differences in the frequency of response/remission were found between the cognitive therapy and the exposure therapy groups. Patients' and therapists' perspectives of clinical improvement corresponded with remission rather than response rates. Improvement rates were compared with those of other mental disorders, and implications for the treatment of hypochondriasis were discussed.