A conceptual and empirical review of the meaning, measurement, development, and teaching of intervention competence in clinical psychology

Development and initial evaluation of a treatment integrity measure for low-intensity group psychoeducational interventions

BACKGROUND

Despite the importance of assessing the quality with which low-intensity (LI) group psychoeducational interventions are delivered, no measure of treatment integrity (TI) has been developed.

AIMS

To develop a psychometrically robust TI measure for LI psychoeducational group interventions.

METHOD

This study had two phases. Firstly, the group psychoeducation treatment integrity measure-expert rater (GPTIM-ER) and a detailed scoring manual were developed. This was piloted by n=5 expert raters rating the same LI group session; n=6 expert raters then assessed content validity. Secondly, 10 group psychoeducational sessions drawn from routine practice were then rated by n=8 expert raters using the GPTIM-ER; n=9 patients also rated the quality of the group sessions using a sister version (i.e. GPTIM-P) and clinical and service outcome data were drawn from the LI groups assessed.

RESULTS

The GPTIM-ER had excellent internal reliability, good test-retest reliability, but poor inter-rater reliability. The GPTIM-ER had excellent content validity, construct validity, formed a single factor scale and had reasonable predictive validity.

CONCLUSIONS

The GPTIM-ER has promising, but not complete, psychometric properties. The low inter-rater reliability scores between expert raters are the main ongoing concern and so further development and testing is required in future well-constructed studies.

Understanding the Role of Therapist Quality in Accounting for Heterogeneity of Patient Outcomes in Psychosocial Chronic Pain Treatments

A variety of evidence-based psychosocial treatments now exist for chronic pain. However, on average, effect sizes have tended to be modest and there is a high degree of heterogeneity in treatment response. In this focus article, we explore the potential role that therapist quality in delivering treatment may have in accounting for a degree of this variability in outcome. Therapist quality refers to the skillful delivery of treatment, harnessing both specific and common therapeutic factors during sessions. While recognized as important to assess and report in clinical trials by some reporting guidelines, few randomized controlled trials evaluating psychosocial treatments for chronic pain have reported on therapist factors. We reviewed the clinical trials included in three systematic reviews and meta-analyses of trials of cognitive-behavioral therapy (mainly), mindfulness-based interventions, and acceptance and commitment therapy. We found that of the 134 trials included, only nine assessed and reported therapist quality indicators, with a variety of procedures used. This is concerning as without knowledge of the quality in which treatments are delivered, the estimates of effect sizes reported may be misrepresented. We contextualize this finding by drawing on the broader psychotherapy literature which has shown that more skillful, effective therapists demonstrated ten times better patient response rates. Examination of the characteristics associated with these more effective therapists tends to indicate that skillful engagement of common factors in therapy sessions represents a distinguishing feature. We conclude by providing recommendations for assessing and reporting on therapist quality within clinical trials evaluating psychosocial treatments for chronic pain. PERSPECTIVE: Therapist quality in the delivery of psychosocial treatments for chronic pain has rarely been assessed in clinical trials. We propose that therapist quality indicators are an under-studied mechanism that potentially contributes to the heterogeneity of treatment outcomes. We provide recommendations for assessing and reporting on therapist quality in future trials.

Reliability and validity of the Self-Reflection and Insight Scale for psychologists and the development and validation of the revised short version

Self-reflection is broadly considered a core competency for psychologists; however, there is an absence of measures of self-reflection, limiting the extent to which self-reflection can be assessed in both research and practice contexts. Whilst the Self-Reflection and Insight Scale (Grant et al., 2002) has been validated in a range of formats with different populations, it has not yet been validated with psychologists. Further, the psychometric properties of a short version of the scale (Silvia, 2021) have not been examined for use with psychologists. This study tested the factor structure, internal consistency and convergent and divergent validity of the Self-Reflection and Insight Scale with registered psychologists (N = 123), finding both the full scale and short version to have sound psychometrics. However, as there were low loading items across both versions of the measure, and the short version also excluded high-loading items, the SRIS-Revised (SRIS-R) was formed through model improvement, retaining a total of 14 items. This revised version of the scale captures high loading items without redundancy of low-loading items, resulting in a measure that parsimoniously captures the construct of self-reflection as relevant to psychologists. The SRIS-R demonstrated good internal consistency (α = .882), convergent, divergent and construct validity. Scores on the SRIS-R were used to test whether there was a correlation between self-reflection and years of professional registration, with this not being significant.

Evaluating training needs in clinical psychology doctoral programs

OBJECTIVE

Advances in clinical psychology must be accompanied by advances in training. This study assessed training content, quality, and needs during clinical psychology doctoral programs among current or past doctoral students.

METHODS

Current or past clinical psychology doctoral students (N = 343) completed an anonymous survey assessing training experiences and needs. A descriptive-focused exploratory factor analysis (EFA) also examined whether common subgroups of academic interests emerged.

RESULTS

Most participants reported that they sought training beyond required coursework, primarily in clinical training, cultural competency, and professional development, and reported having taken one or more unhelpful course, including discipline-specific knowledge requirements. Descriptive results from the EFA demonstrated common training areas of interest: diversity topics, biological sciences, clinical practice, and research methods.

DISCUSSION

This study demonstrates that trainees and early career psychologists are aware of their nuanced and in some cases, unmet training needs.

CONCLUSION

This work foregrounds the need to adapt extant training opportunities to support the next generation of clinical psychologists.

The concise measurement of clinical communication skills: Validation of a short scale

OBJECTIVE

There is a lack of brief rating scales for the reliable assessment of psychotherapeutic skills, which do not require intensive rater training and/or a high level of expertise. Thus, the objective is to validate a 14-item version of the Clinical Communication Skills Scale (CCSS-S).

METHODS

Using a sample of N = 690 video-based ratings of role-plays with simulated patients, we calculated a confirmatory factor analysis and an exploratory structural equation modeling (ESEM), assessed convergent validities, determined inter-rater reliabilities and compared these with those who were either psychology students, advanced psychotherapy trainees, or experts.

RESULTS

Correlations with other competence rating scales were high (rs > 0.86-0.89). The intraclass correlations ranged between moderate and good [ICC_(2,2) = 0.65-0.80], with student raters yielding the lowest scores. The one-factor model only marginally replicated the data, but the internal consistencies were excellent (α = 0.91-95). The ESEM yielded a two-factor solution (Collaboration and Structuring and Exploration Skills).

CONCLUSION

The CCSS-S is a brief and valid rating scale that reliably assesses basic communication skills, which is particularly useful for psychotherapy training using standardized role-plays. To ensure good inter-rater reliabilities, it is still advisable to employ raters with at least some clinical experience. Future studies should further investigate the one- or two-factor structure of the instrument.

Learning Processes and Acquisition of Knowledge and Skills in Training and Supervision of Psychotherapy and Counselling: A Study Protocol for a Scoping Review

Background: Increased awareness of the individual therapist’s vital contribution to treatment processes and outcome, and the potential role of training and supervision in this respect, warrants a close look at the empirical and theoretical literature on teaching and learning of therapists and counselors.

Methods: A scoping review of the literature will be conducted based on an overarching research question: when authors have reported on learning processes and acquisition of knowledge and skills in psychotherapy/counseling and supervision/training literature over the past 30 years (since 1990), what evidence, concepts, theories, and models have they reported? A comprehensive search strategy is carried out to identify publications indexed in Scopus, PsycINFO, and Cochrane Central Register of Controlled Trials. Publications will be sorted according to four categories: (1) conceptual/theoretical; (2) empirical (quantitative, qualitative, mixed methods); (3) review, meta-synthesis or -analysis; (4) training program/model description. Procedures for the upcoming scoping review of conceptual/theoretical, empirical, and training program/model description publications will be outlined.

Conclusion: Besides clarifying existing perspectives, practices, and evidence, and documenting the shifting trends of the field during the past three decades, this scoping review identifies knowledge gaps that point to vital future directions for research and theory development. Moreover, the comprehensive scoping lays the foundation for subsequent, more focused systematic reviews that address identified key research topics more specifically.

Examining the Relation Between Technical and Global Competence in Two Treatments for Youth Anxiety

Although technical (quality of delivering techniques from a specific treatment) and global (general clinical expertise) competence are believed to be important ingredients of successful psychosocial treatment with youth, there have been few empirical efforts to measure both dimensions. Efforts to understand the role that each competence dimension plays in the process and outcome of youth treatment starts with determining whether the dimensions can be measured separately. This study examined whether scores from measures designed to assess technical and global competence were distinct. Treatment sessions (N = 603) from 38 youths (M age = 9.84 years, SD = 1.65; 60.5% White; 52.6% male) treated for primary anxiety problems within a randomized effectiveness trial were coded. Four coders used observational measures designed to assess technical competence, global competence, protocol adherence, and the alliance. Mean item interrater reliability was .70 (SD = .09) for technical competence and .66 (SD = .05) for global competence. While most components of global competence were distinct from technical competence scores, two components showed redundancy (r > .70). Scores on both competence measures were empirically distinct (r < .70) from scores on measures of protocol adherence and the alliance. Although the measures did not fully distinguish between technical and global competence, our findings do indicate that some components of technical and global competence may provide unique information about competence.

Experience-Based Competency Training in Child and Adolescent Psychotherapy

Abstract. Background: Compared to adult psychotherapy training, there is little knowledge about the use of standardized actor-patients (SPs) for the development of professional competencies for younger patient populations. Objective: We assessed the effectiveness of two university-based training programs for diagnostic as well as intervention skills in child and adolescent psychotherapy. Methods: Psychology students took part in two courses on the development of diagnostic competencies ( N = 59) and on intervention skills ( N = 51). We applied active and experience-based learning strategies, including exercises on self-awareness, mindfulness, peer role-play, and the use of SPs. Competence gain as well as a general evaluation of the seminars were assessed by self-report. Results: For both seminars, self-reported competence increased significantly with a large effect. The retrospective evaluation of learning success and acceptance ratings of the didactic formats were high. Conclusions: Experience-based trainings that include SPs are a feasible option for child and adolescent psychotherapy. The findings are discussed from a conceptual as well as a practical perspective.

Implementing the Mindfulness-Based Interventions; Teaching Assessment Criteria (MBI:TAC) in Mindfulness-Based Teacher Training

The Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) was originally developed as a tool to assess the teaching competence of mindfulness-based program (MBP) teachers. The tool was made freely available and has since been used by mindfulness-based teacher training organisations internationally. During this time the MBI:TAC has evolved in its usage, from an assessment tool to one which informally supports how MBP teachers are trained. In this article, we first examine the rationale for implementing the MBI:TAC in MBP teacher training; second, we offer practical guidance on ways of integrating the tool into teacher training pathways with awareness of its potential and possible pitfalls; and third, we offer guidance on using the tool as a framework for giving effective feedback to trainees on their teaching practice. Implementing the MBI:TAC in teacher training may support the quality and integrity of MBP teacher training, and thus ensure high quality MBP teachers graduating. In turn this may help avoid the 'implementation cliff' - that is, the quality of an intervention delivery is delivered in optimal conditions when it is being researched, and drops in quality when delivered in sub-optimal, 'real world' conditions.

An investigation into the factor structure of the Cognitive Therapy Scale - Revised (CTS-R) in a CBT training sample

BACKGROUND

Effective monitoring of cognitive behaviour therapy (CBT) competence depends on psychometrically robust assessment methods. While the UK Cognitive Therapy Scale - Revised (CTS-R; Blackburn et al., 2001) has become a widely used competence measure in CBT training, practice and research, its underlying factor structure has never been investigated.

AIMS

This study aimed to present the first investigation into the factor structure of the CTS-R based on a large sample of postgraduate CBT trainee recordings.

METHOD

Trainees (n = 382) provided 746 mid-treatment audio recordings for depression (n = 373) and anxiety (n = 373) cases scored on the CTS-R by expert markers. Tapes were split into two equal samples counterbalanced by diagnosis and with one tape per trainee. Exploratory factor analysis was conducted. The suggested factor structure and a widely used theoretical two-factor model were tested with confirmatory factor analysis. Measurement invariance was assessed by diagnostic group (depression versus anxiety).

RESULTS

Exploratory factor analysis suggested a single-factor solution (98.68% explained variance), which was supported by confirmatory factor analysis. All 12 CTS-R items were found to contribute to this single factor. The univariate model demonstrated full metric invariance and partial scalar invariance by diagnosis, with one item (item 10 - Conceptual Integration) demonstrating scalar non-invariance.

CONCLUSIONS

Findings indicate that the CTS-R is a robust homogenous measure and do not support division into the widely used theoretical generic versus CBT-specific competency subscales. Investigation into the CTS-R factor structure in other populations is warranted.

Testing the Treatment Integrity of the Managing Cancer and Living Meaningfully Psychotherapeutic Intervention for Patients With Advanced Cancer

Introduction

The Managing Cancer and Living Meaningfully (CALM) therapy for patients with advanced cancer was tested against a supportive psycho-oncological counseling intervention (SPI) in a randomized controlled trial (RCT). We investigated whether CALM was delivered as intended (therapists' adherence); whether CALM therapists with less experience in psycho-oncological care show higher adherence scores; and whether potential overlapping treatment elements between CALM and SPI can be identified (treatment differentiation).

Methods

Two trained and blinded raters assessed on 19 items four subscales of the Treatment Integrity Scale covering treatment domains of CALM (SC: Symptom Management and Communication with Health Care Providers; CSR: Changes in Self and Relationship with Others; SMP: Spiritual Well-being and Sense of Meaning and Purpose; FHM: Preparing for the Future, Sustaining Hope and Facing Mortality). A random sample of 150 audio recordings (75 CALM, 75 SPI) were rated on a three-point Likert scale with 1 = "adherent to some extent," 2 = "adherent to a sufficient extent," 3 = "very adherent."

Results

All 19 treatment elements were applied, but in various frequencies. CALM therapists most frequently explored symptoms and/or relationship to health care providers (SC_1: n__applied = 62; 83%) and allowed expression of sadness and anxiety about the progression of disease (FHM_2: n__applied = 62; 83%). The exploration of CALM treatment element SC_1 was most frequently implemented in a satisfactory or excellent manner (n__{sufficient or very adherent} = 34; 45%), whereas the treatment element SMP_4: Therapist promotes acknowledgment that some life goals may no longer be achievable (n__{sufficient or very adherent} = 0; 0%) was not implemented in a satisfactory manner. In terms of treatment differentiation, no treatment elements could be identified which were applied significantly more often by CALM therapists than by SPI therapists.

Conclusion

Results verify the application of CALM treatment domains. However, CALM therapists' adherence scores indicated manual deviations. Furthermore, raters were not able to significantly distinguish CALM from SPI, implying that overlapping treatment elements were delivered to patients.

Behavioural couple therapy: measuring therapist competence

This paper outlines a pilot validation study of the newly developed Behavioural Couple Therapy Scale for Depression (BCTS-D). The BCTS-D aims to assess therapist competence in delivering behavioural couple therapy (BCT) and provide therapists with summative and formative feedback on their performance. Completed by both therapist and supervisor, this will aid therapists’ reflection on practice and improve performance. This paper will report on two stages in the development of the BCTS-D: (a) a study evaluating content validity, face validity and usability and (b) a focus group examining usability and utility. Both parts of the study were conducted in the context of a BCT training course and included 20 participants who were either BCT supervisors or BCT trainees. Results suggest that the BCTS-D has good face validity, content validity and usability, and provides a useful tool for promoting self-reflection and providing formative feedback. The studies also provided insight into the strengths of the scale and into areas of refinement, and a number of modifications were undertaken to improve the BCTS-D in response to feedback collected. Future research will need to focus on evaluating the psychometric properties of the BCTS-D and continue to adapt the scale to its users’ needs.

Key learning aims

(1)

Can Psychotherapy Trainees Distinguish Standardized Patients From Real Patients?

Abstract. Background: Under the new psychotherapy law in Germany, standardized patients (SPs) are to become a standard component in psychotherapy training, even though little is known about their authenticity. Objective: The present pilot study explored whether, following an exhaustive two-day SP training, psychotherapy trainees can distinguish SPs from real patients. Methods: Twenty-eight psychotherapy trainees ( M = 28.54 years of age, SD = 3.19) participated as blind raters. They evaluated six video-recorded therapy segments of trained SPs and real patients using the Authenticity of Patient Demonstrations Scale. Results: The authenticity scores of real patients and SPs did not differ ( p = .43). The descriptive results indicated that the highest score of authenticity was given to an SP. Further, the real patients did not differ significantly from the SPs concerning perceived impairment ( p = .33) and the likelihood of being a real patient ( p = .52). Conclusions: The current results suggest that psychotherapy trainees were unable to distinguish the SPs from real patients. We therefore strongly recommend incorporating training SPs before application. Limitations and future research directions are discussed.

Prädiktoren des Lernerfolges eines erfahrungsbasierten Trainingsseminars zu psychodynamischen Interventionen

<b><i>Hintergrund:</i></b> Psychodynamische Psychotherapien spielen eine wichtige Rolle im deutschen Gesundheitssystem. Der Nachweis ihrer Lehrbarkeit bekommt vor dem Hintergrund des neuen Psychotherapiestudiums eine neue Bedeutung. Dies bezieht sich insbesondere auf erfahrungsbasierte di­daktische Modelle und auf das spezifische Üben von Interventionen. Hier ist bisher unklar, welche Aspekte kompetenz­orientierter psychodynamischer Lehre mit dem subjektiven Lernerfolg zusammenhängen. <b><i>Methode:</i></b> In einer Stichprobe mit 47 Kandidatinnen und Kandidaten eines Ausbildungsinstitutes wurden psychodynamische Interventionen mit ­Videofeedback, Peer-Rollenspielen und Simulationspatientinnen und -patienten unterrichtet. Der subjektive Kompetenzgewinn von Prä zu Post wurde in Bezug auf die Interventionen Klarifizieren, Konfrontieren und Deuten mithilfe von <i>t</i>-Tests abgebildet. Zudem wurden in einer multiplen linearen Regression verschiedene Prädiktoren für den subjektiven Lernerfolg getestet. <b><i>Ergebnisse:</i></b> Die subjektive Kompetenz nahm für die Interventionen Klarifizieren und Konfrontieren mit großen Effekten, für Deuten mit einem kleinen bis ­mittleren Effekt zu. Der Lernerfolg war korreliert mit einem antizipierten Transfer in den therapeutischen Alltag, einem positiven Erleben der Arbeit mit den Simulationspatientinnen und -patienten sowie mit hilfreich erlebtem Feedback durch die Dozentinnen und Dozenten. <b><i>Schlussfolgerungen:</i></b> Psychodynamische Interventionen sind mit erfahrungsbasierten und kompetenzorientierten Unterrichtsformaten lehrbar. Dabei ist wichtig, dass praxisorientierte Formate eine Lernumgebung schaffen, in der die Arbeit mit Simulationspatientinnen und -patienten und das Feedback als konstruktiv und hilfreich erlebt werden. Zukünftige Untersu­chungen sollten den Transfer in den Alltag sowie longitudinale Curricula in den Fokus nehmen.

Use of role plays to assess therapist competency and its association with client outcomes in psychological interventions: A scoping review and competency research agenda

A major challenge in scaling-up psychological interventions worldwide is how to evaluate competency among new workforces engaged in psychological services. One approach to measuring competency is through standardized role plays. Role plays have the benefits of standardization and reliance on observed behavior rather than written knowledge. However, role plays are also resource intensive and dependent upon inter-rater reliability. We undertook a two-part scoping review to describe how competency is conceptualized in studies evaluating the relationship of competency with client outcomes. We focused on use of role plays including achieving inter-rater reliability and the association with client outcomes. First, we identified 4 reviews encompassing 61 studies evaluating the association of competency with client outcomes. Second, we identified 39 competency evaluation tools, of which 21 were used in comparisons with client outcomes. Inter-rater reliability (intraclass correlation coefficient) was reported for 15 tools and ranged from 0.53 to 0.96 (mean ICC = 0.77). However, we found that none of the outcome comparison studies measured competency with standardized role plays. Instead, studies typically used therapy quality (i.e., session ratings with actual clients) as a proxy for competency. This reveals a gap in the evidence base for competency and its role in predicting client outcomes. We therefore propose a competency research agenda to develop an evidence-base for objective, standardized role plays to measure competency and its association with client outcomes. OPEN SCIENCE REGISTRATION #: https://osf.io/nqhu7/.

Warum nehmen Verhaltenstherapeut_innen an Fortbildungen teil?

Zusammenfassung. Theoretischer Hintergrund: Um psychotherapeutische Kompetenzen zu erhalten, kommt Fortbildungen eine zentrale Rolle zu. Fragestellung: Ziel war es, Verhaltenstherapeut_innen zu Motivatoren und Barrieren bezüglich der Teilnahme an Fortbildungen und zu Einstellungen zu evidenzbasierter Praxis zu befragen. Methode: In die Befragung einbezogen wurden N = 48 Teilnehmer_innen eines zweitägigen Workshops zu aktuellen Ansätzen der Traumatherapie. Fortbildungsmotivation, Teilnahmebarrieren und Einstellungen zu evidenzbasierter Praxis wurden in Zusammenhang mit der aktuellen Fortbildungsteilnahme geprüft. Ergebnisse: Der Wunsch nach beruflicher Verbesserung und persönliches Interesse motivierten am häufigsten, soziale Anreize und beruflicher Aufstieg waren weniger relevant. Die Vereinbarkeit von Fortbildungen mit der Arbeitstätigkeit und Fortbildungskosten waren häufige Barrieren. Schlussfolgerungen: Die Ergebnisse liefern Hinweise zur Diskussion des Konzepts der evidenzbasierten Psychotherapie und zur Dissemination wissenschaftlicher Ergebnisse in die Praxis.

Assessing Treatment Fidelity within an Epilepsy Randomized Controlled Trial: Seizure First Aid Training for People with Epilepsy Who Visit Emergency Departments

Purpose

To measure fidelity with which a group seizure first aid training intervention was delivered within a pilot randomized controlled trial underway in the UK for adults with epilepsy who visit emergency departments (ED) and informal carers. Estimates of its effects, including on ED use, will be produced by the trial. Whilst hardly ever reported for trials of epilepsy interventions—only one publication on this topic exists—this study provides the information on treatment fidelity necessary to allow the trial's estimates to be accurately interpreted. This rare worked example of how fidelity can be assessed could also provide guidance sought by neurology trialists on how to assess fidelity.

Methods

53 patients who had visited ED on ≥2 occasions in prior year were recruited for the trial; 26 were randomized to the intervention. 7 intervention courses were delivered for them by one facilitator. Using audio recordings, treatment “adherence” and “competence” were assessed. Adherence was assessed by a checklist of the items comprising the intervention. Using computer software, competence was measured by calculating facilitator speech during the intervention (didacticism). Interrater reliability was evaluated by two independent raters assessing each course using the measures and their ratings being compared.

Results

The fidelity measures were found to be reliable. For the adherence instrument, raters agreed 96% of the time, PABAK-OS kappa 0.91. For didacticism, raters' scores had an intraclass coefficient of 0.96. In terms of treatment fidelity, not only were courses found to have been delivered with excellent adherence (88% of its items were fully delivered) but also as intended they were highly interactive, with the facilitator speaking for, on average, 55% of course time.

Conclusions

The fidelity measures used were reliable and showed that the intervention was delivered as attended. Therefore, any estimates of intervention effect will not be influenced by poor implementation fidelity.

Therapeutic alliance-not therapist competence or group cohesion-contributes to reduction of psychological distress in group-based mindfulness-based cognitive therapy for cancer patients

Mindfulness‐based cognitive therapy (MBCT) is an innovative evidence‐based intervention in mental and somatic health care. Gaining knowledge of therapeutic factors associated with treatment outcome can improve MBCT. This study focused on predictors of treatment outcome of MBCT for cancer patients and examined whether group cohesion, therapeutic alliance, and therapist competence predicted reduction of psychological distress after MBCT for cancer patients. Moreover, it was examined whether therapist competence facilitated therapeutic alliance or group cohesion. Multilevel analyses were conducted on a subsample of patients collected in a larger randomized controlled trial on individual internet‐based versus group‐based MBCT versus treatment as usual in distressed cancer patients. The current analyses included the 84 patients who completed group‐based MBCT out of 120 patients who were randomized to group‐based MBCT. Group cohesion and therapist competence did not predict reduction in psychological distress, whereas therapeutic alliance did. In addition, therapist competence did not predict therapeutic alliance but was associated with reduced group cohesion. Our findings revealed that therapeutic alliance significantly contributed to reduction of psychological distress in MBCT for cancer patients. Elaborating the clinical implications of the predictive significance of therapeutic alliance might be of added value to enhance the potential effect of MBCT.

Therapists' techniques in the treatment of adolescent depression

When comparing the relative effectiveness of different psychological treatment approaches using clinical trials, it is essential to establish fidelity to each manualized therapy, and differentiation between the treatment arms. Yet few psychological therapy trials include details about the assessment of treatment integrity and little is known about the specific techniques used by therapists, or to what degree these techniques are shared or distinct across different therapeutic approaches. The aims of this study were: to establish the fidelity of two established psychological therapies - cognitive-behaviour therapy (CBT) and short-term psychoanalytic psychotherapy (STPP) - in the treatment of adolescent depression; and to examine whether they were delivered with adherence to their respective treatment modalities, and if they could be differentiated from each other and from a reference treatment (a brief psychosocial intervention; BPI). The study also aimed to identify shared and distinct techniques used within and across the three treatments. Audio-tapes (N=230) of therapy sessions, collected as part of a trial, were blind double-rated using the Comparative Psychotherapy Process Scale (CPPS), which includes subscales for cognitive-behavioural and psychodynamic-interpersonal techniques. The treatments were delivered with reasonable fidelity and there was clear differentiation in the use of cognitive-behavioural and psychodynamic-interpersonal techniques between CBT and STPP, and between these two established psychological therapies and BPI. An item-level analysis identified techniques used across all three treatments, techniques that were shared between BPI and CBT, and techniques that were unique to CBT and STPP.

How beginning cognitive behavioural therapists develop professional confidence

Evidence exists that the effectiveness of psychotherapy depends more on therapists’ variables than on their theoretical orientation or the techniques they use. Nevertheless, relatively little is known regarding the process of cognitive behavioural psychotherapists’ development. The purpose of the study was to explore how beginning cognitive behavioural therapy (CBT) practitioners develop, considering various professional and personal influences. Eight in-depth interviews with beginning therapists were conducted, and the Grounded Theory Method was used for data analysis. The developmental process was conceptualized as Gaining Professional Confidence, and three phases of this process were identified: (1) Externally Based Confidence in CBT Methods, (2) Internalized Confidence in CBT Methods, and (3) Therapists’ Self-Confidence. The results indicate that trainees’ self-reflection on their personal qualities, values, attitudes and preferences should be given more attention in CBT training, as this plays a crucial role in their overall professional development.

A proposed conceptualization of therapeutic competence: the three level model

A model of therapeutic competence that equally satisfies the requirements of practice and research is still lacking. The existing models are not widely accepted, at least partially because the postulated competences can often not be operationalized in a satisfactory manner. Yet, in order to be measurable, therapeutic competences need to be operationalized. We present the Three Level Model of Therapeutic Competence as a working model for studying therapeutic competence. The model proposes that therapeutic competence develops based on rather stable individual Dispositions, which promote the acquisition of therapeutic competences. We further distinguish between Basic Competences, which are mostly independent of the theoretical orientation of the therapeutic approach, and Specific Competences, which are defined based on the theoretical underpinnings of a therapeutic orientation (e.g. Cognitive Behavioral Therapy). We describe this model and outline how it can be used to operationalize and assess therapeutic competence.

Intervention Integrity in Mindfulness-Based Research

Assessing program or intervention fidelity/integrity is an important methodological consideration in clinical and educational research. These critical variables influence the degree to which outcomes can be attributed to the program and the success of the transition from research to practice and back again. Research in the Mindfulness-Based Program (MBP) field has been expanding rapidly over the last 20 years, but little attention has been given to how to assess intervention integrity within research and practice settings. The proliferation of different program forms, inconsistency in adhering to published curriculum guides, and variability of training levels and competency of trial teachers all pose grave risks to the sustainable development of the science of MBPs going forward. Three tools for assessing intervention integrity in the MBP field have been developed and researched to assess adherence and/or teaching competence: the Mindfulness-Based Cognitive Therapy-Adherence Scale (MBCT-AS), the Mindfulness-Based Relapse Prevention-Adherence and Competence Scale (MBRP-AC), and the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC). Further research is needed on these tools to better define their inter-rater reliability and their ability to measure elements of teaching competence that are important for participant outcomes. Research going forward needs to include systematic and consistent methods for demonstrating and verifying that the MBP was delivered as intended, both to ensure the rigor of individual studies and to enable different studies of the same MBP to be fairly and validly compared with each other. The critical variable of the teaching also needs direct investigation in future research. We recommend the use of the "Template for Intervention Description and Replication" (TIDieR) guidelines for addressing and reporting on intervention integrity during the various phases of the conduct of research and provide specific suggestions about how to implement these guidelines when reporting studies of mindfulness-based programs.

The Psychotherapy Outcome Problem: The Development of the Outcome Stability Index

OBJECTIVE

Psychotherapy research commonly utilizes clients' last session score as an indicator of treatment outcome. We examined whether this last session score is consistent with what would be predicted based on clients' general trajectory in psychological functioning across sessions. We focused on the unstandardized residual variance at the last session, as this represents the degree to which the session score is divergent (or not) from what is predicted from the previous sessions (i.e., Outcome Stability Index; OSI).

METHOD

The sample included 27,958 clients who attended on average 9.41 sessions. Each session, clients completed the Behavioral Health Measure-20 as a measure of psychological functioning. We converted the unstandardized residual variance for clients' last session score into a Cohen's d coefficient to aid in interpretation.

RESULTS

The mean OSI was 0.07 (SD = 0.58), suggesting excellent stability in their last session therapy outcome scores. However, approximately 33% of clients demonstrated poor or extremely problematic stability in their last session therapy outcome scores. Clients who demonstrated poor stability were classified as demonstrating reliable deterioration.

CONCLUSIONS

Researchers may want to consider reporting OSI to assist readers' understanding of the stability of therapy outcomes. Clinical or methodological significance of this article: Therapy outcome scores can vary from session to session, which can influence how we understand therapy outcomes that rely on last session scores. Studies examining therapy outcomes could report the Outcome Stability Index to better contextualize the results.

Implementation fidelity of a self-management course for epilepsy: method and assessment

BACKGROUND

Complex interventions such as self-management courses are difficult to evaluate due to the many interacting components. The way complex interventions are delivered can influence the effect they have for patients, and can impact the interpretation of outcomes of clinical trials. Implementation fidelity evaluates whether complex interventions are delivered according to protocol. Such assessments have been used for one-to-one psychological interventions; however, the science is still developing for group interventions.

METHODS

We developed and tested an instrument to measure implementation fidelity of a two-day self-management course for people with epilepsy, SMILE(UK). Using audio recordings, we looked at adherence and competence of course facilitators. Adherence was assessed by checklists. Competence was measured by scoring group interaction, an overall impression score and facilitator "didacticism". To measure "didacticism", we developed a novel way to calculate facilitator speech using computer software. Using this new instrument, implementation fidelity of SMILE(UK) was assessed on three modules of the course, for 28% of all courses delivered.

RESULTS

Using the instrument for adherence, scores from two independent raters showed substantial agreement with weighted Kappa of 0.67 and high percent agreement of 81.2%. For didacticism, the results from both raters were highly correlated with an intraclass coefficient of 0.97 (p < 0.0001). We found that the courses were delivered with a good level of adherence (> 50% of scored items received the maximum of 2 points) and high competence. Groups were interactive (mean score: 1.9-2.0 out of 2) and the overall impression was on average assessed as "good". Didacticism varied from 42% to 93% of total module time and was not associated with the other competence scores.

CONCLUSION

The instrument devised to measure implementation fidelity was reproducible and easy to use. The courses for the SMILE(UK) study were delivered with a good level of adherence to protocol while not compromising facilitator competence.

TRIAL REGISTRATION

ISRCTN57937389 .

Topics and techniques in clinical supervision in psychotherapy training

Clinical supervision is regarded as one of the most important components of psychotherapy training. In clinical practice, it has been found that the implementation of clinical supervision varies substantially and often differs from the recommendations made in the literature. The objective of the current study was to investigate the frequency of topics (e.g. ethical issues) and techniques (e.g. role play) in the clinical supervision of psychotherapy trainees in Germany. To this end, we considered supervisions in cognitive behavioural therapy (CBT) and psychodynamic therapy (PT). A total of 791 psychotherapy trainees (533 CBT and 242 PT) were asked via the internet to provide information about their current supervision sessions. We found that clinical supervision in psychotherapy training addressed topics that are central for the effective treatment of supervised patients (i.e. therapeutic interventions, therapeutic alliance, maintaining factors, and therapeutic goals). However, the most frequently used intervention in clinical supervision in psychotherapy training was case discussion. Rarely were techniques used that allowed the supervisor to give the supervisee feedback based on the supervisee's demonstrated competencies. For example, 46% of the supervisors never used audiotapes or videotapes in the supervision. Differences between CBT and PT were rather small. Current practice regarding the techniques used in clinical supervision for psychotherapy trainees contradicts recommendations for active and feedback-oriented clinical supervision. Thus the potential of clinical supervision might not be fully used in clinical practice.

Impact of Mindfulness-Based Teacher Training on MBSR Participant Well-Being Outcomes and Course Satisfaction

Growing interest in mindfulness-based programs (MBPs) has resulted in increased demand for MBP teachers, raising questions around safeguarding teaching standards. Training literature emphasises the need for appropriate training and meditation experience, yet studies into impact of such variables on participant outcomes are scarce, requiring further investigation. This feasibility pilot study hypothesised that participant outcomes would relate to teachers’ mindfulness-based teacher training levels and mindfulness-based teaching and meditation experience. Teachers (n = 9) with different MBP training levels delivering mindfulness-based stress reduction (MBSR) courses to the general public were recruited together with their course participants (n = 31). A teacher survey collected data on their mindfulness-based teacher training, other professional training and relevant experience. Longitudinal evaluations using online questionnaires measured participant mindfulness and well-being before and after MBSR and participant course satisfaction. Course attendees’ gains after the MBSR courses were correlated with teacher training and experience. Gains in well-being and reductions in perceived stress were significantly larger for the participant cohort taught by teachers who had completed an additional year of mindfulness-based teacher training and assessment. No correlation was found between course participants’ outcomes and their teacher’s mindfulness-based teaching and meditation experience. Our results support the hypothesis that higher mindfulness-based teacher training levels are possibly linked to more positive participant outcomes, with implications for training in MBPs. These initial findings highlight the need for further research on mindfulness-based teacher training and course participant outcomes with larger participant samples.

Therapist competencies necessary for the delivery of compassion-focused therapy: A Delphi study

OBJECTIVES

Compassion-focused therapy (CFT) has shown promising results for a range of clinical presentations. This study explored the therapeutic competencies required to deliver CFT and organized these into a coherent framework.

DESIGN

The Delphi method was used to explore and refine competencies for delivering CFT in three rounds of data collection.

METHODS

The first round involved interviews with 12 experts in CFT. Data were analysed using template analysis to generate a draft competency framework. The main competencies were used to create a survey for rounds two and three involving CFT experts and practitioners. Data collected from the surveys were used to refine the competencies.

RESULTS

The CFT competency framework (CFT-CF) that was produced comprised 25 main competencies within six key areas of competence. The areas were as follows: competencies in creating safeness, meta-skills, non-phase-specific skills, phase-specific skills, knowledge and understanding and use of supervision. The main competencies included several subcompetencies specifying knowledge, skills and attributes needed to demonstrate the main competence. Overall, there was consensus on 14 competencies and 20 competencies exceeded an 80% agreement level.

CONCLUSIONS

Some of the CFT competencies overlapped with existing therapies, whilst others were specific to CFT. The CFT-CF provides useful guidance for clinicians, supervisors and training programmes. Further research could develop the CFT-CF into a therapist rating scale in order to measure the outcome of training and to assess treatment fidelity in clinical trials.

PRACTITIONER POINTS

The compassion-focused therapy competency framework (CFT-CF) identifies therapeutic competencies that overlap with existing treatments as well as those specific to compassion-focused therapy (CFT). The CFT-CF builds guidance for the competencies required to deliver CFT in a range of clinical settings. The CFT-CF provides guidance for those training CFT therapists. The CFT-CF could be used as a basis to develop a therapist rating scale.

The Effects of Bug-in-the-Eye Supervision on Therapeutic Alliance and Therapist Competence in Cognitive-Behavioural Therapy: A Randomized Controlled Trial

Live supervision enables a supervisor to have direct insight into the psychotherapeutic process and allows him or her to provide immediate feedback to the trainee. Therefore, live supervision might be superior to traditional supervisory formats that only allow for the provision of delayed feedback. When considering the different live supervision formats, bug-in-the-eye (BITE) supervision is particularly promising because of its improved and less invasive procedure. The current study compared the efficacy of BITE supervision with that of delayed video-based (DVB) supervision. In the present study, 23 therapists were randomly assigned to either the BITE supervision or DVB supervision groups. The participants were psychotherapy trainees who treated 42 patients (19 under BITE supervision and 23 under DVB supervision) over 25 sessions of cognitive-behavioural therapy. Two independent raters blind to the treatment conditions evaluated therapeutic alliance and therapist competence based on 195 videotapes. Therapeutic alliance was significantly stronger among the treatments conducted under BITE supervision than those conducted under DVB supervision. Moreover, a higher level of therapeutic competence was found in the BITE condition than in the DVB condition. However, no differences between supervision conditions were found when the results were controlled for the level of therapeutic alliance and therapist competence demonstrated in the first session. No differences were observed between the supervision conditions with respect to patient outcomes. There is evidence that BITE supervision is able to improve therapeutic alliance and therapist competence. However, these findings should be interpreted with caution because possible pre-treatment differences between therapists might explain the superiority of BITE supervision. Copyright © 2015 John Wiley & Sons, Ltd.

KEY PRACTITIONER MESSAGE

BITE supervision positively influences the therapeutic alliance and therapeutic competencies during cognitive-behavioural therapy. A supervision format that more directly addresses therapeutic processes is more effective in improving those processes than an indirect supervision format. Pre-treatment differences between therapists might explain the superiority of BITE supervision. BITE supervision can be considered a safe intervention.

Teacher Competence in Mindfulness-Based Cognitive Therapy for Depression and Its Relation to Treatment Outcome

As mindfulness-based cognitive therapy (MBCT) becomes an increasingly mainstream approach for recurrent depression, there is a growing need for practitioners who are able to teach MBCT. The requirements for being competent as a mindfulness-based teacher include personal meditation practice and at least a year of additional professional training. This study is the first to investigate the relationship between MBCT teacher competence and several key dimensions of MBCT treatment outcomes. Patients with recurrent depression in remission (N = 241) participated in a multi-centre trial of MBCT, provided by 15 teachers. Teacher competence was assessed using the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) based on two to four randomly selected video-recorded sessions of each of the 15 teachers, evaluated by 16 trained assessors. Results showed that teacher competence was not significantly associated with adherence (number of MBCT sessions attended), possible mechanisms of change (rumination, cognitive reactivity, mindfulness, and self-compassion), or key outcomes (depressive symptoms at post treatment and depressive relapse/recurrence during the 15-month follow-up). Thus, findings from the current study indicate no robust effects of teacher competence, as measured by the MBI:TAC, on possible mediators and outcome variables in MBCT for recurrent depression. Possible explanations are the standardized delivery of MBCT, the strong emphasis on self-reliance within the MBCT learning process, the importance of participant-related factors, the difficulties in assessing teacher competence, the absence of main treatment effects in terms of reducing depressive symptoms, and the relatively small selection of videotapes. Further work is required to systematically investigate these explanations.

The challenge of training supervisors to use direct assessments of clinical competence in CBT consistently: a systematic review and exploratory training study

Evaluating and enhancing supervisee competence is a key function of supervision and can be aided by the use of direct assessments of clinical competence, e.g. the Cognitive Therapy Scale – Revised (CTS-R). We aimed to review the literature regarding inter-rater reliability and training on the CTS and CTS-R to present exploratory data on training raters to use this measure. We employed a systematic review. An exploratory study evaluated the outcomes of a CTS-R supervisor training workshop (n = 34), including self-reported familiarity with and confidence in using the tool, and inter-rater consistency on three CTS-R subscales, pre- and post-training. CTS and CTS-R inter-rater reliability was variable, with evidence of rater training enhancing reliability, although the form, duration and frequency of such training is unclear. The exploratory study found that supervisors rated themselves as more familiar with and confident in using the CTS-R at the end of training compared to at the beginning. However, inter-rater reliability was poor at the beginning and end of the training. Rating competence requires supervisors to make qualitative judgements, which is inherently variable. Training raters has been shown to improve rater reliability, although this was not demonstrated in the exploratory study. Practice implications and future research priorities are identified.

Assessing fidelity of cognitive behavioral therapy in rural VA clinics: design of a randomized implementation effectiveness (hybrid type III) trial

BACKGROUND

Broadly disseminating and implementing evidence-based psychotherapies with high fidelity, particularly cognitive behavioral therapy (CBT), has proved challenging for many health-care systems, including the Department of Veterans Affairs, especially in primary care settings such as small or remote clinics. A computer-based tool (based on the coordinated anxiety learning and management (CALM) program) was designed to support primary care-based mental health providers in delivering CBT. The objectives of this study are to modify the CALM tool to meet the needs of mental health clinicians in veterans affairs (VA) community-based outpatient clinics (CBOCs) and rural "veterans", use external facilitation to implement CBT and determine the effect of the CALM tool versus a manualized version of CALM to improve fidelity to the CBT treatment model, and conduct a needs assessment to understand how best to support future implementation of the CALM tool in routine care.

METHODS/DESIGN

Focus groups will inform the redesign of the CALM tool. Mental health providers at regional VA CBOCs; CBT experts; VA experts in implementation of evidence-based mental health practices; and veterans with generalized anxiety disorder, panic disorder, social anxiety disorder, posttraumatic stress disorder, "with or without" depression will be recruited. A hybrid type III design will be used to examine the effect of receiving CBT training plus either the CALM tool or a manual version of CALM on treatment fidelity. External facilitation will be used as the overarching strategy to implement both CBT delivery methods. Data will also be collected on symptoms of the targeted disorders. To help prepare for the future implementation of the CALM tool in VA CBOCs, we will perform an implementation need assessment with mental health providers participating in the clinical trial and their CBOC directors.

DISCUSSION

This project will help inform strategies for delivering CBT with high fidelity in VA CBOCs to veterans with anxiety disorders and PTSD with or without depression. If successful, results of this study could be used to inform a national rollout of the CALM tool in VA CBOCs including providing recommendations for optimizing the adoption and sustained use of the computerized CALM tool among mental health providers in this setting.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02488551.

Excessive behaviors in clinical practice--A state of the art article

This paper concerns difficulties with excessive food intake, sexual activities, romantic relationships, gambling, Internet use, shopping, and exercise—behaviors that might cause considerable suffering. Excessive behaviors are seen as expressions of underlying difficulties that often co-occur with other psychological difficulties, and behaviors may accompany or replace each other. Moreover, they might pass unnoticed in clinical practice. Given the complexity of excessive behaviors, integrated and individualized treatment has been recommended. This paper presents an overview of the terminology concerning excessive behaviors, and the impact of naming is acknowledged. Thereafter, methods for identification and assessment, as well as treatment needs are discussed. Because identification, assessment, and treatment occur in an interaction between client and practitioner, this paper presents a discussion of the need to empower practitioners to identify and assess excessive behaviors and provide an integrated treatment. Moreover, the need to support practitioners’ capacity to handle and tolerate the overwhelming suffering and the negative consequences connected to excessive behaviors is discussed. Qualitative studies are suggested in order to understand the meaning of excessive behaviors, treatment needs, and the interaction between client and practitioner.

Competence feedback improves CBT competence in trainee therapists: A randomized controlled pilot study

OBJECTIVE

The development and improvement of therapeutic competencies are central aims in psychotherapy training; however, little is known about which training interventions are suitable for the improvement of competencies.

METHOD

In the current pilot study, the efficacy of feedback regarding therapeutic competencies was investigated in cognitive behavioural therapy (CBT). Totally 19 trainee therapists and 19 patients were allocated randomly to a competence feedback group (CFG) or control group (CG). Two experienced clinicians and feedback providers who were blind to the treatment conditions independently evaluated therapeutic competencies on the Cognitive Therapy Scale at five treatment times (i.e., at Sessions 1, 5, 9, 13, and 17). Whereas CFG and CG included regular supervision, only therapists in the CFG additionally received written qualitative and quantitative feedback regarding their demonstrated competencies in conducting CBT during treatment.

RESULTS

We found a significant Time × Group interaction effect (η² = .09), which indicates a larger competence increase in the CFG in comparison to the CG.

CONCLUSIONS

Competence feedback was demonstrated to be suitable for the improvement of therapeutic competencies in CBT. These findings may have important implications for psychotherapy training, clinical practice, and psychotherapy research. However, further research is necessary to ensure the replicability and generalizability of the findings.

Therapist drift redux: Why well-meaning clinicians fail to deliver evidence-based therapy, and how to get back on track

Therapist drift occurs when clinicians fail to deliver the optimum evidence-based treatment despite having the necessary tools, and is an important factor in why those therapies are commonly less effective than they should be in routine clinical practice. The research into this phenomenon has increased substantially over the past five years. This review considers the growing evidence of therapist drift. The reasons that we fail to implement evidence-based psychotherapies are considered, including our personalities, knowledge, emotions, beliefs, behaviours and social milieus. Finally, ideas are offered regarding how therapist drift might be halted, including a cognitive-behavioural approach for therapists that addresses the cognitions, emotions and behaviours that drive and maintain our avoidance of evidence-based treatments.

Therapist competence and therapeutic alliance are important in the treatment of health anxiety (hypochondriasis)

The role of treatment delivery factors (i.e., therapist adherence, therapist competence, and therapeutic alliance) is rarely investigated in psychotherapeutic treatment for health anxiety. This study aimed to investigate the role of the assessment perspective for the evaluation of treatment delivery factors and their relevance for treatment outcome. Therapist adherence, therapist competence, and therapeutic alliance were evaluated by independent raters, therapists, patients, and supervisors in 68 treatments. Patients with severe health anxiety (hypochondriasis) were treated with cognitive therapy or exposure therapy. Treatment outcome was assessed with a standardized interview by independent diagnosticians. A multitrait-multimethod analysis revealed a large effect for the assessment perspective of therapist adherence, therapist competence, and therapeutic alliance. The rater perspective was the most important for the prediction of treatment outcome. Therapeutic alliance and therapist competence accounted for 6% of the variance of treatment outcome while therapist adherence was not associated with treatment outcome. Therapist competence was only indirectly associated with treatment outcome, mediated by therapeutic alliance. Both therapeutic alliance and therapist competence demonstrated to be important treatment delivery factors in psychotherapy for health anxiety. A stronger consideration of those processes during psychotherapy for health anxiety might be able to improve psychotherapy outcome.

Kompetenzbereiche in der psychotherapeutischen Ausbildung

Theoretischer Hintergrund: Die Kompetenzbasierung der Psychotherapieausbildung wird immer intensiver diskutiert. Ein Konsens über Komponenten psychotherapeutischer Kompetenz und deren Wichtigkeit in der Ausbildung besteht bisher jedoch nicht. Fragestellung: In diesem Beitrag wird der Frage nachgegangen, welche Kompetenzen Ausbildungsteilnehmer am Ende der Ausbildung erreicht haben sollten und für wie wichtig welche Kompetenzen angesehen werden. Methode: Die offenen Antworten von Leitern psychotherapeutischer Ausbildungsinstitute (N = 129) aus dem Forschungsgutachten zur Psychotherapieausbildung wurden mit qualitativer Methodik ausgewertet. Ergebnisse: Vor allem fachlich-konzeptuelle Kompetenzen (57,2 %) wurden häufig genannt, aber auch personale (21,0 %) und interpersonale (21,9 %). Schlussfolgerungen: Die Aussagen der Ausbildungsleiter stimmen im Wesentlichen mit bestehenden Kompetenzdefinitionen überein, wenngleich die Betonung von Theoriewissen bzw. Fachkompetenzen zulasten von Beziehungsgestaltungs- und personalen Kompetenzen sich nicht ganz mit den Befunden und Kompetenzkonzepten der Psychotherapieforschung deckt.

Schlüsselwörter: Psychotherapie, Ausbildung, Kompetenz, Qualitative Analyse