Within-Provider Variability in Motivational Interviewing Integrity for Three Years after MI Training: Does Time Heal?

Clinician motivational interviewing skills in 'simulated' and 'real-life' consultations differ and show predictive validity for 'real life' client change talk under differing integrity thresholds

Background

Accurate and reliable assessment of clinician integrity in the delivery of empirically supported treatments is critical to effective research and training interventions. Assessment of clinician integrity can be performed through recording simulated (SI) or real-life (RL) consultations, yet research examining the equivalence of these data is in its infancy. To explore the strength of integrity assessment between SI and RL samples in Motivational Interviewing (MI) consultations, this article examines whether Motivational Interviewing Treatment Integrity (MITI) assessments differ between SI and RL consultations and reviews the predictive validity of SI and RL MI skills categorisations for RL client response language.

Methods

This study first compared MITI coding obtained in SI and RL consultations for 36 veterinary clinicians. Multilevel models of 10 MITI behaviour counts and four MITI global scores were run using MLwiN 3.02 to assess if a significant difference existed between SI and RL MITI data, with consultation within clinician within cohort (A and B) as nested random effects. Second, we investigated the effect of SI and RL MI skills groupings on rate of RL client response talk using three multivariable regression models. Two Poisson regression models, with random intercepts for farm and veterinarian and offset for number of minutes of the recordings, were estimated in the statistical software R using the package glmmTMB for the two response variables Change Talk and Sustain Talk. A logistic regression model, with the same random intercepts, with the response variable Proportion Change Talk was also estimated using the same package.

Results

Veterinary clinicians were less MI consistent in RL consultations, evidenced through significantly lower global MITI Cultivating Change Talk (p < 0.001), Partnership (p < 0.001) and Empathy (p = 0.003) measures. Despite lower objective MI skills groupings in RL consultations, ranking order of veterinary clinicians by MI skills was similar between contexts. The predictive validity of SI and RL MI skills groupings for RL client Change Talk was therefore similar, with significantly more RL client Change Talk associated with veterinarians categorised in the highest grouping ('moderate') in both SI (p = 0.01) and RL (p = 0.02) compared to untrained veterinarians in each respective context.

Conclusions

Findings suggest SI and RL data may not be interchangeable. Whilst both data offer useful insights for specific research and training purposes, differing contextual MI skills thresholds may offer a more equitable assessment of clinician RL client-facing MI integrity. Further research is needed to explore the applicability of these findings across health contexts.

Enhancing and assessing fidelity in the TANDEM (Tailored intervention for ANxiety and DEpression Management in COPD) trial: development of methods and recommendations for research design

BACKGROUND

Development of complex interventions for management of chronic conditions has become increasingly common, with guidance now provided. Fidelity (whether the intervention is designed, delivered and received as intended) is critical to understanding if, and how an intervention works (or not). However, methods for achieving this are still evolving. This study describes the methods used in the TANDEM trial - a large multicentre study evaluating the impact of a cognitive behavioural intervention preceding routine pulmonary rehabilitation for people with chronic obstructive pulmonary disease and anxiety and or depression. Results for enhancement and training aspects of fidelity, are presented.

METHODS

Using the National Institute for Health Behaviour Change Consortium (NIH BCC) framework of fidelity, a set of enhancement strategies and a fidelity measurement strategy were developed with input from a multidisciplinary team. The Cognitive First Aid Rating Scale (CFARS) was used to assess Facilitator (the respiratory professional delivering TANDEM) therapeutic competence at the end of the initial training and throughout treatment delivery (on a randomly selected set of cases). A TANDEM specific treatment adherence measure was developed following previously recommended procedures. Together these (the CFARS and adherence measure) comprised the TANDEM treatment delivery fidelity tool.

RESULTS

Hiring of respiratory professionals to the initial training programme was successful, with 44% of those expressing initial interest in being a Facilitator successfully completing the process. Video recordings of potential Facilitators conducting standardized patient role plays at the end of the initial training demonstrated fidelity of training.

CONCLUSIONS

Addressing fidelity in complex intervention trials is a time and resource intensive process but has significant potential to increase understanding of results and strengthen the evidence base for effective interventions. By defining a full fidelity assessment method prior to analysis we aimed to minimise bias when interpreting results.

TRIAL REGISTRATION

ISRCTN59537391 . Registered on 20 March 2017. Trial protocol version 6.0, 22 April 2018. Process evaluation protocol version 4.0, 1 November 2020.

Self-Coding of Fidelity as a Potential Active Ingredient of Consultation to Improve Clinicians' Fidelity

A key goal for implementation science is the identification of evidence-based consultation protocols and the active ingredients within these protocols that drive clinician behavior change. The current study examined clinicians' self-coding of fidelity as a potential active ingredient of consultation for the Attachment and Biobehavioral Catch-up (ABC) intervention. It also examined two other potential predictors of clinician fidelity in response to consultation: dosage of consultation and working alliance. Twenty-nine clinicians (97% female, 62% White, M age = 34 years) participated in a year of weekly fidelity-focused ABC consultation sessions, for which clinicians self-coded fidelity and received consultant feedback on both their coding and their fidelity. Data from the ABC fidelity measure were available for 1067 sessions coded by consultants, and clinicians' self-coding accuracy was calculated from 1044 sessions coded by both clinicians and consultants. Alliance was measured with the Working Alliance Inventory-Trainee and Supervisor Versions. The study was observational, and fidelity and self-coding accuracy were modeled across time using hierarchical linear modeling. Clinicians' ABC fidelity, as well as their self-coding accuracy, increased over the course of consultation. Clinicians' self-coding accuracy predicted their initial fidelity and growth in fidelity. Working alliance was also linked to fidelity and self-coding accuracy. These results suggest that clinician self-coding should be further examined as an active ingredient of consultation. The study has important implications for the design of consultation procedures and fidelity assessments.

Barriers and Facilitators for Implementing Motivational Interviewing as a Return to Work Intervention in a Norwegian Social Insurance Setting: A Mixed Methods Process Evaluation

Purpose The aim of this study was to evaluate potential barriers and facilitators for implementing motivational interviewing (MI) as a return to work (RTW) intervention in a Norwegian social insurance setting. Methods A mixed-methods process evaluation was conducted alongside a randomized controlled trial involving MI sessions delivered by social insurance caseworkers. The study was guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework using focus groups with the caseworkers. MI fidelity was evaluated through audio-recordings of MI sessions and questionnaires to sick-listed participants. Results Lack of co-worker and managerial support, time and place for practicing to further develop MI skills, and a high workload made the MI intervention challenging for the caseworkers. The MI method was experienced as useful, but difficult to master. MI fidelity results showed technical global scores over the threshold for "beginning proficiency" whereas the relational global score was under the threshold. The sick-listed workers reported being satisfied with the MI sessions. Conclusions Despite caseworker motivation for learning and using MI in early follow-up sessions, MI was hard to master and use in practice. Several barriers and facilitators were identified; these should be addressed before implementing MI in a social insurance setting.Trial registration ClinicalTrials.gov: NCT03212118 (registered July 11, 2017).

Ongoing supervision as a method to implement Motivational interviewing: Results of a randomized controlled trial

OBJECTIVE

To assess skills in Motivational interviewing (MI) at the Swedish National Board of Institutional Care (SiS), and to evaluate different ways to provide MI supervision.

METHODS

SiS practitioners (n = 134) were randomized to regular group supervision, or individual telephone supervision based on only the behavioral component of a feedback protocol, or the full protocol. Participant's mean age was 43.2 (SD =10.2), and the majority (62.7%) were females.

RESULTS

Many participants showed beginning proficiency already at baseline, indicating a successful implementation. Still, results varied widely. The regular supervision and the supervision based on objective feedback were equally effective, and the group receiving feedback based on fewer variables of the protocol performed better on only one of the seven skill measures. The objective feedback did not provoke supervisee discomfort/distress, or negatively affect the supervisory relationship.

CONCLUSIONS

Extensive MI implementation can increase practitioners' skills in MI, but the question of the best mode of ongoing supervision needs further attention.

PRACTICE IMPLICATIONS

Objective feedback does not seem to negatively affect the supervisee's skill acquisition or the supervisor-supervisee working alliance, but the question of how to most efficiently provide feedback from multifaceted feedback tools remains unanswered.

Measuring motivational interviewing self-efficacy of pre-service students completing a competency-based motivational interviewing course

Background

Improving health outcomes requires health care practitioners to work collaboratively with clients to make healthy lifestyle changes. Motivational interviewing (MI) is an evidence-based approach found to evoke and support behavior change.

Objective

The aim of this study was to examine changes over time in pre-service professional students' confidence in their MI skills after a 15-week interprofessional MI course.

Methods

Students (N = 22) completed a newly developed 24-item Motivational Interviewing Confidence Survey (MICS pre and post participation in the course). Summary statistics, initial scale reliability assessment and t-tests were carried out.

Results

MICS was a reliable measure (Cronbach's a = 0.98) and detected significant changes in students' self-perceived skill set. Using t-tests, significant differences were noted in pre- and post-assessments in students' confidence in their skills; students (p values<0.001) demonstrated significant gains in confidence on 23 of 24 MICS items.

Conclusion

After participating in the course, students' confidence in their MI skills improved significantly. Adding pre-service training in MI may increase future healthcare practitioners' confidence in their MI skills and improve their capacity to engage in individually tailored, client-driven practice.

Fidelity of Motivational Interviewing in School-Based Intervention and Research

Educational researchers and school-based practitioners are increasingly infusing motivational interviewing (MI) into new and existing intervention protocols to provide support to students, parents, teachers, and school administrators. To date, however, the majority of the research in this area has focused on feasibility of implementation rather than fidelity of implementation. In this manuscript, we will present MI fidelity data from 245 audio-recorded conversations with 113 unique caregivers and 20 coaches, who implemented a school-based, positive parenting intervention. The aggregate fidelity scores across coaches, parents, and sessions provide evidence the training and support procedures were effective in assisting school-based personnel to implement MI with reasonable levels of fidelity in practice settings. Further, results suggest that MI fidelity varied between sessions and coaches and that within-coach variation (e.g., session-level variation in the quality of MI delivered) greatly exceeded between-coach variation. Implications for practice and future research are discussed.

Effects of Fidelity-Focused Consultation on Clinicians' Implementation: An Exploratory Multiple Baseline Design

Identification of effective consultation models could inform implementation efforts. This study examined the effects of a fidelity-focused consultation model among community-based clinicians implementing Attachment and Biobehavioral Catch-up. Fidelity data from 1217 sessions from 7 clinicians were examined in a multiple baseline design. In fidelity-focused consultation, clinicians received feedback from consultants' fidelity coding, and also coded their own fidelity. Clinicians' fidelity increased after fidelity-focused consultation began, but did not increase during other training periods. Fidelity was sustained for 30 months after consultation ended. Findings suggest that consultation procedures involving fidelity coding feedback and self-monitoring of fidelity may promote implementation outcomes.

Evaluation of a Level I trauma center provider training in patient-centered alcohol brief interventions using the Behavior Change Counseling Index rated by standardized patients

Background

Traumatic injury requiring hospitalization is common in the USA and frequently related to alcohol consumption. The American College of Surgeons requires that Level I and II verified trauma centers implement universal alcohol screening and brief intervention for injured patients. We examined whether Level I trauma center provider skill in patient-centered alcohol brief interventions improved after training and whether professional role (eg, nursing, social work) and education were associated with these skills.

Methods

We present evaluation data collected as part of training in alcohol brief interventions embedded within a larger clinical trial of a collaborative care intervention targeting posttraumatic stress disorder and related comorbidities. Sixty-five providers from 25 US Level I trauma centers engaged in a 1-day workshop, with 2 hours dedicated to training in patient-centered alcohol brief interventions followed by 6 months of weekly coaching in a collaborative care model. Providers completed standardized patient role-plays prior to and 6 months after the workshop training. The standardized patient actors rated provider quality of alcohol brief interventions immediately after each role-play using the Behavior Change Counseling Index (BECCI), a pragmatic measure designed to assess the quality of behavior change counseling, an adaptation of motivational interviewing suitable for brief healthcare consultations about behavior change.

Results

Seventy-two percent of providers completed both standardized patient role-play assessments. A statistically significant improvement in overall BECCI scores (t(41)=−2.53, p=0.02, Cohen’s d=−0.39) was observed among those providers with available pre–post data. Provider professional role was associated with BECCI scores at pre-training (F(3, 58)=11.25, p<0.01) and post-training (F(3, 41)=8.10, p<0.01).

Discussion

Findings underscore the need for training in patient-centered alcohol brief interventions and suggest that even a modest training helps providers engage in a more patient-centered way during a role-play assessment.

Level of evidence

Level V, therapeutic/care management.

Providing objective feedback in supervision in motivational interviewing: results from a randomized controlled trial

BACKGROUND

The effects of the use of objective feedback in supervision on the supervisory relationship and skill acquisition is unknown.

AIMS

The objective of this study was to evaluate the effects of two different types of objective feedback provided during supervision in motivational interviewing (MI) on: (a) the supervisory relationship, including potential feelings of discomfort/distress, provoked by the supervision sessions, and (b) the supervisees' skill acquisition.

METHOD

Data were obtained from a MI dissemination study conducted in five county councils across five county councils across Sweden. All 98 practitioners recorded sessions with standardized clients and were randomized to either systematic feedback based on only the behavioral component of a feedback protocol, or systematic feedback based on the entire protocol.

RESULTS

The two different ways to provide objective feedback did not negatively affect the supervisory relationship, or provoke discomfort/distress among the supervisees, and the group that received the behavioural component of the feedback protocol performed better on only two of the seven skill measures.

CONCLUSIONS

Objective feedback does not seem to negatively affect either the supervisor-supervisee working alliance or the supervisees' supervision experience. The observed differences in MI skill acquisition were small, and constructive replications are needed to ascertain the mode and complexity of feedback that optimizes practitioners' learning, while minimizing the sense of discomfort and distress.

An intervention to improve provider-patient interaction at methadone maintenance treatment in China

BACKGROUND

This study evaluated an intervention aiming to improve methadone maintenance therapy (MMT) service providers' interaction with their patients in China.

METHODS

Sixty-eight MMT clinics were randomized to either an intervention or a control condition. Providers in the intervention group attended three group training sessions to enhance their communication skills. Trained providers were encouraged to practice the taught communication skills through provider-initiated individual sessions with their patients. A total of 418 service providers completed assessments from baseline to 24-month. Linear mixed-effects regression models were used to compare self-reported short-term and sustained improvement in provider-patient interaction between the intervention and the control conditions.

RESULTS

The intervention group service providers perceived significantly greater short-term and sustained improvement in provider-patient interaction compared to the control group service providers (estimated difference (±SE): 1.20 (0.24) and 1.35 (0.33), respectively; p-values < 0.0001). Providers' baseline job satisfaction was significantly associated with a greater perceived improvement in provider-patient interaction for both periods (reg. coef. (±SE): 0.02 (0.01) and 0.04 (0.01) for short-term and sustained periods, respectively; p-values < 0.01).

CONCLUSION

Study findings suggest that the intervention could be beneficial for improving perceived provider-patient interaction in MMT programs. Service providers' job satisfaction should be addressed in training programs for the improvement of provider-patient interaction.

Variability in motivational interviewing adherence across sessions, providers, sites, and research contexts

Providers' adherence in the delivery of behavioral interventions for substance use disorders is not fixed, but instead can vary across sessions, providers, and intervention sites. This variability can substantially impact the quality of intervention that clients receive. However, there has been limited work to systematically evaluate the extent to which substance use intervention adherence varies from session-to-session, provider-to-provider, and site-to-site. The present study quantifies the extent to which adherence to Motivational Interviewing (MI) for alcohol and drug use varies across sessions, providers, and intervention sites and compares the extent of this variability across three common MI research contexts that evaluate MI efficacy, MI effectiveness, and MI training. Independent raters coded intervention adherence to MI from 1275 sessions delivered by 216 providers at 15 intervention sites. Multilevel models indicated that 57%-94% of the variance in MI adherence was attributable to variability between sessions (i.e., within providers), while smaller proportions of variance were attributable to variability between providers (3%-26%) and between intervention sites (0.1%-28%). MI adherence was typically lowest and most variable within contexts evaluating MI training (i.e., where MI was not protocol-guided and delivered by community treatment providers) and, conversely, adherence was typically highest and least variable in contexts evaluating MI efficacy and effectiveness (i.e., where MI was highly protocolized and delivered by trained therapists). These results suggest that MI adherence in efficacy and effectiveness trials may be substantially different from that obtained in community treatment settings, where adherence is likely to be far more heterogeneous.

Benchmarking Treatment Adherence and Therapist Competence in Individual Cognitive-Behavioral Treatment for Youth Anxiety Disorders

Evidence-based treatments (EBTs) for youth are typically developed and established through studies in research settings designed to ensure treatment integrity, that is, protocol adherence and competence by therapists. An important question for implementation science is how well integrity is maintained when these EBTs are delivered in community settings. The present study investigated whether the integrity achieved by therapists in community settings achieved a benchmark set by therapists in a research setting when they delivered the same EBT-an individual cognitive-behavioral treatment (ICBT) for youth anxiety. Therapists (N = 29; 68.97% White; 13.79% male) provided ICBT to 68 youths (M age = 10.60 years, SD = 2.03; 82.35% White; 52.94% male) diagnosed with a principal anxiety disorder in research or community settings. Training and supervision protocols were the same across settings. Two independent teams of trained coders rated 744 sessions using observational instruments designed to assess ICBT adherence and competence. Both adherence and competence were higher in the research setting. Group differences in competence were consistent across treatment, but differences in adherence were most pronounced when treatment shifted to exposure, widely viewed as the most critical component of ICBT. When using the benchmarks from the research setting, therapists from the community settings fell short for indices of adherence and competence. However, given differences between therapists and clients, as well as the fact that treatment outcomes were similar across settings, our findings raise questions about whether it is appropriate to use treatment integrity benchmarks from research settings for community.

The dissemination of motivational interviewing in Swedish county councils: Results of a randomized controlled trial

Objective

A significant number of Swedish practitioners are offered workshop trainings in motivational interviewing through community-based implementation programs. The objective of this randomized controlled trial was to evaluate to what extent the practitioners acquire and retain skills from additional supervision consisting of feedback based on monitoring of practice.

Materials and methods

A total of 174 practitioners in five county councils across Sweden were randomized to one of the study's two groups: 1) Regular county council workshop training, 2) Regular county council workshop training followed by six sessions of supervision. The participant’s mean age was 43.3 years, and the majority were females (88.1%).

Results

Recruiting participants proved difficult, which may have led to a biased sample of practitioners highly motivated to learn the method. Although slightly different in form and content, all the workshop trainings increased the participants’ skills to the same level. Also, consistent with previous research, the additional supervision group showed larger gains in proficiency compared to the group who received workshop training only at the six-month follow-up. However, analyses showed generally maintained levels of skills for all the participants at the follow-up assessment, and the majority of participants did not attain beginning proficiency levels at either post-workshop or follow-up.

Conclusions

The results of this study address the real-life implications of dissemination of evidence-based practices. The maintained level of elevated skills for all participants is a promising finding. However, the low interest for obtaining additional supervision among the Swedish practitioners is problematic. In addition, neither the workshop trainings nor the additional supervision, although improving skills, were sufficient for most of the participants to reach beginning proficiency levels. This raises questions regarding the most efficient form of training to attain and sustain adequate practice standards, and how to create incentive and interest among practitioners to participate in such training.

A randomized controlled trial to influence client language in substance use disorder treatment

BACKGROUND

Client language is hypothesized to be a mechanism of action in motivational interviewing (MI). Despite the association of change and sustain talk with substance treatment outcomes, it not known whether providers can intentionally influence this language as hypothesized.

OBJECTIVE

This is a randomized controlled trial to investigate whether substance use providers can be trained to influence client language.

METHODS

Treatment providers specializing in substance use disorders (n=190) were randomly assigned to standard training in MI (MI-AU) or training emphasizing an influence of client language (MI-LEAF). Treatment sessions with actual clients were evaluated 3, 6 and 12 months after training by masked raters. Frequencies of client change and sustain talk were the outcome variables.

RESULTS

Sustain talk, but not change talk, was significantly lower in clients whose providers had received the specialized training (b=-0.175, SE=0.087, p=0.046, CI[-0.348 to 0.002], d=-0.325). Mediation analyses supported a causal chain between a) training, b) providers' attempts to minimize sustain talk in treatment sessions via directive reflective listening and c) client sustain talk in the treatment session (κ²=0.0833, bootstrap SE=0.0394, 95% CI [0.0148, 0.1691]).

CONCLUSIONS

With specialized training, providers can reduce the amount of opposition language their clients offer when considering a change in their substance use. Demonstrating that client language is under partial control of the provider supports the feasibility of clinical trials to investigate the impact of shaping client language on treatment outcomes.