Feasibility and acceptability of two incentive-based implementation strategies for mental health therapists implementing cognitive-behavioral therapy: a pilot study to inform a randomized controlled trial

Acceptability and feasibility of policy implementation strategies for taxes earmarked for behavioral health services

Background

This study's aims are to: (1) Compare the acceptability and feasibility of five types of implementation strategies that could be deployed to increase the reach of evidence-based practices (EBPs) with revenue from policies that earmark taxes for behavioral health services, and (2) Illustrate how definitions of implementation strategies and measures of acceptability and feasibility can be used in policy-focused implementation science research.

Methods

Web-based surveys of public agency and community organization professionals involved with earmarked tax policy implementation were completed in 2022-2023 (N = 211, response rate = 24.9%). Respondents rated the acceptability and feasibility of five types of implementation strategies (dissemination, implementation process, integration, capacity-building, and scale-up). Aggregate acceptability and feasibility scores were calculated for each type of strategy (scoring range 4-20). Analyses of variance compared scores across strategies and between organizational actor types.

Findings

For acceptability, capacity-building strategies had the highest rating (M = 16.3, SD = 3.0), significantly higher than each of the four other strategies, p ≤ . 004), and scale-up strategies had the lowest rating (M = 15.6). For feasibility, dissemination strategies had the highest rating (M = 15.3, significantly higher than three of the other strategies, p ≤ .002) and scale-up strategies had the lowest rating (M = 14.4).

Conclusions

Capacity-building and dissemination strategies may be well-received and readily deployed by policy implementers to support EBPs implementation with revenue from taxes earmarked for behavioral health services. Adapting definitions of implementation strategies for policy-focused topics, and applying established measures of acceptability and feasibility to these strategies, demonstrates utility as an approach to advance research on policy-focused implementation strategies.

An evaluation of implementation climate in inpatient maternity care: a cross-sectional survey study

OBJECTIVE

Researchers in obstetrics and gynecology are continuously generating new evidence to inform clinical care delivery. Yet, much of this newly emerging evidence fails to be rapidly and effectively integrated into routine clinical practice. Implementation climate refers to clinicians' perceptions of to what degree organizations support and reward use of an evidence-based practice (EBP) and is an important construct in the science of implementation in healthcare. Little is known about implementation climate for EBPs in maternity care. Thus, we aimed to (a) determine the reliability of the Implementation Climate Scale (ICS) for use in inpatient maternity care, (b) describe implementation climate in inpatient maternity care overall, and (c) compare individual perceptions of implementation climate between physician and nursing clinicians on these units.

STUDY DESIGN

We performed a cross-sectional survey of clinicians in inpatient maternity units across 2 urban, academic hospitals in the northeastern United States in 2020. Clinicians completed the 18-question validated ICS [scored 0-4]. Scale reliability by role was evaluated using Cronbach's α. Subscale and total scores were described overall and compared by physician versus nursing role using independent t-tests, as well as linear regression to control for confounders.

RESULTS

111 clinicians completed the survey (physicians = 65; nursing = 46). Physicians were less likely to identify as female (75.4% vs. 100.0%, p < .001), but were of similar age and years of experience as nursing clinicians. Reliability of the ICS was excellent, with Cronbach's α of 0.91 and 0.86 among physicians and nursing clinicians, respectively. Scores were notably low for implementation climate in maternity care overall and for all subscales. ICS total scores were also higher among physicians as compared to nurses (2.18(±0.56) vs. 1.92(±0.50), p = .02), which remained significant in multivariable modeling (p = .02). Unadjusted subscale scores were higher among physicians in Recognition for EBP (2.68(±0.89) vs. 2.30(±0.86), p = .03) and Selection for EBP (2.24(±0.93) vs. 1.62(±1.04), p = .002). After adjustment for potential confounders, subscale scores for Focus on EBP (p = .04) and Selection for EBP (p = .002) were all higher among physicians.

CONCLUSIONS

This study supports the ICS as a reliable scale for measuring implementation climate in the inpatient maternity care setting. Notably lower implementation climate scores across subscales and roles compared to other settings may underlie the vast evidence to practice gap in obstetrics. In order to successfully implement practices that reduce maternal morbidity, we may need to focus on building educational support and rewarding EBP utilization on labor and delivery units, with an emphasis on nursing clinicians.

Strategies for enacting health policy codesign: a scoping review and direction for research

BACKGROUND

Strategies for supporting evidence-informed health policy are a recognized but understudied area of policy dissemination and implementation science. Codesign describes a set of strategies potentially well suited to address the complexity presented by policy formation and implementation. We examine the health policy literature describing the use of codesign in initiatives intended to combine diverse sources of knowledge and evidence in policymaking.

METHODS

The search included PubMed, MEDLINE, PsychInfo, CINAHL, Web of Science, and Google Scholar in November 2022 and included papers published between 1996 and 2022. Terms included codesign, health, policy, and system terminology. Title and abstracts were reviewed in duplicate and included if efforts informed policy or system-level decision-making. Extracted data followed scoping review guidelines for location, evaluation method, health focus, codesign definition, description, level of health system user input, sectors involved, and reported benefits and challenges.

RESULTS

From 550 titles, 23 citations describing 32 policy codesign studies were included from multiple continents (Australia/New Zealand, 32%; UK/Europe, 32%; South America, 14%; Africa, 9%; USA/Canada 23%). Document type was primarily case study (77%). The area of health focus was widely distributed. Policy type was more commonly little p policy (47%), followed by big p policy (25%), and service innovations that included policy-enabled funding (25%). Models and frameworks originated from formal design (e.g., human-centered or participatory design (44%), political science (38%), or health service research (16%). Reported outcomes included community mobilization (50%), policy feasibility (41%), improved multisector alignment (31%), and introduction of novel ideas and critical thinking (47%). Studies engaging policy users in full decision-making roles self-reported higher levels of community mobilization and community needs than other types of engagement.

DISCUSSION

Policy codesign is theoretically promising and is gaining interest among diverse health sectors for addressing the complexity of policy formation and implementation. The maturity of the science is just emerging. We observed trends in the association of codesign strategies and outcomes that suggests a research agenda in this area could provide practical insights for tailoring policy codesign to respond to local contextual factors including values, needs, and resources.

Acceptability, appropriateness, and feasibility of Rural School Support Strategies for behavioral interventions: a mixed methods evaluation over two years of a hybrid type 3 implementation-effectiveness trial

BACKGROUND

Positive Behavioral Interventions and Supports (PBIS) is a framework for implementing evidence-based interventions for preventing behavioral issues and improving climate in schools. The implementation of school-wide PBIS with fidelity is complex, requiring leadership commitment, teaming, and coordination of systems for tracking behaviors and consequences. Putting these components in place while ensuring alignment with the values and needs of the school community can be difficult for schools with fewer resources, such as rural schools. Implementation supports are needed, including strategies such as technical assistance, but it is unclear whether lower-cost modalities such as virtual support are acceptable, appropriate, and feasible and whether perceptions vary throughout the implementation process.

METHODS

A type 3 hybrid implementation-effectiveness trial is taking place in 40 Idaho schools, testing a bundle of implementation supports selected to meet the needs of schools in rural areas. Supports include technical assistance from an implementation support practitioner (ISP), didactic trainings, virtual learning sessions, and an online resource portal. Surveys and interviews in the first 2 years of implementation (fall 2019 to spring 2021) explored outcomes of acceptability, appropriateness, and feasibility regarding the implementation supports among more than 150 school stakeholders.

RESULTS

Evaluations showed high acceptability and appropriateness of the PBIS concepts and training. The 20 schools receiving additional implementation support rated the technical assistance and support from the project's ISPs as the most acceptable and appropriate resource. Reasons for acceptability were the relationship built with the ISP, the ISP's expertise, and being a "neutral party." Although in-person support from the ISP was preferred, remote support was acceptable and increased feasibility of attendance. Virtual learning sessions were acceptable for learning and collaboration, particularly in the second year of implementation, once ISPs had developed closer relationships with school teams.

CONCLUSIONS

School staff found training, technical assistance, and virtual learning sessions to be acceptable and appropriate. Virtual formats of training and technical assistance decreased in acceptability but increased feasibility of attendance. In-person support was preferred during initial implementation, and virtual support was more acceptable thereafter.

TRIAL REGISTRATION

This trial was prospectively registered on ClinicalTrials.gov ( NCT03736395 ), on November 9, 2018.

Identifying Methods to Select and Tailor Implementation Strategies to Context-Specific Determinants in Child Mental Health Settings: A Scoping Review

This scoping review describes the state of the literature regarding Implementation Strategy Mapping Methods (ISMMs) within the context of child mental health practice delivery. Goals included (a) identify and describe ISMMs addressing determinants of implementing mental health evidence-based interventions (MH-EBIs) for children and (b) describe the scope of the literature (e.g., outcomes, remaining gaps) related to identified ISMMs. Following PRISMA-ScR guidelines, 197 articles were identified. After removing 54 duplicates, 152 titles and abstracts were screened, yielding 36 articles that were screened during the full-text review. The final sample included four studies and two protocol papers (n = 6). A data charting codebook was developed a priori to capture relevant information (e.g., outcomes) and content analysis was utilized to synthesize findings. Six ISMMs were identified: innovation tournament, concept mapping, modified conjoint analysis, COAST-IS, focus group, and intervention mapping. ISMMs were successful in leading to the identification and selection of implementation strategies at participating organizations, and all ISMMs included stakeholders throughout these processes. Findings revealed the novelty of this research area and highlighted numerous areas for future investigation. Implications related to implementation, service, and client outcomes are discussed, including the possible impact of utilizing ISMMs to increase access to MH-EBIs for children receiving services in community settings. Overall, these findings contribute to our understanding of one of the five priority areas within implementation strategy research-enhancing methods used to design and tailor implementation strategies-by providing an overview of methods that may be utilized to facilitate MH-EBI implementation in child mental health care settings. Trial Registration: Not applicable.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00086-3.

Implementation of a Low-Cost, Multi-component, Web-Based Training for Trauma-Focused Cognitive-Behavioral Therapy

Effective, interactive trainings in evidence-based practices remain expensive and largely inaccessible to most practicing clinicians. To address this need, the current study evaluated the impact of a low-cost, multi-component, web-based training for Trauma-Focused Cognitive-Behavioral Therapy (TF-CBT) on clinicians' TF-CBT knowledge, strategy use, adherence and skill. Clinician members of a practice-based research network were recruited via email and randomized to either an immediate training group (N = 89 assigned) or waitlist control group (N = 74 assigned) that was offered access to the same training after six months, with half of each group further randomized to receive or not receive incentives for participation. Clinicians completed assessments at baseline, 6 months, and 12 months covering TF-CBT knowledge, strategy use, and for a subset of clinicians (n = 28), TF-CBT adherence and skill. Although significant differences in overall TF-CBT skillfulness and readiness were found, there were no significant differences between the training and waitlist control group on TF-CBT knowledge and strategy use at six months. However, there was considerable variability in the extent of training completed by clinicians. Subsequent post-hoc analyses indicated a significant, positive association between the extent of training completed by clinicians and clinician TF-CBT knowledge, strategy use, demonstrated adherence and skill across the three TF-CBT components, and overall TF-CBT readiness. We also explored whether incentives predicted training participation and found no differences in training activity participation between clinicians who were offered an incentive and those who were not. Findings highlight the limitations of self-paced web-based trainings. Implications for web-based trainings are discussed.

A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy

Clinician fidelity to cognitive behavioral therapy (CBT) is an important mechanism by which desired clinical outcomes are achieved and is an indicator of care quality. Despite its importance, there are few fidelity measurement methods that are efficient and have demonstrated reliability and validity. Using a randomized trial design, we compared three methods of assessing CBT adherence-a core component of fidelity-to direct observation, the gold standard. Clinicians recruited from 27 community mental health agencies (n = 126; M age = 37.69 years, SD = 12.84; 75.7% female) were randomized 1:1:1 to one of three fidelity conditions: self-report (n = 41), chart-stimulated recall (semistructured interviews with the chart available; n = 42), or behavioral rehearsal (simulated role-plays; n = 43). All participating clinicians completed fidelity assessments for up to three sessions with three different clients that were recruited from clinicians' caseloads (n = 288; M age = 13.39 years SD = 3.89; 41.7% female); sessions were also audio-recorded and coded for comparison to determine the most accurate method. All fidelity measures had parallel scales that yielded an adherence maximum score (i.e., the highest-rated intervention in a session), a mean of techniques observed, and a count total of observed techniques. Results of three-level mixed effects regression models indicated that behavioral rehearsal produced comparable scores to observation for all adherence scores (all ps > .01), indicating no difference between behavioral rehearsal and observation. Self-report and chart-stimulated recall overestimated adherence compared to observation (ps < .01). Overall, findings suggested that behavioral rehearsal indexed CBT adherence comparably to direct observation, the gold-standard, in pediatric populations. Behavioral rehearsal may at times be able to replace the need for resource-intensive direct observation in implementation research and practice.

Economic Precarity, Financial Strain, and Job-Related Stress Among Philadelphia's Public Mental Health Clinicians

OBJECTIVE

Efforts to improve mental health treatment delivery come at a time of rising inequality and cuts or insufficient increases to mental health funding. Public mental health clinicians face increased demands, experience economic stress, and treat underresourced patients disproportionately burdened by trauma. The authors sought to understand clinicians' current economic and psychological conditions and the relationship of these conditions to the delivery of an evidence-based intervention (EBI) designed to treat posttraumatic stress disorder among youths.

METHODS

In July 2020, 49 public mental health clinicians from 16 Philadelphia clinics who were trained in an EBI, trauma-focused cognitive-behavioral therapy (TF-CBT), were surveyed by e-mail. Respondents reported on their economic precarity, financial strain, burnout, secondary traumatic stress (i.e., the stress response associated with caring for people exposed to trauma), and TF-CBT use. Associations between clinicians' job-related stressors and their use of TF-CBT were examined with mixed models. Content coding was used to organize clinicians' open-ended responses to questions regarding financial strain related to the COVID-19 pandemic.

RESULTS

Clinicians' economic precarity, financial strain, and job-related stress were high; 37% of clinicians were independent contractors, 44% of whom wanted a salaried position. Of 37 clinicians with education debt, 38% reported owing ≥$100,000. In the past year, 29% of clinicians reported lack of personal mental health care because of cost, and 22% met the cutoff for experiencing secondary traumatic stress symptoms. Education debt was negatively associated with use of TF-CBT (p<0.001).

CONCLUSIONS

The stress of providing care in underresourced clinical settings may interfere with efforts to integrate scientific evidence into mental health care.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis

Background

Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers.

Methods

Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery).

Results

On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships.

Conclusions

Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

Implementation of the Wolverine Mental Health Program, Part 3: Sustainment Phase

Sustaining the implementation of an evidence-based practice (EBP) is the ultimate goal of often years of significant personnel and financial investment. Some conceptualize sustainment as a distinct phase following an active implementation period where the contextual factors, processes, and supports are bolstered to ensure continued EBP delivery. This study provides an overview of the sustainment strategies deployed to embed cognitive-behavioral therapy (CBT) in a Midwestern residential treatment facility serving youth with complex mental health needs. Seven key strategies and their outcomes are described: use of CBT teams, new hire orientation plans, monthly campaigns, change in job descriptions and performance evaluations, development of a behavioral reinforcement system for youth, and a pathway to CBT certification. This study provides a window into how one might sustain an EBP by addressing barriers unique to this phase of work.

Assessing Causal Pathways and Targets of Implementation Variability for EBP use (Project ACTIVE): a study protocol

Background

Advancing causal implementation theory is critical for designing tailored implementation strategies that target specific mechanisms associated with evidence-based practice (EBP) use. This study will test the generalizability of a conceptual model that integrates organizational constructs and behavioral theory to predict clinician use of cognitive-behavioral therapy (CBT) techniques in community mental health centers. CBT is a leading psychosocial EBP for psychiatric disorders that remains underused despite substantial efforts to increase its implementation.

Methods

We will leverage ongoing CBT implementation efforts in two large public health systems (Philadelphia and Texas) to recruit 300 mental health clinicians and 600 of their clients across 40 organizations. Our primary implementation outcomes of interest are clinician intentions to use CBT and direct observation of clinician use of CBT. As CBT comprises discrete components that vary in complexity and acceptability, we will measure clinician use of six discrete components of CBT. After finishing their CBT training, participating clinicians will complete measures of organizational and behavior change constructs delineated in the model. Clinicians also will be observed twice via audio recording delivering CBT with a client. Within 48 h of each observation, theorized moderators of the intention-behavior gap will be collected via survey. A subset of clinicians who report high intentions to use CBT but demonstrate low use will be purposively recruited to complete semi-structured interviews assessing reasons for the intention-behavior gap. Multilevel path analysis will test the extent to which intentions and determinants of intention predict the use of each discrete CBT component. We also will test the extent to which theorized determinants of intention that include psychological, organizational, and contextual factors explain variation in intention and moderate the association between intentions and CBT use.

Discussion

Project ACTIVE will advance implementation theory, currently in its infancy, by testing the generalizability of a promising causal model of implementation. These results will inform the development of implementation strategies targeting modifiable factors that explain substantial variance in intention and implementation that can be applied broadly across EBPs.

Implementation of the Kidney Team at Home Intervention: Evaluating Generalizability, Implementation Process, and Effects

Research has shown that a home-based educational intervention for patients with chronic kidney disease results in better knowledge and communication, and more living donor kidney transplantations (LDKT). Implementation research in the field of renal care is almost non-existent. The aims of this study were (1) to demonstrate generalizability, (2) evaluate the implementation process, and (3) to assess the relationship of intervention effects on LDKT-activity. Eight hospitals participated in the project. Patients eligible for all kidney replacement therapies (KRT) were invited to participate. Effect outcomes were KRT-knowledge and KRT-communication, and treatment choice. Feasibility, fidelity and intervention costs were assessed as part of the process evaluation. 332 patients completed the intervention. There was a significant increase in KRT-knowledge and KRT-communication among participants. 129 out of 332 patients (39%) had LDKT-activity, which was in line with the results of the clinical trials. Protocol adherence, knowledge and age were correlated with LDKT-activity. This unique implementation study shows that the results in practice are comparable to the previous trials, and show that the intervention can be implemented, while maintaining quality. Results from the project resulted in the uptake of the intervention in standard care. We urge other countries to investigate the uptake of the intervention.

The effects of the type of delivery of cognitive-behavioral therapy for healthcare workers: A systematic review

OBJECTIVE

The aim of this literature review is to assess the extent of knowledge of the effectiveness of cognitive-behavioral therapy (CBT) for healthcare workers by type of delivery.

METHODS

The study examined relevant literature published between 2009 and 2020 using searches of four scientific databases: PubMed, CINAHL, Embase, and PsycINFO.

RESULTS

For work stress, work-related outcomes, and mental and musculoskeletal problems, the effectiveness of face-to-face (FTF) interventions were confirmed, whereas, for stress and insomnia, the effectiveness of non-face-to-face (NFTF) interventions was confirmed. FTF interventions were proven to make a significant difference to the outcome of the goal; however, their effects were not clearly distinguished from those of NFTF interventions.

CONCLUSION

Results of this review highlight the lack of research aimed at establishing the effectiveness of NFTF interventions. In addition, further research is needed on the long-term effectiveness of the intervention process and perception of participants and clinicians on group delivery interventions.

Stakeholder selected strategies for obesity prevention in childcare: results from a small-scale cluster randomized hybrid type III trial

BACKGROUND

Together, We Inspire Smart Eating (WISE) is an intervention for the early care and education setting to support children's exposure to and intake of fruits and vegetables. WISE emphasizes 4 evidence-based practices (EBPs): (1) use of a mascot; (2) educators' role modeling; (3) positive feeding practices; and (4) hands-on exposures. The current study reports on a small-scale implementation trial aimed at improving the use of WISE EBPs by teachers.

METHODS

A Hybrid Type III Cluster Randomized Design compared a Basic and Enhanced implementation strategy. The Basic Strategy included training and reminders only; the Enhanced strategy was a multi-faceted package of stakeholder-selected strategies including a leadership commitment, an implementation blueprint, a local champion, an environmental reminder of the EBPs, facilitation, and tailored educational resources and incentives. All study sites were Head Starts. Sites were randomized using a balancing technique that considered site characteristics; 4 sites (20 classrooms, 39 educators, 305 children) received Enhanced support; 5 sites (18 classrooms, 36 educators, 316 children) received Basic support. RE-AIM guided the evaluation, and implementation fidelity was the primary outcome. Strategies were assessed using examination of data distributions and unadjusted comparisons (t tests) as well as general linear and mixed effects models controlling for covariates.

RESULTS

For the primary outcome of fidelity, the Enhanced group had significantly higher means for 3 of 4 EBPs. Multivariate models explained a significant portion of variance for both mascot use and hands-on exposure with a significant positive effect observed for treatment condition. The Enhanced group also had higher rates of Appropriateness and Organizational Readiness for Implementing Change (as indicators of implementation and adoption, respectively). There was no significant difference between groups for indicators of Reach, Effectiveness or Maintenance. Formative interviews indicated key targets for iteration and potential mechanisms. Key events were catalogued to provide context for interpretation (e.g., 61% of classrooms with turnover).

CONCLUSIONS

Findings were mixed but suggested promise for the Enhanced strategy, especially considering key events of the study. Implementation fidelity improvements occurred mainly in the last 3 months of the school year; additional time may be needed to translate to improvements in child outcomes.

TRIAL REGISTRATION

NCT03075085 Registered 20 February 2017.

Analytic Training for Junior Investigators in Minority Aging Research: The Michigan Model

Since 1997, the Resource Centers for Minority Aging Research Program at the National Institute on Aging has been the model for training social and behavioral scientists in minority aging and health disparities research. The latest cycle of these Centers implemented a new structure for the analytic training of junior investigators and for advancing methodologic work relevant to improving the rigor of minority aging research. In this article, we describe the conceptual framework, logistical approaches, challenges, and lessons learned from our experience training junior investigators in methodology through the Michigan Center for Urban African American Aging Research over the past 20 years, with the goal of informing future analytic training efforts for the next generation of scholars focused on minority aging issues.

Acceptability and Feasibility of Perioperative Music Listening: A Rapid Qualitative Inquiry Approach

Perioperative music decreases pain, anxiety, and analgesia requirements while increasing patient satisfaction. We investigated the acceptability of perioperative music at a women's and children's hospital in Singapore, seeking to uncover barriers and facilitators to implementation in routine practice over a 12-week period. We used an undergraduate-led Rapid Qualitative Inquiry approach that includes data collection from multiple sources, iterative analysis, and additional data collection when necessary. Participants consisted of anesthesiologists and nurses working in the preoperative area, operating room, and postoperative recovery areas. In Stage 1, nurses and anesthesiologists answered a survey assessing attitudes and knowledge about perioperative music and attended a presentation introducing the intervention. In Stage 2, the results of the Stage 1 survey were disclosed, and nurses and anesthesiologists completed a second survey (Stage 1 survey questions with an additional query about implementation). Twenty-nine nurses were interviewed with semi-structured questions on barriers and facilitators to implementation. In Stage 3, nurses retook the Stage 2 survey after one month of implementation. The implementation of perioperative music was both acceptable and feasible in Singapore. The nurses' and anesthesiologists' acceptability increased over time with mostly positive qualitative feedback. Implementation barriers, including patient interest, timing of patient turnover, and added staff workload, appeared minor and could potentially be overcome by utilizing the intervention in the preoperative instead of postoperative area, deploying music in operating rooms with slower turnover, and evenly distributing staff workload. Local implementation stakeholders may make additional recommendations for scaling-up perioperative music interventions to fit other workflows while improving the patient experience.

Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment

BACKGROUND

Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling.

METHODS

A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles.

RESULTS

On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics.

CONCLUSIONS

The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

Using behavioral insights to design implementation strategies in public mental health settings: a qualitative study of clinical decision-making

BACKGROUND

Trauma-focused cognitive behavioral therapy (TF-CBT) is an evidence-based intervention for youth with posttraumatic stress disorder. An important component of TF-CBT is the trauma narrative (TN), a phase in the intervention in which youth are guided to process the memories, thoughts, and feelings associated with their traumatic experience(s). Previous work has shown that TF-CBT clinicians complete TNs with only half of their clients, yet little is known about what determines TF-CBT clinicians' use of TNs. The behavioral insights literature-an interdisciplinary field studying judgment and decision-making-offers theoretical and empirical tools to conceptualize what drives complex human behaviors and decisions. Drawing from the behavioral insights literature, the present study seeks to understand what determines clinician use of TNs and to generate strategies that target these determinants.

METHODS

Through semi-structured qualitative interviews, we sought the perspectives of trained TF-CBT clinicians working in public mental health settings across the city of Philadelphia (N = 17) to understand their decisions to use TNs with clients. We analyzed the qualitative data using a coding approach informed by the behavioral insights literature. We used an iterative process of structured hypothesis generation, aided by a behavioral insights guide, and rapid validation informed by behavioral insights to uncover the determinants of TN use. We then generated implementation strategies that targeted these determinants using the "Easy Attractive Social Timely" framework, a behavioral insights design approach.

RESULTS

We generated and validated three broad themes about what determines clinician implementation of TNs: decision complexity, clinician affective experience, and agency norms. We hypothesized behavioral insights that underlie these implementation determinants and designed a list of nine corresponding behavioral insights strategies that may facilitate TN implementation.

CONCLUSIONS

Our study investigated why an effective component of an evidence-based intervention is difficult to implement. We leveraged robust scientific theories and empirical regularities from the behavioral insights literature to understand clinician perspectives on TN implementation. These factors were theoretically linked to implementation strategies. Our work revealed the potential for using behavioral insights in the diagnosis of evidence-based intervention determinants and the design of implementation strategies.

Social norms interventions to change clinical behaviour in health workers: a systematic review and meta-analysis

Background

A social norms intervention seeks to change the clinical behaviour of a target health worker by exposing them to the values, beliefs, attitudes or behaviours of a reference group or person. These low-cost interventions can be used to encourage health workers to follow recommended professional practice.

Objective

To summarise evidence on whether or not social norms interventions are effective in encouraging health worker behaviour change, and to identify the most effective social norms interventions.

Design

A systematic review and meta-analysis of randomised controlled trials.

Data sources

The following databases were searched on 24 July 2018: Ovid MEDLINE (1946 to week 2 July 2018), EMBASE (1974 to 3 July 2018), Cumulative Index to Nursing and Allied Health Literature (1937 to July 2018), British Nursing Index (2008 to July 2018), ISI Web of Science (1900 to present), PsycINFO (1806 to week 3 July 2018) and Cochrane trials (up to July 2018).

Participants

Health workers took part in the study.

Interventions

Behaviour change interventions based on social norms.

Outcome measures

Health worker clinical behaviour, for example prescribing (primary outcome), and patient health outcomes, for example blood test results (secondary), converted into a standardised mean difference.

Methods

Titles and abstracts were reviewed against the inclusion criteria to exclude any that were clearly ineligible. Two reviewers independently screened the remaining full texts to identify relevant papers. Two reviewers extracted data independently, coded for behaviour change techniques and assessed quality using the Cochrane risk-of-bias tool. We performed a meta-analysis and presented forest plots, stratified by behaviour change technique. Sources of variation were explored using metaregression and network meta-analysis.

Results

A total of 4428 abstracts were screened, 477 full texts were screened and findings were based on 106 studies. Most studies were in primary care or hospitals, targeting prescribing, ordering of tests and communication with patients. The interventions included social comparison (in which information is given on how peers behave) and credible source (which refers to communication from a well-respected person in support of the behaviour). Combined data suggested that interventions that included social norms components were associated with an improvement in health worker behaviour of 0.08 standardised mean differences (95% confidence interval 0.07 to 0.10 standardised mean differences) (n = 100 comparisons), and an improvement in patient outcomes of 0.17 standardised mean differences (95% confidence interval 0.14 to 0.20) (n = 14), on average. Heterogeneity was high, with an overall I 2 of 85.4% (primary) and 91.5% (secondary). Network meta-analysis suggested that three types of social norms intervention were most effective, on average, compared with control: credible source (0.30 standardised mean differences, 95% confidence interval 0.13 to 0.47); social comparison combined with social reward (0.39 standardised mean differences, 95% confidence interval 0.15 to 0.64); and social comparison combined with prompts and cues (0.33 standardised mean differences, 95% confidence interval 0.22 to 0.44).

Limitations

The large number of studies prevented us from requesting additional information from authors. The trials varied in design, context and setting, and we combined different types of outcome to provide an overall summary of evidence, resulting in a very heterogeneous review.

Conclusions

Social norms interventions are an effective method of changing clinical behaviour in a variety of health service contexts. Although the overall result was modest and very variable, there is the potential for social norms interventions to be scaled up to target the behaviour of a large population of health workers and resulting patient outcomes.

Future work

Development of optimised credible source and social comparison behaviour change interventions, including qualitative research on acceptability and feasibility.

Study registration

This study is registered as PROSPERO CRD42016045718.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 41. See the NIHR Journals Library website for further project information.

Determinants of Fall Prevention Guideline Implementation in the Home- and Community-Based Service Setting

Background and Objectives

Home- and community-based service (HCBS) recipients often possess multiple fall risk factors, suggesting that the implementation of evidence-based fall prevention guidelines may be appropriate for the HCBS setting. The purpose of this exploratory study was to examine the determinants of fall prevention guideline implementation and the potential strategies that can support implementation in HCBS organizations.

Research Design and Methods

Semistructured interview and focus group data were collected from 26 HCBS professionals representing the home-delivered meals, personal care, and wellness programs. Qualitative codes were mapped to the Consolidated Framework for Implementation Research by means of directed content analysis. The Consolidated Criteria for Reporting Qualitative research checklist was used to report the findings of this study.

Results

We identified 7 major determinants of guideline implementation: recipient needs and resources, cosmopolitanism, external policy and incentives, networks and communication, compatibility, available resources, and knowledge/beliefs. Strategies to support guideline implementation included the involvement of recipient and caregiver feedback, building fall prevention networks, and conducting educational meetings for HCBS staff.

Discussion and Implications

Falls and fall-related injuries will continue to plague the older adult community unless innovative approaches to fall prevention are developed and adopted. The implementation of fall prevention guidelines in the HCBS setting can help mitigate fall risk among a highly vulnerable older adult population and may be facilitated through the application of multifaceted implementation strategies.

Effectiveness of training methods for delivery of evidence-based psychotherapies: a systematic review

BACKGROUND

Extensive efforts have been made to train mental health providers in evidence-based psychotherapies (EBPs); there is increasing attention focused on the methods through which providers are trained to deliver EBPs. Evaluating EBP training methods is an important step in determining which methods are most effective in increasing provider skill and improving client outcomes.

METHODS

We searched MEDLINE (Ovid) and PsycINFO for randomized controlled trials published from 1990 through June 2019 that evaluated EBP training methods to determine the effectiveness of EBP training modalities on implementation (provider and cost) and client outcomes. Eligible studies (N = 28) were evaluated for risk of bias, and the overall strength of evidence was assessed for each outcome. Data was extracted by a single investigator and confirmed by a second; risk of bias and strength of evidence were independently rated by two investigators and determined by consensus.

RESULTS

Overall, EBP training improved short-term provider satisfaction, EBP knowledge, and adherence compared to no training or self-study of training materials (low to moderate strength of evidence). Training in an EBP did not increase treatment adoption compared to no training or self-study. No specific active EBP training modality was found to consistently increase provider EBP knowledge, skill acquisition/adherence, competence, adoption, or satisfaction compared to another active training modality. Findings were mixed regarding the additive benefit of post-training consultation on these outcomes. No studies evaluated changes in provider outcomes with regards to training costs and few studies reported on client outcomes.

LIMITATIONS

The majority of included studies had a moderate risk of bias and strength of evidence for the outcomes of interest was generally low or insufficient. Few studies reported effect sizes. The ability to identify the most effective EBP training methods was limited by low strength of evidence for the outcomes of interest and substantial heterogeneity among studies.

CONCLUSIONS

EBP training may have increased short-term provider satisfaction, EBP knowledge, and adherence though not adoption. Evidence was insufficient on training costs and client outcomes. Future research is needed on EBP training methods, implementation, sustainability, client outcomes, and costs to ensure efforts to train providers in EBPs are effective, efficient, and durable.

TRIAL REGISTRATION

The protocol for this review is registered in PROSPERO (CRD42018093381).

The implementation and adherence to evidence-based protocols for psychotherapy for depression: the perspective of therapists in Dutch specialized mental healthcare

BACKGROUND

Although psychotherapy is an effective treatment for depression, a large number of patients still do not receive care according to the protocols that are used in clinical trials. Instead, patients often receive a modified version of the original intervention. It is not clear how and when treatment protocols are used or modified in the Dutch specialized mental health care and whether these changes lead to suboptimal adherence to treatment protocols.

METHODS

In the context of an ongoing multicenter trial that investigates whether twice-weekly sessions of protocolized interpersonal psychotherapy (IPT) and cognitive behavioral therapy (CBT) for depression lead to better treatment outcomes compared to once-weekly sessions, two focus groups using semi-structured interviews were organized. Aims were to increase insight in the adherence to and modifications of CBT and IPT protocols in the Dutch specialized mental health care for depression. Participants were fifteen therapists from seven mental health locations part of five mental health organizations. Verbatim transcripts were coded and analyzed using qualitative software.

RESULTS

Three themes emerged: modification as the common practice, professional and patient factors influencing the adherence to protocols and organizational boundaries and flexibility. Treatment modification appeared to happen on a frequent basis, even in the context of a trial. Definitions of treatment modifications were multiple and varied from using intuition to flexible use of the same protocol. Therapist training and supervision, the years of work experience and individual characteristics of the therapist and the patient were mentioned to influence the adherence to protocols. Modifications of the therapists depended very much on the culture within the mental health locations, who differed in terms of the flexibility offered to therapists to choose and modify treatment protocols.

CONCLUSIONS

Not all treatment modifications were in line with existing evidence or guidelines. Regular supervision, team meetings and a shared vision were identified as crucial factors to increase adherence to treatment protocols, whereas additional organizational factors, among which a change of mindset, may facilitate adequate implementation.