Evidence-based practice implementation strategies: results of a qualitative study

Development of the Clinical High Risk for Psychosis Services Fidelity Scale (CHRPS-FS) for Team-Based Care

OBJECTIVE

This study aimed to develop and pilot the Clinical High Risk for Psychosis Services Fidelity Scale (CHRPS-FS).

METHODS

A literature review was conducted to identify evidence-based treatments for individuals at clinical high risk for psychosis (CHRP). These findings were compared with the First-Episode Psychosis Services Fidelity Scale (FEPS-FS). Common items were retained, and others were added, modified, or deleted. Next, the Delphi process was conducted with 17 clinical and academic experts in CHRP care to determine consensus on the importance and validity of each item. Concurrently, the preliminary tool was piloted in eight coordinated specialty care (CSC) clinics serving individuals with CHRP.

RESULTS

The literature review identified two components of CHRP care that were not detailed in the FEPS-FS and were added to the CHRPS-FS; furthermore, one FEPS-FS item was modified and six were removed. In the Delphi process, clinical and academic experts achieved a consensus of >80% in two rounds, with some changes in item wording and the addition of one item (stepped care approach). A CHRPS-FS assessment was successfully piloted in eight CSC clinics. The mean CHRPS-FS rating score was 3.96 (range 3.75-4.23), and the median proportion of items rated at good to high fidelity was 72% (range 66%-78%).

CONCLUSIONS

The CHRP-FS is feasible to implement, has face validity based on expert consensus, can be completed in conjunction with a FEPS-FS assessment or alone, and captures variability across programs. The CHRPS-FS measures service delivery and is suitable for clinical trials, learning health care systems, and quality improvement efforts.

Implementing absolute cardiovascular disease risk assessment into pathology collection services

BACKGROUND

Pathology services represent an ideal setting to integrate absolute cardiovascular disease (CVD) risk estimation when patients attend for routine cholesterol testing. This study aimed to explore the process of implementing CVD risk estimation into point-of-care service delivery by pathology staff to inform future implementation and sustainability.

METHODS

A new service for CVD risk estimation via a self-directed screening station was implemented into 14 pathology service sites across Tasmania, Australia. Before implementation, observations at pathology services (n = 26) and semi-structured interviews were undertaken with 26 pathology staff (88% female, 77% aged 41-60 years) to identify factors that could impact implementation of the service. The process of implementation was then evaluated using participant observations and clinical trial recruitment data. Transcripts and field notes were analysed thematically according to the Medical Research Council Framework and used to develop a programme logic model to understand how the service could be adapted to be successfully integrated into routine workflow at pathology services.

RESULTS

Eight key themes were identified during the pre-implementation phase as important factors that could impact upon integration of CVD risk estimation into pathology services. Themes related to factors within the organisation, including available resources, logistics and workflow, as well as having sufficient time to complete the intervention. Additional factors related to the individual motivations of staff, collaborative leadership and patient characteristics. Success of implementation varied among sites, requiring the trialling of different strategies to support uptake of the service and patient recruitment.

CONCLUSIONS

Implementing CVD risk estimation into point-of-care pathology services required an understanding of the core implementation components specific to each context, and for implementation strategies to be targeted to the individual and organisational contexts. The generated programme logic model may be useful in guiding future implementation endeavours within these services and aiding the selection of apt implementation strategies.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT04896021, registered 19/05/2021, https://clinicaltrials.gov/study/NCT04896021.

Supporting Emotionally Exhausted Community Mental Health Therapists in Appropriately Adapting EBPs for Children and Adolescents

Evidence-based practices (EBPs) are often adapted during community implementation to improve EBP fit for clients and the service context. Augmenting EBPs with additional dosing and content may improve fit. However, reducing EBP content can reduce EBP effectiveness. Using multilevel regression models, this study examined whether supportive program climate and program-furnished EBP-specific implementation strategies (e.g., materials, ongoing training, in-house experts) are associated with augmenting and reducing adaptations, and whether therapist emotional exhaustion moderated these associations. Data were collected from surveys completed by 439 therapists from 102 programs 9 years after a system-driven EBP implementation initiative. Supportive program climate was associated with more augmenting adaptations. Emotional exhaustion was a significant moderator. When organizations used more EBP-specific implementation strategies, more emotionally exhausted therapists reduced EBPs less and less emotionally exhausted therapists augmented EBPs more. Findings provide guidance on how organizations can support appropriate EBP adaptations in spite of therapist emotional exhaustion.

Development and validation of a fidelity instrument for Cognitive Adaptation Training

Purpose

Cognitive Adaptation Training (CAT) is a psychosocial intervention with demonstrated effectiveness. However, no validated fidelity instrument is available. In this study, a CAT Fidelity Scale was developed and its psychometric properties, including interrater reliability and internal consistency, were evaluated.

Methods

The fidelity scale was developed in a multidisciplinary collaboration between international research groups using the Delphi method. Four Delphi rounds were organized to reach consensus for the items included in the scale. To examine the psychometric properties of the scale, data from a large cluster randomized controlled trial evaluating the implementation of CAT in clinical practice was used. Fidelity assessors conducted 73 fidelity reviews at four mental health institutions in the Netherlands.

Results

After three Delphi rounds, consensus was reached on a 44-item CAT Fidelity Scale. After administration of the scale, 24 items were removed in round four resulting in a 20-item fidelity scale. Psychometric properties of the 20-item CAT Fidelity Scale shows a fair interrater reliability and an excellent internal consistency.

Conclusions

The CAT fidelity scale in its current form is useful for both research purposes as well as for individual health professionals to monitor their own adherence to the protocol. Future research needs to focus on improvement of items and formulating qualitative anchor point to the items to increase generalizability and psychometric properties of the scale. The described suggestions for improvement provide a good starting point for further development.

Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77-.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242.

Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology

Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.

An evidence-based co-occurring disorder intervention in VA homeless programs: outcomes from a hybrid III trial

BACKGROUND

Evidence-based treatment for co-occurring disorders is needed within programs that serve homeless Veterans to assist with increasing engagement in care and to prevent future housing loss. A specialized co-occurring disorders treatment engagement intervention called Maintaining Independence and Sobriety Through Systems Integration, Outreach and Networking - Veterans Edition (MISSION-Vet) was implemented within the Housing and Urban Development - Veterans Affairs Supportive Housing (HUD-VASH) Programs with and without an implementation strategy called Getting To Outcomes (GTO). While implementation was modest for the GTO group, no one adopted MISSION in the non-GTO group. This paper reports Veteran level outcome data on treatment engagement and select behavioral health outcomes for Veterans exposed to the MISSION-Vet model compared to Veterans without access to MISSION-Vet.

METHODS

This hybrid Type III trial compared 81 Veterans in the GTO group to a similar group of 87 Veterans with mental health and substance use disorders from the caseload of staff in the non-GTO group. Comparisons were made on treatment engagement, negative housing exits, drug and alcohol abuse, inpatient hospitalizations, emergency department visits and income level over time, using mixed-effect or Cox regression models.

RESULTS

Treatment engagement, as measured by the overall number of case manager contacts with Veterans and others (e.g. family members, health providers), was significantly higher among Veterans in the GTO group (B = 2.30, p = .04). Supplemental exploratory analyses between Veterans who received "higher" and "lower" intensity MISSION-Vet services in the GTO group failed to show differences in alcohol and drug use, inpatient hospitalization and emergency department use.

CONCLUSIONS

Despite modest MISSION-Vet fidelity among staff treating Veterans in the GTO group, differences were found in treatment engagement. However, this study failed to show differences in alcohol use, drug use, mental health hospitalizations and negative housing exits over time among those Veterans receiving higher intensity MISSION-Vet services versus low intensity services. This project suggests that MISSION-Vet could be used in HUD-VASH to increase engagement among Veterans struggling with homelessness, a group often disconnected from care.

TRIAL REGISTRATION

Clinicaltrials.gov, registration number: NCT01430741 , registered July 26, 2011.

Training in Evidence-Based Practices Increases Likelihood to Integrate Different HIV Prevention Services with Substance-Using Clients

Providers of social and public health services ("providers") often use HIV prevention strategies with substance-using clients to decrease HIV transmission and infection. This article examines factors that facilitate providers' use of select HIV-prevention strategies. Sample comprises 379 providers from 36 agencies in New York City.

OUTCOMES

sexual risk assessments; risk reduction counseling; condom demonstration; and referrals to HIV testing.

PREDICTORS

training; job satisfaction; staff collaboration. The authors used multilevel logistic regression and linear multilevel models. HIV prevention training was associated with increased performance of each outcome. The odds of conducting several outcomes were higher for providers trained in evidence-based interventions. Staff collaboration and job satisfaction were associated with provision of multiple outcomes. This study shows training and collaboration/satisfaction as significantly influencing providers to use prevention strategies. Providers ought to be trained in multiple modalities, and agencies ought to prioritize collaborative environments that promote job satisfaction.

Themes arising during implementation consultation with teams applying family-based treatment: a qualitative study

BACKGROUND

This study describes themes arising during implementation consultation with teams providing Family-Based Treatment (FBT) to adolescents with eating disorders.

METHODS

Participants were implementation teams (one lead therapist, one medical practitioner and one administrator) at four sites. These teams agreed to support the implementation of FBT, and participated in monthly consultation calls which were audio-recorded, transcribed verbatim and coded for themes. Twenty percent of the transcripts were double-coded to ensure consistency. Fundamental qualitative description guided the sampling and data collection.

RESULTS

Twenty-five (average per site = 6) transcripts were coded using thematic content analysis. Six major themes emerged: 1) system barriers and facilitators 2) the role of the medical practitioner, 3) research implementation, 4) appropriate cases, 5) communication, and 6) program impact.

CONCLUSIONS

Implementation themes aligned with previous research examining the adoption of FBT, and provide additional insight for clinical programs seeking to implement FBT, emphasizing the importance of role clarity, and team communication.

Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial

BACKGROUND

Improving the extent to which evidence-based practices (EBPs)-treatments that have been empirically shown to be efficacious or effective-are integrated within routine practice is a well-documented challenge across numerous areas of health. In 2014, the National Institute on Drug Abuse funded a type 2 effectiveness-implementation hybrid trial titled the substance abuse treatment to HIV Care (SAT2HIV) Project. Aim 1 of the SAT2HIV Project tests the effectiveness of a motivational interviewing-based brief intervention (MIBI) for substance use as an adjunct to usual care within AIDS service organizations (ASOs) as part of its MIBI Experiment. Aim 2 of the SAT2HIV Project tests the effectiveness of implementation and sustainment facilitation (ISF) as an adjunct to the Addiction Technology Transfer Center (ATTC) model for training staff in motivational interviewing as part of its ISF Experiment. The current paper describes the study protocol for the ISF Experiment.

METHODS

Using a cluster randomized design, case management and leadership staff from 39 ASOs across the United States were randomized to receive either the ATTC strategy (control condition) or the ATTC + ISF strategy (experimental condition). The ATTC strategy is staff-focused and includes 10 discrete strategies (e.g., provide centralized technical assistance, conduct educational meetings, provide ongoing consultation). The ISF strategy is organization-focused and includes seven discrete strategies (e.g., use an implementation advisor, organize implementation team meetings, conduct cyclical small tests of change). Building upon the exploration-preparation-implementation-sustainment (EPIS) framework, the effectiveness of the ISF strategy is examined via three staff-level measures: (1) time-to-proficiency (i.e., preparation phase outcome), (2) implementation effectiveness (i.e., implementation phase outcome), and (3) level of sustainment (i.e., sustainment phase outcome).

DISCUSSION

Although not without limitations, the ISF experiment has several strengths: a highly rigorous design (randomized, hypothesis-driven), high-need setting (ASOs), large sample size (39 ASOs), large geographic representation (23 states and the District of Columbia), and testing along multiple phases of the EPIS continuum (preparation, implementation, and sustainment). Thus, study findings will significantly improve generalizable knowledge regarding the best preparation, implementation, and sustainment strategies for advancing EBPs along the EPIS continuum. Moreover, increasing ASO's capacity to address substance use may improve the HIV Care Continuum. Trial registration ClinicalTrials.gov: NCT03120598.

Factors That Influence Linkages to HIV Continuum of Care Services: Implications for Multi-Level Interventions

Worldwide, the human immunodeficiency virus (HIV) continuum of care involves health promotion providers (e.g., social workers and health educators) linking patients to medical personnel who provide HIV testing, primary care, and antiretroviral treatments. Regrettably, these life-saving linkages are not always made consistently and many patients are not retained in care. To design, test and implement effective interventions, we need to first identify key factors that may improve linkage-making. To help close this gap, we used in-depth interviews with 20 providers selected from a sample of 250 participants in a mixed-method longitudinal study conducted in New York City (2012–2017) in order to examine the implementation of HIV services for at-risk populations. Following a sociomedical framework, we identified provider-, interpersonal- and environmental-level factors that influence how providers engage patients in the care continuum by linking them to HIV testing, HIV care, and other support services. These factors occurred in four domains of reference: Providers’ Professional Knowledge Base; Providers’ Interprofessional Collaboration; Providers’ Work-Related Changes; and Best Practices in a Competitive Environment. Of particular importance, our findings show that a competitive environment and a fear of losing patients to other agencies may inhibit providers from engaging in linkage-making. Our results suggest relationships between factors within and across all four domains; we recommend interventions to modify factors in all domains for maximum effect toward improving care continuum linkage-making. Our findings may be applicable in different areas of the globe with high HIV prevalence.

Agency responses to a system-driven implementation of multiple evidence-based practices in children's mental health services

BACKGROUND

Large mental health systems are increasingly using fiscal policies to encourage the implementation of multiple evidence-based practices (EBPs). Although many implementation strategies have been identified, little is known about the types and impacts of strategies that are used by organizations within implementation as usual. This study examined organizational-level responses to a fiscally-driven, rapid, and large scale EBP implementation in children's mental health within the Los Angeles County Department of Mental Health.

METHODS

Qualitative methods using the principles of grounded theory were used to characterize the responses of 83 community-based agencies to the implementation effort using documentation from site visits conducted 2 years post reform.

RESULTS

Findings indicated that agencies perceived the rapid system-driven implementation to have both positive and negative organizational impacts. Identified challenges were primarily related to system implementation requirements rather than to characteristics of specific EBPs. Agencies employed a variety of implementation strategies in response to the system-driven implementation, with agency size associated with implementation strategies used. Moderate- and large-sized agencies were more likely than small agencies to have employed systematic strategies at multiple levels (i.e., organization, therapist, client) to support implementation.

CONCLUSIONS

These findings are among the first to characterize organizational variability in response to system-driven implementation and suggest ways that implementation interventions might be tailored by organizational characteristics.

Using Getting To Outcomes to facilitate the use of an evidence-based practice in VA homeless programs: a cluster-randomized trial of an implementation support strategy

BACKGROUND

Incorporating evidence-based integrated treatment for dual disorders into typical care settings has been challenging, especially among those serving Veterans who are homeless. This paper presents an evaluation of an effort to incorporate an evidence-based, dual disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) into case management teams serving Veterans who are homeless, using an implementation strategy called Getting To Outcomes (GTO).

METHODS

This Hybrid Type III, cluster-randomized controlled trial assessed the impact of GTO over and above MISSION-Vet Implementation as Usual (IU). Both conditions received standard MISSION-Vet training and manuals. The GTO group received an implementation manual, training, technical assistance, and data feedback. The study occurred in teams at three large VA Medical Centers over 2 years. Within each team, existing sub-teams (case managers and Veterans they serve) were the clusters randomly assigned. The trial assessed MISSION-Vet services delivered and collected via administrative data and implementation barriers and facilitators, via semi-structured interview.

RESULTS

No case managers in the IU group initiated MISSION-Vet while 68% in the GTO group did. Seven percent of Veterans with case managers in the GTO group received at least one MISSION-Vet session. Most case managers appreciated the MISSION-Vet materials and felt the GTO planning meetings supported using MISSION-Vet. Case manager interviews also showed that MISSION-Vet could be confusing; there was little involvement from leadership after their initial agreement to participate; the data feedback system had a number of difficulties; and case managers did not have the resources to implement all aspects of MISSION-Vet.

CONCLUSIONS

This project shows that GTO-like support can help launch new practices but that multiple implementation facilitators are needed for successful execution of a complex evidence-based program like MISSION-Vet.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01430741.

The development of a national nutrition and mental health research agenda with comparison of priorities among diverse stakeholders

OBJECTIVE

To develop a national nutrition and mental health research agenda based on the engagement of diverse stakeholders and to assess research priorities by stakeholder groups.

DESIGN

A staged, integrated and participatory initiative was implemented to structure a national nutrition and mental health research agenda that included: (i) national stakeholder consultations to prioritize research questions; (ii) a workshop involving national representatives from research, policy and practice to further define priorities; (iii) triangulation of data to formulate the agenda; and (iv) test hypotheses about stakeholder influences on decision making.

SETTING

Canada.

SUBJECTS

Diverse stakeholders including researchers, academics, administrators, service providers, policy makers, practitioners, non-profit, industry and funding agency representatives, front-line workers, individuals with lived experience of a mental health condition and those who provide care for them.

RESULTS

This first-of-its-kind research priority-setting initiative showed points of agreement among diverse stakeholders (n 899) on research priorities aimed at service provision; however, respondents with lived experience of a mental health condition (themselves or a family member) placed emphasis on prevention and mental health promotion-based research. The final integrated agenda identified four research priorities, including programmes and services, service provider roles, the determinants of health and knowledge translation and exchange. These research priorities aim to identify effective models of care, enhance collaboration, inform policy makers and foster knowledge dissemination.

CONCLUSIONS

Since a predictor of research uptake is the involvement of relevant stakeholders, a sustained and deliberate effort must continue to engage collaboration that will lead to the optimization of nutrition and mental health-related outcomes.

Client, Contextual and Program Elements Influencing Supported Employment: A Literature Review

Supported employment is an evidence-based practice with a well-established research base. Most studies track such outcomes as employment rates, time to employment and wages earned. Few studies address client and contextual factors that impact outcomes or consider program elements beyond those that comprise the individual placement and support model. This paper reviews existing literature to shed light on the following questions: (1) What impact do labour market trends have on the effectiveness of SE? (2) How lasting are the effects of SE and what factors influence longevity of SE effects? (3) What levels and types of employment are targeted by SE? (4) What are the characteristics of people who benefit from SE? (5) What is the role of peer support in SE? and (6) What are the barriers to effective SE implementation? Research findings are synthesized and suggestions for service enhancements are offered so that the model can continue to evolve.

Practitioner characteristics and organizational contexts as essential elements in the evidence-based practice versus cultural competence debate

The different pathways chosen to efficiently and effectively provide relief to those struggling with mental health challenges reflect different assumptions about the human condition and have led to disagreements over which intervention strategies are best suited to particular individuals or populations. Evidence-based practice and culturally competent services, as discussed within the United States, have been characterized as opposites. However, neither approach captures all of the elements that embody the full treatment experience. This article offers a framework that includes the personal identity of the practitioner and the organizational context as two elements that serve as active agents in the helping relationship, although they have rarely been included in the discourse about evidence-based practice or cultural competence. Suggestions for practice, education, and research are included based on this analysis.

Facilitators and barriers to implementation of the AIDES initiative, a social innovation for participative assessment of children in need and for coordination of services

As part of an implementation evaluation, this study aims to identify the conditions of practice that facilitated or hindered implementation of the AIDES initiative, a social innovation to support collaboration between partners involved with vulnerable children. Evaluators conducted qualitative telephone interviews with 36 respondents (19 practitioners and 17 managers) who participated in the AIDES initiative trial. Respondents were chosen to include all participating organisations (child protection services, prevention social services). Participants' comments were submitted to descriptive content analysis. Conditions facilitating or hindering implementation of the initiative included the following dimensions: (1) implementation quality; (2) organisational elements (organisational functioning, cooperation between organisations); (3) socio-political issues; and (4) personal and professional characteristics. The study highlights critical elements to consider in implementing and maintaining significant changes in practice in organisations providing assistance to vulnerable children and their families. Social innovations that do not consider such elements are likely to compromise their implementation and sustainability. We must prevent promising social changes from being considered unrealistic or inappropriate due to contextual barriers.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the 'how to' component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of 'packaged' approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and 'reproducibility.' We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the 'how to' component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of 'packaged' approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and 'reproducibility.' We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

Implementation strategies: recommendations for specifying and reporting

Implementation strategies have unparalleled importance in implementation science, as they constitute the ‘how to’ component of changing healthcare practice. Yet, implementation researchers and other stakeholders are not able to fully utilize the findings of studies focusing on implementation strategies because they are often inconsistently labelled and poorly described, are rarely justified theoretically, lack operational definitions or manuals to guide their use, and are part of ‘packaged’ approaches whose specific elements are poorly understood. We address the challenges of specifying and reporting implementation strategies encountered by researchers who design, conduct, and report research on implementation strategies. Specifically, we propose guidelines for naming, defining, and operationalizing implementation strategies in terms of seven dimensions: actor, the action, action targets, temporality, dose, implementation outcomes addressed, and theoretical justification. Ultimately, implementation strategies cannot be used in practice or tested in research without a full description of their components and how they should be used. As with all intervention research, their descriptions must be precise enough to enable measurement and ‘reproducibility.’ We propose these recommendations to improve the reporting of implementation strategies in research studies and to stimulate further identification of elements pertinent to implementation strategies that should be included in reporting guidelines for implementation strategies.

Mental health clinicians' experiences of implementing evidence-based treatments

Implementation research has tremendous potential to bridge the research-practice gap; however, we know more about barriers to evidence-based care than the factors that contribute to the adoption and sustainability of evidence-based treatments. In this qualitative study the authors explore the experiences of clinicians (N = 11) who were implementing evidence-based treatments, highlighting the factors that they perceived to be most critical to successful implementation. The clinicians' narratives reveal many leverage points that can inform administrators, clinical supervisors, and clinicians who wish to implement evidence-based treatments, as well as other stakeholders who wish to develop and test strategies for moving evidence-based treatments into routine care.

Transdisciplinary collaboration and endorsement of pharmacological and psychosocial evidence-based practices by medical and psychosocial substance abuse treatment providers in the United States

AIMS

To examine the relative contribution of providers' professional affiliation (medical vs. non-medical), involvement in research, and training needs for associations with endorsement of the following evidence-based practices (EBPs): (1) pharmacological - buprenorphine treatment and (2) psychosocial - Cognitive Behavioural Therapy (CBT).

METHODS

Secondary analysis from a 2008 survey of a national sample (n = 571) of substance abuse treatment providers (medical, social workers, psychologists and counsellors) affiliated with the United States National Institute on Drug Abuse's National Drug Abuse Treatment Clinical Trials Network. Multivariate linear regression models to analyze cross-sectional survey data.

FINDINGS

Results demonstrated that medical providers and providers with previous research involvement more strongly endorsed the effectiveness of buprenorphine over CBT. Compared to medical providers, psychosocial providers more strongly endorsed CBT. There was a positive association between needing training in rapport with patients and endorsement of buprenorphine and a negative association with CBT. There was a positive association between needing training in behavioural management and needs assessment and endorsement of CBT.

CONCLUSIONS

Results underscore the importance of providers' involvement in research and the need for training medical and non-medical providers in practice areas that can purposely enhance their use of pharmacological and psychosocial EBPs.

The experts rate: supervisory behaviors that impact the implementation of evidence-based practices

The purpose of this study was to identify the critical behaviors of supervisors for the successful implementation of evidence-based practice in adult mental health. Experts who work with supervisors to support implementation in three evidence-based practices were surveyed. The three evidence-based practices included Assertive Community Treatment, Integrated Dual Diagnosis Treatment, and Supported Employment. There was substantial agreement among experts as to the importance of supervisory behaviors in the areas of facilitating team meetings, building and enhancing staff skills, monitoring and using outcomes, and continuous quality improvement activities.

Lessons learned in developing community mental health care in North America

This paper summarizes the findings for North America of the WPA Task Force on Steps, Obstacles and Mistakes to Avoid in the Implementation of Community Mental Health Care. Community mental health has evolved over five decades in the United States and Canada. The United States has led the world in innovation and spending, but provide variable quality of care; Canada has steadily developed a more uniform public health system for less cost. Lessons learned from North America include: team-based approaches and other evidence-based practices, when implemented with high fidelity, can improve outcomes in routine mental health care settings; recovery ideology and peer support enhance care, though they have not been studied rigorously; effective community-based care for people with serious mental disorders is expensive.

A compilation of strategies for implementing clinical innovations in health and mental health

Efforts to identify, develop, refine, and test strategies to disseminate and implement evidence-based treatments have been prioritized in order to improve the quality of health and mental health care delivery. However, this task is complicated by an implementation science literature characterized by inconsistent language use and inadequate descriptions of implementation strategies. This article brings more depth and clarity to implementation research and practice by presenting a consolidated compilation of discrete implementation strategies, based on a review of 205 sources published between 1995 and 2011. The resulting compilation includes 68 implementation strategies and definitions, which are grouped according to six key implementation processes: planning, educating, financing, restructuring, managing quality, and attending to the policy context. This consolidated compilation can serve as a reference to stakeholders who wish to implement clinical innovations in health and mental health care and can facilitate the development of multifaceted, multilevel implementation plans that are tailored to local contexts.

A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping

Background

The goal of this study was to assess potential differences between administrators/policymakers and those involved in direct practice regarding factors believed to be barriers or facilitating factors to evidence-based practice (EBP) implementation in a large public mental health service system in the United States.

Methods

Participants included mental health system county officials, agency directors, program managers, clinical staff, administrative staff, and consumers. As part of concept mapping procedures, brainstorming groups were conducted with each target group to identify specific factors believed to be barriers or facilitating factors to EBP implementation in a large public mental health system. Statements were sorted by similarity and rated by each participant in regard to their perceived importance and changeability. Multidimensional scaling, cluster analysis, descriptive statistics and t-tests were used to analyze the data.

Results

A total of 105 statements were distilled into 14 clusters using concept-mapping procedures. Perceptions of importance of factors affecting EBP implementation varied between the two groups, with those involved in direct practice assigning significantly higher ratings to the importance of Clinical Perceptions and the impact of EBP implementation on clinical practice. Consistent with previous studies, financial concerns (costs, funding) were rated among the most important and least likely to change by both groups.

Conclusions

EBP implementation is a complex process, and different stakeholders may hold different opinions regarding the relative importance of the impact of EBP implementation. Implementation efforts must include input from stakeholders at multiple levels to bring divergent and convergent perspectives to light.

Evidence-based practice: perceptions, skills, and activities of pediatric health care professionals

The purposes of this descriptive study were to evaluate pediatric professional staffs' perceptions of evidence-based practice (EBP), to identify individual and organizational barriers experienced when applying EBP into practice, and to determine differences in perceptions by those attending and not attending an EBP education series. A total of 486 pediatric health care professionals, 56 of whom attended all of the sessions, completed an anonymous online survey. Professional staff participated in and valued EBP activities but identified barriers to full implementation. Participants in the EBP series were significantly different in several positive ways. Implications for further education and research are delineated based on survey results.

Evidence-based practice implementation in Kansas

Over the last 8 years, Kansas has been successful in the implementation of evidence-based practices. This paper describes the strategies used at multiple levels of the mental health system including: state policy, provider agency management, fidelity and outcome monitoring, supervisor training and support, and practitioner training. The challenges going forth are described.

An effective intervention delivered at sub-therapeutic dose becomes an ineffective intervention

Evidence to date indicates that the individual placement and support model of supported employment helps people with mental illness to obtain competitive jobs. The study by Howard et al (this issue) is the first unsuccessful trial of this model. Vocational workers had far fewer contacts with clients and employers than normal.

The sustainability of evidence-based practices in routine mental health agencies

The research presented here reports on sustainability of the practices within the National Implementing Evidence Based Practices Project for people with serious mental illness. Forty-nine sites completed the initial 2-year implementation phase and were the focus of our study. Our aims were to discern the number of sites that sustained practices 2 years after implementation, the reasons for sustaining or not sustaining, differences in characteristics between the two groups, and the extent and nature of practice adaptations. We used a mixed-methods approach, based on a telephone survey that gathered qualitative and quantitative data from site representatives and others familiar with the sites and practices during the follow-up period. We found that 80% of sites sustained their practices for 2 years post-implementation, that sustainers differed from non-sustainers in several domains: financing, training, fidelity and agency leadership, and that most sites adapted practices moderately to meet state and local needs.

A pan-Canadian evaluation of supported employment programs dedicated to people with severe mental disorders

Supported employment (SE) is an evidence-based practice that helps people with severe mental disorders obtain competitive employment. The implementation of SE programs in different social contexts has led to adaptations of the SE components, therefore impacting the fidelity/quality of these services. The objective of this study was to assess the implementation of SE services in three Canadian provinces by assessing the fidelity and describing components of SE services using the Quality of Supported Employment Implementation Scale. About 23 SE programs participated in this study. Cluster analyses revealed six profiles of SE programs that varied from high to low level of fidelity with a stronger focus on a particular component, and reflected the reality of service delivery settings. Future investigations are warranted to evaluate relationships between the levels of implementation of SE components and work outcomes while considering individual characteristics of people registered in SE programs.

The 2009 schizophrenia PORT psychosocial treatment recommendations and summary statements

The Schizophrenia Patient Outcomes Research Team (PORT) psychosocial treatment recommendations provide a comprehensive summary of current evidence-based psychosocial treatment interventions for persons with schizophrenia. There have been 2 previous sets of psychosocial treatment recommendations (Lehman AF, Steinwachs DM. Translating research into practice: the Schizophrenia Patient Outcomes Research Team (PORT) treatment recommendations. Schizophr Bull. 1998;24:1-10 and Lehman AF, Kreyenbuhl J, Buchanan RW, et al. The Schizophrenia Patient Outcomes Research Team (PORT): updated treatment recommendations 2003. Schizophr Bull. 2004;30:193-217). This article reports the third set of PORT recommendations that includes updated reviews in 7 areas as well as adding 5 new areas of review. Members of the psychosocial Evidence Review Group conducted reviews of the literature in each intervention area and drafted the recommendation or summary statement with supporting discussion. A Psychosocial Advisory Committee was consulted in all aspects of the review, and an expert panel commented on draft recommendations and summary statements. Our review process produced 8 treatment recommendations in the following areas: assertive community treatment, supported employment, cognitive behavioral therapy, family-based services, token economy, skills training, psychosocial interventions for alcohol and substance use disorders, and psychosocial interventions for weight management. Reviews of treatments focused on medication adherence, cognitive remediation, psychosocial treatments for recent onset schizophrenia, and peer support and peer-delivered services indicated that none of these treatment areas yet have enough evidence to merit a treatment recommendation, though each is an emerging area of interest. This update of PORT psychosocial treatment recommendations underscores both the expansion of knowledge regarding psychosocial treatments for persons with schizophrenia at the same time as the limitations in their implementation in clinical practice settings.

A conditional model of evidence-based decision making

RATIONALE, AIMS AND OBJECTIVES

Efforts to describe how individual treatment decisions are informed by systematic knowledge have been hindered by a standard that gauges the quality of clinical decisions by their adherence to guidelines and evidence-based practices. This paper tests a new contextual standard that gauges the incorporation of knowledge into practice and develops a model of evidence-based decision making. Previous work found that the forecasted outcome of a treatment guideline exerts a highly significant influence on how it is used in making decisions. This study proposed that forecasted outcomes affect the recognition of a treatment scenario, and this recognition triggers distinct contextual decision strategies.

METHODS

Twenty-one volunteers from a psychiatric residency programme responded to 64 case vignettes, 16 in each of the four treatment scenarios. The vignettes represented a fully balanced within-subjects design that included guideline switching criteria and patient-specific factors. For each vignette, participants indicated whether they endorsed the guideline's recommendation.

RESULTS

Clinicians used consistent contextual decision strategies in responding to clearly positive or negative forecasts. When forecasts were more ambiguous or risky, their strategies became complex and relatively inconsistent.

CONCLUSION

The results support a three-step model of evidence-based decision making, in which clinicians recognize a decision scenario, apply a simple contextual strategy, then if necessary engage a more complex strategy to resolve discrepancies between general guidelines and specific cases. The paper concludes by noting study limitations and discussing implications of the model for future research in clinical and shared decision making, training and guideline development.

Implementing evidence-based practices for people with schizophrenia

Over the last decade, a consensus has emerged regarding a set of evidence-based practices for schizophrenia that address symptom management and psychosocial functioning. Yet, surveys suggest that the great majority of the population of individuals with schizophrenia do not receive evidence-based care. In this article, we review the empirical literature on implementation of evidence-based practices for schizophrenia patients. We first examine lessons learned from implementation studies in general medicine. We then summarize the implementation literature specific to schizophrenia, including medication practices, psychosocial interventions, information technology, and state- and federal-level interventions. We conclude with recommendations for future directions.

Individualization and quality improvement: two new scales to complement measurement of program fidelity

Fidelity scales have been widely used to assess program adherence to the principles of an evidence-based practice, but they do not measure important aspects of quality of care. Pragmatic scales measuring clinical quality of services are needed to complement fidelity scales measuring structural aspects of program implementation. As part of the instrumentation developed for the National Implementing Evidence-Based Practices Project, we piloted a new instrument with two 5-item quality scales, Individualization (a client-level quality scale) and Quality Improvement (an organizational-level quality scale). Pairs of independent fidelity assessors conducted fidelity reviews in 49 sites in 8 states at baseline and at four subsequent 6-month intervals over a 2-year follow-up period. The assessors followed a standardized protocol to administer these quality scales during daylong site visits; during these same visits they assessed programs on fidelity to the evidence-based practice that the site was seeking to implement. Assessors achieved acceptable interrater reliability for both Individualization and Quality Improvement. Principal components factor analysis confirmed the 2-scale structure. The two scales were modestly correlated with each other and with the evidence-based practice fidelity scales. Over the first year, Individualization and Quality Improvement improved, but showed little or no improvement during the last year of follow-up. The two newly developed scales showed adequate psychometric properties in this preliminary study, but further research is needed to assess their validity and utility in routine clinical practice.

Scales to evaluate quality of medication management: development and psychometric properties

This paper describes the psychometric properties of two fidelity scales created as part of the Substance Abuse and Mental Health Services Administration (SAMHSA) sponsored medication management toolkit and their metric properties when used in 26 public mental health clinics with 50 prescribers. A 23-item scale, based on chart reviews, was developed to assess whether prescribers are following good medication practices, in conjunction with a 17-item scale to assess organizational support for and evaluation of prescriber adherence to recommended medication-related practices. Fundamental gaps in routine practice, including poor documentation of medication history and infrequent monitoring of symptoms and side effects were found.