Evidence-Based Practice Implementation in Community Mental Health Settings: The Relative Importance of Key Domains of Implementation Activity

Lessons From Implementing Research-Supported Practices to Address Psychiatric Illnesses in Two Countries

Across the globe, psychiatric illnesses are common, painful, often disabling, and sometimes deadly. Although well-researched practices exist to address these disorders, most people with psychiatric illnesses do not have access to care that has been demonstrated to be effective. Practical clinical leadership experience and engagement in evidence-based practice (EBP) implementation research in Colombia and the United States have demonstrated that multisite EBP implementation is possible and that effective implementation improves outcomes for people who develop psychiatric illnesses. Implementation requires financial and policy support but is effective only if onsite operational leaders actively overcome practical, sometimes site-specific barriers. Operational leaders have the greatest impact when they focus on changing the flow of work to help clinicians offer the desired EBP in their day-to-day care. Discovery science may bring new solutions in the future, but implementing practices that have already been demonstrated to be effective can relieve patients' suffering now.

Implementation of a crisis resolution team service improvement programme: a qualitative study of the critical ingredients for success

BACKGROUND

Crisis Resolution Teams (CRTs) offer home-based care for people in mental health crisis, as an alternative to hospital admission. The success of CRTs in England has been variable. In response to this, the CRT Optimization and RElapse prevention (CORE) study developed and trialled a 12-month Service Improvement Programme (SIP) based on a fidelity model. This paper describes a qualitative evaluation of the perspectives of CRT staff, managers, and programme facilitators. We identify barriers and facilitators to implementation, and mechanisms by which service improvements took place.

METHODS

Managers and staff from six purposively sampled CRTs were interviewed, as well as six facilitators who were employed to support the implementation of service improvement plans. Semi-structured focus groups and individual interviews were conducted and analysed using thematic analysis.

FINDINGS

A majority of participants viewed all components of the SIP as helpful in improving practice, although online resources were under-used. Perceived barriers to implementation centred principally around lack of staff time and ownership. Support from both senior staff and facilitators was essential in enabling teams to undertake the work associated with the SIP. All participating stakeholder groups reported that using the fidelity model to benchmark their CRT work to best practice and feel part of a 'bigger whole' was valuable.

CONCLUSION

CRT staff, managers and programme facilitators thought that a structured service improvement programme helped to increase fidelity to a best practice model. Flexibility (from all stakeholders) was key to enable service improvement actions to be manageable within time- and resource-poor teams.

Using audit and feedback to guide tailored implementations of measurement-based care in community mental health: a multiple case study

BACKGROUND

Audit and feedback (A&F) is an implementation strategy that can facilitate implementation tailoring by identifying gaps between desired and actual clinical care. While there are several theory-based propositions on which A&F components lead to better implementation outcomes, many have not been empirically investigated, and there is limited guidance for stakeholders when applying A&F in practice. The current study aims to illustrate A&F procedures in six community mental health clinics, with an emphasis on reporting A&F components that are relevant to theories of how feedback elicits behavior change.

METHODS

Six clinics from a larger trial using a tailored approach to implement measurement-based care (MBC) were analyzed for feedback content, delivery mechanisms, barriers to feedback, and outcomes of feedback using archival data. Pattern analysis was conducted to examine relations between A&F components and changes in MBC use.

RESULTS

Several sites utilized both aggregate and individualized data summaries, and data accuracy concerns were common. Feedback cycles featuring individual-level clinician data, data relevant to MBC barriers, and information requested by data recipients were related to patterns of increased MBC use.

CONCLUSIONS

These findings support extant theory, such as Feedback Intervention Theory. Mental health professionals wishing to apply A&F should consider establishing reciprocal feedback mechanisms on the quality and amount of data being received and adopting specific roles communicating and addressing data quality concerns.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02266134.

Implementation of strengths model case management in seven mental health agencies in Canada: Direct-service practitioners' implementation experience

RATIONALE

Implementation of strengths model case management is increasing internationally. However, few studies have focused on its implementation process, and none have specifically addressed the implementation experience of direct-service practitioners.

OBJECTIVE

This paper presents factors that facilitate and impede the successful implementation of the strengths model, with a specific focus on practitioners who deliver the intervention directly to service recipients.

METHOD

To address this objective, a qualitative study of seven mental health agencies that implemented the model was conducted, involving a combination of participant observations and qualitative semistructured interviews with case managers, team supervisors, and senior managers. Qualitative data were analyzed using open coding followed by axial coding. Finally, the findings were aligned with an adapted Consolidated Framework for Implementation Research.

RESULTS

Implementation of the strengths model involved a significant change in practice for case management practitioners. The results confirm that at the beginning of implementation, the strengths model was perceived as complex and not always adaptable to on-the-ground realities. With time, and with support from management, ongoing training and supervision sessions, and reflection and discussion, practitioners regained feelings of competence and resistance to the model diminished. The use of the model's structured team-based supervision tools was fundamental to supporting the implementation process by enabling an interactive and concrete training approach.

CONCLUSIONS

The more an approach leads to changes in daily practice and is perceived as complex, the more concrete support is needed during implementation. This article highlights the importance of attending to a practitioner's sense of personal effectiveness and competence in the adoption of new practices.

Can systematic implementation support improve programme fidelity by improving care providers' perceptions of implementation factors? A cluster randomized trial

BACKGROUND

Investigations of implementation factors (e.g., collegial support and sense of coherence) are recommended to better understand and address inadequate implementation outcomes. Little is known about the relationship between implementation factors and outcomes, especially in later phases of an implementation effort. The aims of this study were to assess the association between implementation success (measured by programme fidelity) and care providers' perceptions of implementation factors during an implementation process and to investigate whether these perceptions are affected by systematic implementation support.

METHODS

Using a cluster-randomized design, mental health clinics were drawn to receive implementation support for one (intervention) and not for another (control) of four evidence-based practices. Programme fidelity and care providers' perceptions (Implementation Process Assessment Tool questionnaire) were scored for both intervention and control groups at baseline, 6-, 12- and 18-months. Associations and group differences were tested by means of descriptive statistics (mean, standard deviation and confidence interval) and linear mixed effect analysis.

RESULTS

Including 33 mental health centres or wards, we found care providers' perceptions of a set of implementation factors to be associated with fidelity but not at baseline. After 18 months of implementation effort, fidelity and care providers' perceptions were strongly correlated (B (95% CI) = .7 (.2, 1.1), p = .004). Care providers perceived implementation factors more positively when implementation support was provided than when it was not (t (140) = 2.22, p = .028).

CONCLUSIONS

Implementation support can facilitate positive perceptions among care providers, which is associated with higher programme fidelity. To improve implementation success, we should pay more attention to how care providers constantly perceive implementation factors during all phases of the implementation effort. Further research is needed to investigate the validity of our findings in other settings and to improve our understanding of ongoing decision-making among care providers, i.e., the mechanisms of sustaining the high fidelity of recommended practices.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03271242 (registration date: 05.09.2017).

FAST: A Framework to Assess Speed of Translation of Health Innovations to Practice and Policy

The 17-year time span between discovery and application of evidence in practice has become a unifying challenge for implementation science and translational science more broadly. Further, global pandemics and social crises demand timely implementation of rapidly accruing evidence to reduce morbidity and mortality. Yet speed remains an understudied metric in implementation science. Prevailing evaluations of implementation lack a temporal aspect, and current approaches have not yielded rapid implementation. In this paper, we address speed as an important conceptual and methodological gap in implementation science. We aim to untangle the complexities of studying implementation speed, offer a framework to assess speed of translation (FAST), and provide guidance to measure speed in evaluating implementation. To facilitate specification and reporting on metrics of speed, we encourage consideration of stakeholder perspectives (e.g., comparison of varying priorities), referents (e.g., speed in attaining outcomes, transitioning between implementation phases), and observation windows (e.g., time from intervention development to first patient treated) in its measurement. The FAST framework identifies factors that may influence speed of implementation and potential effects of implementation speed. We propose a research agenda to advance understanding of the pace of implementation, including identifying accelerators and inhibitors to speed.

An evaluation of the feasibility of implementing a novel tobacco dependence treatment program for high-risk individuals into clinical practice within a community mental health center

Background

Individuals with serious mental illnesses experience deaths related to smoking at a higher prevalence than individuals without a psychotic-spectrum disorders. Traditional smoking cessation programs are often not effective among individuals with chronic mental disorders. Little is known about how to implement a tobacco cessation treatment program for this at-risk population within a community health center. The current study used qualitative methods to examine the factors that may enhance or impede the delivery of a novel tobacco cessation treatment for smokers with a psychotic-spectrum disorder diagnosis in an integrated care community health center.

Methods

Using a case study design, we conducted 22 semi-structured interviews with primary care providers, mental health providers, addiction counselors, case managers, intake specialists, schedulers, pharmacists, and administrative staff employed at the organization. Interviews were transcribed and themes were identified through a rich coding process.

Results

We identified environmental factors, organizational factors, provider factors and patient factors which describe the potential factors that may enhance or impede the implementation of a smoking cessation program at the integrated care community health center. Most notably, we identified that community mental health centers looking to implement a smoking cessation program for individuals with chronic mental health disorders should ensure the incentives for providers to participate align with the program’s objectives. Additionally, organizations should invest in educating providers to address stigma related to smoking cessation and nicotine use.

Conclusions

The findings of our study provide valuable insight for administrators to consider when implementing a smoking cessation program in an integrated care community health center. Our findings provide public health practitioners with potential considerations that should be discussed when designing and implementing a smoking cessation program for individuals with chronic mental disorders.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13033-022-00517-y.

Developing a tailored implementation action plan for a suicide prevention clinical intervention in an Australian mental health service: A qualitative study using the EPIS framework

Background: Tailoring implementation strategies to local contexts is a promising approach to supporting implementation and sustainment of evidence-based practices in health settings. While there is increasing research on tailored implementation of mental health interventions, implementation research on suicide prevention interventions is limited. This study aimed to evaluate implementation and subsequently develop a tailored action plan to support sustainment of an evidence-based suicide prevention intervention; Collaborative Assessment and Management of Suicidality (CAMS) in an Australian public mental health service. Methods: Approximately 150 mental health staff working within a regional and remote Local Health District in Australia were trained in CAMS. Semi-structured interviews and focus groups with frontline staff and clinical leaders were conducted to examine barriers and facilitators to using CAMS. Data were analysed using a reflexive thematic analysis approach and mapped to the Exploration, Preparation, Implementation and Sustainment (EPIS) framework and followed by stakeholder engagement to design a tailored implementation action plan based on a ‘tailored blueprint’ methodology. Results: A total of 22 barriers to implementing CAMS were identified. Based on the perceived impact on implementation fidelity and the feasibility of addressing identified barriers, six barriers were prioritised for addressing through an implementation action plan. These barriers were mapped to evidence-based implementation strategies and, in collaboration with local health district staff, goals and actionable steps for each strategy were generated. This information was combined into a tailored implementation plan to support the sustainable use of CAMS as part of routine care within this mental health service. Conclusions: This study provides an example of a collaborative approach to tailoring strategies for implementation on a large scale. Novel insights were obtained into the challenges of evaluating the implementation process and barriers to implementing an evidence-based suicide prevention treatment approach within a geographically large and varied mental health service in Australia. Plain language abstract: This study outlines the process of using a collaborative stakeholder engagement approach to develop tailored implementation plans. Using the Exploration Preparation Implementation Sustainment Framework, findings identify the barriers to and strategies for implementing a clinical suicide prevention intervention in an Australian community mental health setting. This is the first known study to use an implementation science framework to investigate the implementation of the clinical suicide prevention intervention (Collaborative Assessment and Management of Suicidality) within a community mental health setting. This work highlights the challenges of conducting implementation research in a dynamic public health service.

A mixed-methods study to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings

It has been well-documented that the degree to which interventions are implemented with fidelity in typical service settings has varied. Frequently, interventions are developed and tested in highly controlled or early adopter settings. Less attention has been given to what implementation looks like in usual care, and which factors promote practitioners' ability to implement with fidelity. Individuals and organizations implementing interventions in the real world receive varying levels of external supports and may apply a new intervention unaided. The purpose of this mixed-methods study was to explore factors that support implementation as intended in local community agencies. In the quantitative phase of this study, 32 case planners implementing Family Connections (FC), a child maltreatment preventive intervention, completed a survey about their perceptions of practitioner and organizational factors related to fidelity. The survey data were connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner's ability to implement an intervention in usual care. The quantitative and qualitative results suggest supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioners' ability to learn and use FC. The quantitative results suggest that the level of education was positively associated with fidelity and perceptions of the intervention's limitations may be negatively related to implementation. Additional components that influence implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

Plain Language Abstract

This mixed-methods study sought to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings. The study first asked case planners about their perceptions of practitioner and organizational factors related to fidelity through an online survey. This survey was connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner’s ability to implement a maltreatment prevention intervention. Both methods of the study suggest that various aspects of supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioner's ability to learn and use the intervention. Additional components that influence the implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

Contributions et défis de l’utilisation des technopédagogies à des fins de soutien à l’appropriation des meilleures pratiques en santé mentale

Contexte Malgré les ressources considérables consacrées et les efforts des nombreux acteurs concernés, l’écart entre la production de connaissances scientifiques et leur utilisation dans la pratique demeure un défi. L’utilisation des technologies de l’information et des communications (TICs) constitue un outil précieux pour réduire cet écart. Afin de relever ce défi, un projet de démonstration misant sur l’utilisation des technologies à des fins d’application des connaissances a été déployé auprès de 23 équipes de soutien dans la communauté de 5 régions du Québec (2016-2018). Plus de 324 professionnels de la santé mentale, chefs d’équipe et gestionnaires ont bénéficié de l’initiative À portée de main, les meilleures pratiques axées vers le rétablissement.

Objectif Cet article présente les résultats de l’étude de satisfaction effectuée auprès des chefs d’équipe responsables du soutien clinique dans les équipes de soutien dans la communauté à l’étude. L’objectif de cette étude consiste à enrichir la compréhension d’enjeux colligés en cours d’implantation et d’émettre des recommandations en vue d’une mise à l’échelle pérenne du programme d’application des connaissances mis en oeuvre.

Méthode Un devis qualitatif en recherche évaluative a été privilégié. Au terme du processus d’implantation du programme, 2 entrevues de groupe ont été effectuées avec les chefs d’équipe. Une analyse de contenu suivant une approche inductive à 3 niveaux de codification a été réalisée.

Résultats Les résultats montrent un décalage numérique important au sein du réseau de la santé et des services sociaux québécois comparativement à d’autres secteurs d’activités. Les participants soulignent l’importance de se doter de mécanismes d’échange et de transfert des connaissances intégrés aux pratiques organisationnelles (temps dédié, supervision clinique formelle, etc.) mettant à contribution les TICs.

Conclusion Malgré une importante mise à niveau technologique requise, les résultats confirment la pertinence d’utiliser les technopédagogies comme principal moyen pour soutenir l’application des connaissances et la transformation des pratiques. Les outils conçus et les modalités de soutien explorés semblent faciliter l’accès aux meilleures pratiques en santé mentale et favoriser leur adoption.

Community providers' experiences with evidence-based practices: The role of therapist race/ethnicity

OBJECTIVES

Examining therapists' experiences implementing evidence-based practices (EBPs) is fundamental to understanding how these interventions are perceived, adapted, and delivered in community settings. However, little is known about racial/ethnic variation in the experiences of therapists serving racial/ethnic minority youth and their families. Through an innovative QUAN → qual → QUAN mixed-methods approach, we examined differences in therapists' perceptions, adaptations performed, and client-engagement challenges in the largest county-operated department of mental health in the United States.

METHOD

Surveys were completed by 743 therapists (Latinx [44%], White [34%], other ethnic minority [22%]), most of whom were female (88%), master's level (85%), and unlicensed (58%). A subset of therapists (n = 60) completed semistructured interviews.

RESULTS

Latinx therapists reported more positive experiences implementing EBPs, making more adaptations to EBPs, and encountering fewer client-engagement challenges than therapists from other racial/ethnic groups. Qualitative analyses expanded on these results, revealing that Latinx therapists commonly described adapting EBPs in terms of language and culture to improve fit and promote client engagement. Informed by these qualitative themes, a refined statistical model revealed that the ability to deliver EBPs in languages other than English might have accounted for differences in therapist-reported EBP adaptations and client-engagement challenges.

CONCLUSIONS

The findings suggest that racial/ethnic minority therapists have positive experiences in implementing EBPs in community settings. In the case of Latinx therapists, bilingual/bicultural competence may facilitate adapting EBPs in ways that reduce perceptions of engagement challenges with racially/ethnically diverse clients. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How practice setting affects family physicians' views on genetic screening: a qualitative study

BACKGROUND

Genetic screening (GS), defined as the clinical testing of a population to identify asymptomatic individuals with the aim of providing those identified as high risk with prevention, early treatment, or reproductive options. Genetic screening (GS) improves patient outcomes and is accessible to the community. Family physicians (FPs) are ideally placed to offer GS. There is a need for FPs to adopt GS to address anticipated genetic specialist shortages.

OBJECTIVE

To explore FP attitudes, perceived roles, motivators and barriers, towards GS; and explore similarities and differences between private and public sector FPs.

METHODS

We developed a semi-structured interview guide using existing literature. We interviewed private and public sector FPs recruited by purposive, convenience and snowballing strategies, by telephone or video to theme saturation. All sessions were audio-recorded, transcribed and coded for themes by two independent researchers with an adjudicator.

RESULTS

Thirty FPs were interviewed (15 private, 15 public). Theme saturation was reached for each group. A total of 12 themes (6 common, 3 from private-practice participants, 3 public-employed participants) emerged. Six common major themes emerged: personal lack of training and experience, roles and relevance of GS to family medicine, reluctance and resistance to adding GS to practice, FP motivations for adoption, patient factors as barrier, and potential solutions. Three themes (all facilitators) were unique to the private group: strong rapport with patients, high practice autonomy, and high patient literacy. Three themes (all barriers) were unique to the public group: lack of control, patients' lower socioeconomic status, and rigid administrative infrastructure.

CONCLUSION

FPs are motivated to incorporate GS but need support for implementation. Policy-makers should consider the practice setting when introducing new screening functions. Strategies to change FP behaviours should be sensitive to their sense of autonomy, and the external factors (either as facilitators or as barriers) shaping FP practices in a given clinical setting.

Development of an Evidence-Based Best Practice Model for Teams Managing Crisis in Dementia: Protocol for a Qualitative Study

Background

Teams working in the community to manage crisis in dementia currently exist, but with widely varying models of practice, it is difficult to determine the effectiveness of such teams.

Objective

The aim of this study is to develop a “best practice model” for dementia services managing crisis, as well as a set of resources to help teams implement this model to measure and improve practice delivery. These will be the best practice tool and toolkit to be utilized by teams to improve the effectiveness of crisis teams working with older people with dementia and their caregivers. This paper describes the protocol for a prospective study using qualitative methods to establish an understanding of the current practice to develop a “best practice model.”

Methods

Participants (people with dementia, caregivers, staff members, and stakeholders) from a variety of geographical areas, with a broad experience of crisis and noncrisis work, will be purposively selected to participate in qualitative approaches including interviews, focus groups, a consensus workshop, and development and field testing of both the best practice tool and toolkit.

Results

Data were collected between October 2016 and August 2018. Thematic analysis will be utilized to establish the current working of teams managing crisis in dementia in order to draw together elements of the best practice.

Conclusions

This is the first study to systematically explore the requirements needed to fulfill effective and appropriate home management for people with dementia and their caregivers at the time of mental health crisis, as delivered by teams managing crisis in dementia. This systematic approach to development will support greater acceptability and validity of the best practice tool and toolkit and lay the foundation for a large scale trial with teams managing crisis in dementia across England to investigate the effects on practice and impact on service provision, as well as the associated experiences of people with dementia and their caregivers.

International Registered Report Identifier (IRRID)

RR1-10.2196/14781

A Systematic Review of the Characteristics and Efficacy of Recovery Training for Mental Health Staff: Implications for Supported Accommodation Services

Evidence suggests a link between recovery-oriented practise and service user outcomes in supported accommodation settings. Current clinical guidelines recommend recovery training for supported accommodation staff, however evidence relating to the effectiveness of this type of training is unclear. This review aimed to describe and compare the characteristics and efficacy of existing recovery training packages for mental health staff. The appropriateness and applicability of the interventions was considered in relation to UK supported accommodation services. Initial search processes returned 830 papers. After duplicate removal, inclusion and exclusion criteria were applied to 489 papers, leaving a final sample of seven papers. Data were reviewed using a narrative synthesis approach. The reviewed papers showed variation in the aims, frequency, and duration of the training interventions, although all included content consistent with the five-domains of the CHIME model. All interventions used direct, in-person teaching, and prioritised interactive, experiential learning, however a number were limited by the absence of feedback, the use of one-off, rather than repeated/follow-up sessions, and a reliance on classroom-based, rather than in-vivo, training. There was limited evidence to suggest a consistent effect of training on staff or service user outcomes, and there was no clear association between the delivery and design characteristics of the interventions and reported outcomes. In considering the development of recovery training for supported accommodation staff, little guidance can be taken from the reviewed literature. Any training package must be developed with consideration of the unique contextual and organisational characteristics of these services. The authors recommend viewing training as one component of a broader goal of service transformation.

A Tale of Four States: Factors Influencing the Statewide Adoption of IPS

Evidence-based supported employment has become a core community mental health service in much of the U.S. Although a national learning community has facilitated progress in about half of the states, other states have tried to implement evidence-based supported employment on their own. Many studies have examined site-level factors influencing implementation of supported employment, but few have focused on the role of state agency policies and actions. This study examined four states that have not joined the learning community, comparing two that have implemented with success (adopting states) and two that have faced challenges (non-adopting states). This comparative case study approach compared barriers, facilitators, and strategies in two states adopting IPS to two states that did not. The authors examined quantitative data from public records and conducted content analysis of qualitative and quantitative data from key informant interviews. The two non-adopting states lacked model clarity, funding, focus on people with serious mental illness, and collaboration between state mental health and vocational rehabilitation agencies. The two successful states experienced similar barriers but overcame them following lawsuit settlements that required implementation of evidence-based supported employment. Key strategies for successful implementation were funding, fidelity monitoring, technical assistance, and collaboration between state mental health and vocational rehabilitation agencies. With legal settlements serving as the catalyst, states facing challenges to implementing evidence-based supported employment can achieve success using standard implementation strategies to fund and ensure the quality of services.

Leadership that promotes successful implementation of community-based mental health interventions

Community-based interventions (CBIs) have become an effective approach to promote mental well-being. To increase organizational capacity and effectively deliver CBIs, leadership is crucial. The current study aims to identify leadership qualities that are facilitators of CBI implementation. This paper presents evaluation findings from a national effort emphasizing CBIs for improving the mental well-being of men and boys in 16 local communities. The study utilized a mixed-method design, including both examining each grantee as a case study and conducting a cross-site evaluation. Findings revealed five leadership qualities that contributed to CBIs' success: personal vision, value-based leadership, relationship-oriented leadership, task-oriented leadership, and leadership development. The results outline strategies for strengthening organizational capacity through these leadership qualities. This study adds to the limited literature on leadership qualities that strengthen organizational capacity to successfully implement CBIs and can serve as a guide for funders, program planners, and evaluators.

The CORE service improvement programme for mental health crisis resolution teams: results from a cluster-randomised trial

BACKGROUND

Crisis resolution teams (CRTs) offer brief, intensive home treatment for people experiencing mental health crisis. CRT implementation is highly variable; positive trial outcomes have not been reproduced in scaled-up CRT care.

AIMS

To evaluate a 1-year programme to improve CRTs' model fidelity in a non-masked, cluster-randomised trial (part of the Crisis team Optimisation and RElapse prevention (CORE) research programme, trial registration number: ISRCTN47185233).

METHOD

Fifteen CRTs in England received an intervention, informed by the US Implementing Evidence-Based Practice project, involving support from a CRT facilitator, online implementation resources and regular team fidelity reviews. Ten control CRTs received no additional support. The primary outcome was patient satisfaction, measured by the Client Satisfaction Questionnaire (CSQ-8), completed by 15 patients per team at CRT discharge (n = 375). Secondary outcomes: CRT model fidelity, continuity of care, staff well-being, in-patient admissions and bed use and CRT readmissions were also evaluated.

RESULTS

All CRTs were retained in the trial. Median follow-up CSQ-8 score was 28 in each group: the adjusted average in the intervention group was higher than in the control group by 0.97 (95% CI -1.02 to 2.97) but this was not significant (P = 0.34). There were fewer in-patient admissions, lower in-patient bed use and better staff psychological health in intervention teams. Model fidelity rose in most intervention teams and was significantly higher than in control teams at follow-up. There were no significant effects for other outcomes.

CONCLUSIONS

The CRT service improvement programme did not achieve its primary aim of improving patient satisfaction. It showed some promise in improving CRT model fidelity and reducing acute in-patient admissions.

Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) in Pragmatic Clinical Trials of Mental Health Services Implementation: Methods and Applied Case Study

Pragmatic clinical trials of mental health services are increasingly being developed to establish comparative effectiveness, influence sustainable implementation, and address real world policy decisions. However, use of time and resource intensive qualitative methods in pragmatic trials may be inconsistent with the aims of efficiency and cost minimization. This paper introduces a qualitative method known as Rapid Assessment Procedure-Informed Clinical Ethnography (RAPICE) that combines the techniques of Rapid Assessment Procedures with clinical ethnography. A case study is presented to illustrate how RAPICE can be used to efficiently understand pragmatic trial implementation processes and associated real world policy implications.

Getting to the table: Agency characteristics and evidence-based intervention adoption in children's mental health care

Scaling evidence-based interventions (EBI) for children and families across healthcare systems can expand public health impact. Research has identified EBI adoption determinants. However, less understood are characteristics of agencies that opt in across the stages of adoption. This study examined the relationship between agency (N=69) characteristics (e.g., revenue) and four adoption stages during a large-scale trial of an EBI for children with significant behavioral difficulties and their families. 48 (70%) of agencies demonstrated interest, 28 (41%) scheduled an informational meeting, 20 (29%) received training, and 16 (22%) demonstrated EBI uptake. Analyses indicated no differences in characteristics and initial interest. However, agencies with small-sized revenue had significantly reduced odds at other adoption stages. Implications for strategies to bring EBI access to scale are discussed.

A virtual learning collaborative to implement health promotion in routine mental health settings: Protocol for a cluster randomized trial

BACKGROUND

Despite widespread use of learning collaboratives in health care, few randomized trials have evaluated their effectiveness. The primary aim of this cluster randomized implementation trial is to evaluate the effectiveness of a virtual learning collaborative (VLC) in the implementation of a lifestyle intervention for persons with serious mental illness (SMI) in routine mental health settings, compared to standard individual technical assistance.

METHODS

Forty-eight mental health provider organizations from across the United States will be recruited to participate in the trial. The evidence-based practice to be implemented is the InSHAPE health promotion intervention for persons with SMI. Sites will be stratified by size and randomized to receive an 18-month intensive group-based VLC with monthly learning sessions or individual technical assistance with four scheduled conference calls over 18 months. Sites will be enrolled in three blocks of 16 sites each. The primary outcomes are InSHAPE program participation and fidelity, and participant weight loss; secondary outcomes are program operation, program uptake, participant health behaviors of physical activity and nutrition, organizational change, and program sustainment. Implementation outcomes are measured at 3, 6, 12, 18, and 24 months after the program start-up. Participant-level outcomes are measured at fixed intervals every 3 months after each participant enrolls in the study.

DISCUSSION

This study will determine whether VLCs are an effective implementation strategy among resource-limited providers when the new practice necessitates a shift in mission, scope of practice, type of services delivered, and new financing.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT03891368 Registered 25 March 2019, retrospectively registered. https://clinicaltrials.gov/ct2/show/NCT03891368?term=NCT03891368&rank=1.

Provider Attitudes, Organizational Readiness for Change, and Uptake of Research Supported Treatment

PURPOSE

As implementation and evaluation of research supported treatment (RST) expands across diverse mental health settings, it is essential to understand the facilitators and inhibitors of uptake. The current study examined the relationships between organizational readiness for change, attitudes toward RST, and use of RST among a sample primarily of social workers.

METHODS

Participants included 158 providers from public child mental health outpatient clinics in the New York metropolitan area. Data were analyzed using structural equation modeling.

RESULTS

Use of RST was greater among providers who were younger and had fewer years of professional experience. Both organizational need for change and climate directly and indirectly related to use of RST through attitudes toward RST (partial mediator).

DISCUSSION

The organizational context is an important factor that relates to attitudes toward and use of RST in practice. Future research is needed to continue examining factors of RST uptake and sustainability.

Stepped Care as an Implementation and Service Delivery Model for Cognitive Behavioral Therapy for Psychosis

Cognitive behavioral therapy for schizophrenia spectrum disorders is an evidence-based treatment that is recommended by United States schizophrenia treatment guidelines. Based on recent estimates, only 0.3% of individuals with a primary psychotic disorder are able to access this treatment in the United States. Stepped care interventions have shown promise as an applied treatment delivery model in other settings and for other psychotherapeutic interventions. The current paper describes how the stepped care model can be applied to CBT for psychosis in the US to increase access to the intervention in community mental health settings by leveraging the multidisciplinary team.

The technology specialist: a 21st century support role in clinical care

Mental health clinicians, clients, and researchers have shown keen interest in using technology to support mental health recovery. However, technology has not been routinely integrated into clinical care. Clients use a wide range of digital tools and apps to help manage their mental health, but clinicians rarely discuss this form of self-management in clinical interactions. This absence of communication is concerning because the safety and quality of the digital tools and apps people use may negatively affect their mental health outcomes. Mental health systems could benefit from someone to help identify technology-based supports that reflect current evidence and minimize privacy and security concerns. This technology specialist may also enhance the therapeutic bond between the client and the clinician. In working with a technology specialist, clients may begin to gain a sense of control over their mental health, and perhaps use fewer mental health services.

Crisis resolution teams for people experiencing mental health crises: the CORE mixed-methods research programme including two RCTs

Background

Crisis resolution teams (CRTs) seek to avert hospital admissions by providing intensive home treatment for people experiencing a mental health crisis. The CRT model has not been highly specified. CRT care is often experienced as ending abruptly and relapse rates following CRT discharge are high.

Aims

The aims of CORE (Crisis resolution team Optimisation and RElapse prevention) workstream 1 were to specify a model of best practice for CRTs, develop a measure to assess adherence to this model and evaluate service improvement resources to help CRTs implement the model with high fidelity. The aim of CORE workstream 2 was to evaluate a peer-provided self-management programme aimed at reducing relapse following CRT support.

Methods

Workstream 1 was based on a systematic review, national CRT manager survey and stakeholder qualitative interviews to develop a CRT fidelity scale through a concept mapping process with stakeholders (n = 68). This was piloted in CRTs nationwide (n = 75). A CRT service improvement programme (SIP) was then developed and evaluated in a cluster randomised trial: 15 CRTs received the SIP over 1 year; 10 teams acted as controls. The primary outcome was service user satisfaction. Secondary outcomes included CRT model fidelity, catchment area inpatient admission rates and staff well-being. Workstream 2 was a peer-provided self-management programme that was developed through an iterative process of systematic literature reviewing, stakeholder consultation and preliminary testing. This intervention was evaluated in a randomised controlled trial: 221 participants recruited from CRTs received the intervention and 220 did not. The primary outcome was re-admission to acute care at 1 year of follow-up. Secondary outcomes included time to re-admission and number of days in acute care over 1 year of follow-up and symptoms and personal recovery measured at 4 and 18 months’ follow-up.

Results

Workstream 1 – a 39-item CRT fidelity scale demonstrated acceptability, face validity and promising inter-rater reliability. CRT implementation in England was highly variable. The SIP trial did not produce a positive result for patient satisfaction [median Client Satisfaction Questionnaire score of 28 in both groups at follow-up; coefficient 0.97, 95% confidence interval (CI) –1.02 to 2.97]. The programme achieved modest increases in model fidelity. Intervention teams achieved lower inpatient admission rates and less inpatient bed use. Qualitative evaluation suggested that the programme was generally well received. Workstream 2 – the trial yielded a statistically significant result for the primary outcome, in which rates of re-admission to acute care over 1 year of follow-up were lower in the intervention group than in the control group (odds ratio 0.66, 95% CI 0.43 to 0.99; p = 0.044). Time to re-admission was lower and satisfaction with care was greater in the intervention group at 4 months’ follow-up. There were no other significant differences between groups in the secondary outcomes.

Limitations

Limitations in workstream 1 included uncertainty regarding the representativeness of the sample for the primary outcome and lack of blinding for assessment. In workstream 2, the limitations included the complexity of the intervention, preventing clarity about which were effective elements.

Conclusions

The CRT SIP did not achieve all its aims but showed potential promise as a means to increase CRT model fidelity and reduce inpatient service use. The peer-provided self-management intervention is an effective means to reduce relapse rates for people leaving CRT care.

Study registration

The randomised controlled trials were registered as Current Controlled Trials ISRCTN47185233 and ISRCTN01027104. The systematic reviews were registered as PROSPERO CRD42013006415 and CRD42017043048.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Barriers and facilitators to improving health care for adults with intellectual and developmental disabilities: what do staff tell us?

INTRODUCTION

Adults with intellectual and developmental disabilities (IDD) have high rates of morbidity and are less likely to receive preventive care. Emergency departments and primary care clinics are important entry points into the health care system. Improving care in these settings can lead to increased prevention activities, early disease identification, and ongoing management. We studied barriers and facilitators to improving the care of patients with IDD in three primary and three emergency care sites in Ontario.

METHODS

Data sources included structured implementation logs at each site, focus groups (n = 5) and interviews (n = 8). Barriers and facilitators were coded deductively based on the Consolidated Framework for Implementation Research (CFIR). Synthesis to higher level themes was achieved through review and discussion by the research team. Focus was given to differences between higher and lower implementing sites.

RESULTS

All sites were challenged to prioritize care improvement for a small, complex population and varied levels of implementation were achieved. Having national guidelines, using local data to demonstrate need and sharing evidence on value were important engagement strategies. Factors present at higher implementing sites included strong champions, alignment with site mandate, and use of electronic prompts/reminders. Lower implementing sites showed more passive endorsement of the innovation and had lower capacity to implement.

CONCLUSION

Providing effective care for small, complex groups, such as adults with IDD, is critical to improving long-term health outcomes but is challenging to achieve. At a systemic level, funding incentives, access to expertise and improved electronic record systems may enhance capacity.

Who Wants Feedback? Effects of Transformational Leadership and Leader-Member Exchange on Mental Health Practitioners' Attitudes Toward Feedback

OBJECTIVE

The purpose of this study was to identify the mechanisms through which different aspects of leadership affect mental health practitioners' attitudes toward supervisory feedback.

METHODS

Data were collected from 363 practitioners nested in 68 treatment teams in public-sector mental health organizations. A multilevel path analysis was conducted to examine the associations of transformational leadership (supervisor's ability to inspire others to follow a course of action) and leader-member exchange (quality of the supervisor-practitioner relationship) with practitioner attitudes toward feedback.

RESULTS

Transformational leadership and leader-member exchange were directly and positively associated with practitioners' attitudes toward feedback. Transformational leadership was also indirectly associated with practitioners' attitudes toward feedback through the quality of supervisor-practitioner relationships.

CONCLUSIONS

Study results contribute to the growing body of evidence suggesting that leaders play a key role in shaping mental health service delivery. Both leadership behavior and high-quality supervisor-practitioner relationships are important in supporting practitioners in delivering evidence-based mental health care. Policymakers, administrators, and researchers should consider an integrative approach when developing leadership training interventions.

The housing first technical assistance and training (HFTAT) implementation strategy: outcomes from a mixed methods study of three programs

BACKGROUND

This paper discusses the initial testing of the Housing First Training and Technical Assistance (HFTAT) Program, a multifaceted, distance-based strategy for the implementation of the Housing First (HF) supportive housing model. HF is a complex housing intervention for serving people living with serious mental illness and a substance use disorder that requires significant individual- and structural-level changes to implement. As such, the HFTAT employs a combined training and consultation approach to target different levels of the organization. Training delivered to all organizational staff focuses on building individual knowledge and uses narrative storytelling to overcome attitudinal implementation barriers. Consultation seeks to build skills through technical assistance and fidelity audit and feedback.

METHOD

We employed a mixed method design to understand both individual-level (e.g., satisfaction with the HFTAT, HF knowledge acquisition and retention, and HF acceptability and appropriateness) and structural-level (e.g., fidelity) outcomes. Quantitative data were collected at various time points, and qualitative data were collected at the end of HFTAT activities. Staff and administrators (n = 113) from three programs across three states participated in the study.

RESULTS

Satisfaction with both training and consultation was high, and discussions demonstrated both activities were necessary. Flexibility of training modality and narrative storytelling were particular strengths, while digital badging and the community of practice were perceived as less valuable because of incompatibilities with the work context. HF knowledge was high post training and retained after 3-month follow-up. Participants reported training helped them better understand the model. Attitudes toward evidence-based interventions improved over 6 months, with qualitative data supporting this but demonstrating some minor concerns related to acceptability and appropriateness. Fidelity scores for all programs improved over 9 months.

CONCLUSION

The HFTAT was a well-liked and generally useful implementation strategy. Results support prior research pointing to the value of both (a) multifaceted strategies and (b) combined training and consultation approaches. The study also provides evidence for narrative storytelling as an approach for changing attitudinal implementation barriers. The need for compatibility between specific elements of an implementation strategy and the work environment was also observed.

A Cross-Site Partnership to Examine Implementation and Sustainability of a School-Based Trauma Program

Objectives

Schools have been identified as an ideal setting for increasing access to mental health services particularly for underserved minority youth. The emerging field of implementation science has begun to systematically investigate strategies for more efficiently integrating evidence-based practices into community settings. Significantly less translational research has focused specifically on the school setting. To address this need, we examined the implementation of a school-based trauma intervention across three distinct regions.

Design

We conducted key informant interviews guided by Mendel's Framework of Dissemination in Health Services Intervention Research with multiple school stakeholders to examine what school organizational characteristics influence the adoption and implementation process and sustainability of Cognitive Behavioral Intervention for Trauma in Schools (CBITS). Participants were selected from schools in three geographic regions in the United States: Western, Midwestern, and Southern.

Results

Our findings reveal that while sites had some common organizational factors that appeared to facilitate implementation, regions differed in how they compensated for less robust implementation domains. Across all regions, school stakeholders recognized the need for services to support students impacted by trauma. In the Western region, there was no centralized district policy for implementation; therefore, implementation was facilitated by school-level change agents and supervision support from the district mental health unit. In the Midwestern region, centralized district policies drove implementation. In both the Midwestern and Southern regions, implementation was facilitated by collaboration with a local mental health agency.

Conclusions

This study contributes to the paucity of empirical information on the organizational factors that influence the implementation of evidence-based mental health interventions in schools. Our findings reveal that different implementation strategies across policies, structures, and resources can result in implementation of a school-based intervention. Frameworks such as Mendel's can be helpful in identifying areas of strength and improvement of implementation within a school organization.

Clinical Case Consultation and Experiential Learning in Cognitive Behavioral Therapy Implementation: Brief Qualitative Investigation

There has been an increased emphasis on the implementation of cognitive behavioral therapy (CBT) to community mental health (CMH) systems due to its broad efficacy. Previous research has highlighted the importance of ongoing consultation in this process. The current study clarifies the role and process of clinical case consultation in the implementation of CBT to CMH from the consultants' perspective. Trainers from two large-scale implementation initiatives (n = 27) were surveyed regarding their strategies used in the consultation process. Historically, researchers have focused on trainees who view relational variables as the most effective elements of consultation; however, in the current study, trainers perceived experiential learning as being the most effective consultation strategy for helping CMH clinicians learn CBT. Other aspects of consultation such as case conceptualization and practice sample review are discussed in terms of their relative perceived utility in raising CMH clinician's competency to treat comorbid patients.

Community Therapist Reports of Client Engagement Challenges During the Implementation of Multiple EBPs in Children's Mental Health

This study investigated therapist reports of client engagement challenges in delivering evidence-based practices (EBPs), within the context of a large-scale implementation of multiple practices in children's mental health services. Data were drawn from an online survey of 668 therapists (88% female, 35.0% Non-Hispanic White). The majority of therapists (75.4%) endorsed at least one client engagement challenge during their implementation of an EBP with an identified client over the previous two months. Two types of EBP client engagement challenges with distinct correlates but similar overall frequencies could be readily differentiated - Expressed Client Concerns and Limited Client Engagement in therapy activities. Limited Client Engagement were more commonly reported for clients with externalizing problems and by therapists with higher emotional exhaustion and negative perceptions of the specific EBP being delivered, whereas Hispanic/Latino therapists were less likely to report Limited Client Engagement. In contrast, Expressed Client Concerns were more commonly reported by therapists with positive general attitudes towards EBPs, and among therapists delivering a parent training EBP. Limited Client Engagement but not Expressed Concerns were linked with therapists' self-reported ability to carry out the EBP with the target client. Findings suggest that client engagement challenges are frequent during the delivery of EBPs, but may impact implementation differently based on whether they relate to challenges in engaging clients in therapy activities versus addressing concerns raised by clients.

Leadership, Organizational Climate, and Perceived Burden of Evidence-Based Practice in Mental Health Services

The use of evidence-based practices (EBPs) is associated with favorable client outcomes, yet perceived burden of using EBPs may affect the adoption and implementation of such practices. Multilevel path analysis was used to examine the associations of transformational leadership with organizational climate, and their associations with perceived burden of using EBPs. Results indicated significant relationships between transformational leadership and empowering and demoralizing climates, and between demoralizing climate and perceived burden of EBPs. We found significant indirect associations of leadership and perceived burden through organizational climate. Findings suggest that further research is needed to examine the extent to which improving leadership and organizational climate may reduce perceived burden and use of EBPs with the ultimate goal of enhancing quality of care.

Exploring drivers and challenges in implementation of health promotion in community mental health services: a qualitative multi-site case study using Normalization Process Theory

Background

There is an increased interest in improving the physical health of people with mental illness. Little is known about implementing health promotion interventions in adult mental health organisations where many users also have physical health problems. The literature suggests that contextual factors are important for implementation in community settings. This study focused on the change process and analysed the implementation of a structural health promotion intervention in community mental health organisations in different contexts in Denmark.

Methods

The study was based on a qualitative multiple-case design and included two municipal and two regional provider organisations. Data were various written sources and 13 semi-structured interviews with 22 key managers and frontline staff. The analysis was organised around the four main constructs of Normalization Process Theory: Coherence, Cognitive Participation, Collective Action, and Reflexive Monitoring.

Results

Coherence: Most respondents found the intervention to be meaningful in that the intervention fitted well into existing goals, practices and treatment approaches. Cognitive Participation: Management engagement varied across providers and low engagement impeded implementation. Engaging all staff was a general problem although some of the initial resistance was apparently overcome. Collective Action: Daily enactment depended on staff being attentive and flexible enough to manage the complex needs and varying capacities of users. Reflexive Monitoring: During implementation, staff evaluations of the progress and impact of the intervention were mostly informal and ad hoc and staff used these to make on-going adjustments to activities. Overall, characteristics of context common to all providers (work force and user groups) seemed to be more important for implementation than differences in the external political-administrative context.

Conclusions

In terms of research, future studies should adopt a more bottom-up, grounded description of context and pay closer attention to the interplay between different dimensions of implementation. In terms of practice, future interventions need to better facilitate the translation of the initial sense of general meaning into daily practice by active local management support that occurs throughout the implementation process and that systematically connects the intervention to existing practices.

Building flexibility and managing complexity in community mental health: lessons learned in a large urban centre

BACKGROUND

Across many jurisdictions, adults with complex mental health and social needs face challenges accessing appropriate supports due to system fragmentation and strict eligibility criteria of existing services. To support this underserviced population, Toronto's local health authority launched two novel community mental health models in 2014, inspired by Flexible Assertive Community Team principles. This study explores service user and provider perspectives on the acceptability of these services, and lessons learned during early implementation.

METHODS

We purposively sampled 49 stakeholders (staff, physicians, service users, health systems stakeholders) and conducted 17 semi-structured qualitative interviews and 5 focus groups between October 23, 2014 and March 2, 2015, exploring stakeholder perspectives on the newly launched team based models, as well as activities and strategies employed to support early implementation. Interviews and focus groups were audio recorded, transcribed verbatim and analyzed using thematic analysis.

RESULTS

Findings revealed wide-ranging endorsement for the two team-based models' success in engaging the target population of adults with complex service needs. Implementation strengths included the broad recognition of existing service gaps, the use of interdisciplinary teams and experienced service providers, broad partnerships and collaboration among various service sectors, training and team building activities. Emerging challenges included lack of complementary support services such as suitable housing, organizational contexts reluctant to embrace change and risk associated with complexity, as well as limited service provider and organizational capacity to deliver evidence-based interventions.

CONCLUSIONS

Findings identified implementation drivers at the practitioner, program, and system levels, specific to the implementation of community mental health interventions for adults with complex health and social needs. These can inform future efforts to address the health and support needs of this vulnerable population.

A Survey of Health Equity Practices in Early Psychosis Intervention Programs: A Starting Point for Improvement

Equity has been identified as a core component of quality healthcare in Ontario. However, translating policy into practice can be challenging. This paper reports results from a province-wide survey of early psychosis intervention programs to assess the extent to which equity has been incorporated into program delivery. All 56 programs (100%) completed the survey. Results found that while most programs perceive that they are meeting equity aims, they reported limited use of practices to support this aim, and few systematically collect information on performance. Strategies to improve equity in practice are discussed.

How effective is the comprehensive approach to rehabilitation (CARe) methodology? A cluster randomized controlled trial

BACKGROUND

The CARe methodology aims to improve the quality of life of people with severe mental illness by supporting them in realizing their goals, handling their vulnerability and improving the quality of their social environment. This study aims to investigate the effectiveness of the CARe methodology for people with severe mental illness on their quality of life, personal recovery, participation, hope, empowerment, self-efficacy beliefs and unmet needs.

METHODS

A cluster Randomized Controlled Trial (RCT) was conducted in 14 teams of three organizations for sheltered and supported housing in the Netherlands. Teams in the intervention group received training in the CARe methodology. Teams in the control group continued working according to care as usual. Questionnaires were filled out at baseline, after 10 months and after 20 months. A total of 263 clients participated in the study.

RESULTS

Quality of life increased in both groups, however, no differences between the intervention and control group were found. Recovery and social functioning did not change over time. Regarding the secondary outcomes, the number of unmet needs decreased in both groups. All intervention teams received the complete training program. The model fidelity at T1 was 53.4% for the intervention group and 33.4% for the control group. At T2 this was 50.6% for the intervention group and 37.2% for the control group.

CONCLUSION

All clients improved in quality of life. However we did not find significant differences between the clients of the both conditions on any outcome measure. Possible explanations of these results are: the difficulty to implement rehabilitation-supporting practice, the content of the methodology and the difficulty to improve the lives of a group of people with longstanding and severe impairments in a relatively short period. More research is needed on how to improve effects of rehabilitation trainings in practice and on outcome level.

TRIAL REGISTRATION

ISRCTN77355880 , retrospectively registered (05/07/2013).

Alpha test results for a Housing First eLearning strategy: the value of multiple qualitative methods for intervention design

Background

Detailed descriptions of implementation strategies are lacking, and there is a corresponding dearth of information regarding methods employed in implementation strategy development. This paper describes methods and findings related to the alpha testing of eLearning modules developed as part of the Housing First Technical Assistance and Training (HFTAT) program's development. Alpha testing is an approach for improving the quality of a product prior to beta (i.e., real world) testing with potential applications for intervention development.

Methods

Ten participants in two cities tested the modules. We collected data through (1) a structured log where participants were asked to record their experiences as they worked through the modules; (2) a brief online questionnaire delivered at the end of each module; and (3) focus groups.

Results

The alpha test provided useful data related to the acceptability and feasibility of eLearning as an implementation strategy, as well as identifying a number of technical issues and bugs. Each of the qualitative methods used provided unique and valuable information. In particular, logs were the most useful for identifying technical issues, and focus groups provided high quality data regarding how the intervention could best be used as an implementation strategy.

Conclusions

Alpha testing was a valuable step in intervention development, providing us an understanding of issues that would have been more difficult to address at a later stage of the study. As a result, we were able to improve the modules prior to pilot testing of the entire HFTAT. Researchers wishing to alpha test interventions prior to piloting should balance the unique benefits of different data collection approaches with the need to minimize burdens for themselves and participants.

Barriers and Facilitators to Sustainment of an Evidence-Based Supported Employment Program

Large-scale initiatives to expand evidence-based practices are often poorly implemented and rarely endure. The purpose of this study was to identify the perceived barriers and facilitators to sustainment of an evidence-based supported employment program, Individual Placement and Support (IPS). Within a 2-year prospective study of sustainment among 129 IPS programs in 13 states participating in a national learning community, we interviewed IPS team leaders and coded their responses to semi-structured interviews using a conceptual framework adapted from another large-scale implementation study. Leaders in 122 agencies (95%) that sustained their IPS programs identified funding, prioritization, and workforce characteristics as both key facilitators and barriers. Additional key factors were lack of local community supports as a barrier and leadership and structured workflow as facilitators. Within the IPS learning community, team leaders attributed the sustainment of their program to funding, prioritization, workforce, agency leadership, and structured workflow. The actions of the learning community's leadership, state governments, and local programs together may have contributed to the high sustainment rate.

Influence of Organizational Characteristics on Success in Implementing Process Improvement Goals in Correctional Treatment Settings

Although research indicates that organizational characteristics substantially influence the adoption and use of evidence-based practices (EBPs), there has been little empirical research on organizational factors most likely to influence successful implementation of EBPs, particularly in criminal justice settings. This study examined organizational characteristics related to the success of change teams in achieving improvements in assessment and case-planning procedures for persons leaving correctional settings and receiving community services. In this evaluation of the Organizational Process Improvement Intervention (OPII), part of the National Institute on Drug Abuse's (NIDA's) Criminal Justice Drug Abuse Treatment Studies (CJDATS) cooperative, 21 sites were randomized to an early-start or a delayed-start condition. For this analysis, data from both conditions were combined. Agencies with fewer program needs, good communication, adequate staffing levels, good supervision, positive attitude toward rehabilitation, and higher institutional capacity for change were better able to implement planned changes in assessment and case-planning procedures. Such agencies may be better candidates for implementation improvement strategies, whereas other agencies could benefit from pre-intervention efforts aimed at strengthening these characteristics before attempting to improve assessment procedures.

Integrating the Integrated: Merging Integrated Dual Diagnosis Treatment (IDDT) with Housing First

This study reports on a housing program that merged two evidence-based practices frequently applied in tandem: Integrated Dual Diagnosis Treatment and Housing First. Quantitative measures show that consumers in the program were receptive to supportive housing and core services. These quantitative measures, when considered alongside qualitative interviews, suggest that in order for more consumers to move through the IDDT stages of educational and vocational advancement, the staff will need to emphasize the permanence of supportive housing. Thus, the study also demonstrates the importance of integrating results of qualitative evaluations with quantitative data to strengthen a program's evidence base.

Pediatric eMental healthcare technologies: a systematic review of implementation foci in research studies, and government and organizational documents

Background

Researchers, healthcare planners, and policymakers convey a sense of urgency in using eMental healthcare technologies to improve pediatric mental healthcare availability and access. Yet, different stakeholders may focus on different aspects of implementation. We conducted a systematic review to identify implementation foci in research studies and government/organizational documents for eMental healthcare technologies for pediatric mental healthcare.

Methods

A search of eleven electronic databases and grey literature was conducted. We included research studies and documents from organization and government websites if the focus included eMental healthcare technology for children/adolescents (0–18 years), and implementation was studied and reported (research studies) or goals/recommendations regarding implementation were made (documents). We assessed study quality using the Mixed Methods Appraisal Tool and document quality using the Appraisal of Guidelines for Research & Evaluation II. Implementation information was grouped according to Proctor and colleagues’ implementation outcomes—acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability—and grouped separately for studies and documents.

Results

Twenty research studies and nine government/organizational documents met eligibility criteria. These articles represented implementation of eMental healthcare technologies in the USA (14 studies), United Kingdom (2 documents, 3 studies), Canada (2 documents, 1 study), Australia (4 documents, 1 study), New Zealand (1 study), and the Netherlands (1 document). The quality of research studies was excellent (n = 11), good (n = 6), and poor (n = 1). These eMental health studies focused on the acceptability (70%, n = 14) and appropriateness (50%, n = 10) of eMental healthcare technologies to users and mental healthcare professionals. The quality of government and organizational documents was high (n = 2), medium (n = 6), and low (n = 1). These documents focused on cost (100%, n = 9), penetration (89%, n = 8), feasibility (78%, n = 7), and sustainability (67%, n = 6) of implementing eMental healthcare technology.

Conclusion

To date, research studies have largely focused on acceptability and appropriateness, while government/organizational documents state goals and recommendations regarding costs, feasibility, and sustainability of eMental healthcare technologies. These differences suggest that the research evidence available for pediatric eMental healthcare technologies does not reflect the focus of governments and organizations. Partnerships between researchers, healthcare planners, and policymakers may help to align implementation research with policy development, decision-making, and funding foci.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-017-0608-6) contains supplementary material, which is available to authorized users.

New directions for psychiatric rehabilitation in the USA

American researchers have led the world in developing, evaluating, and disseminating evidence-based psychiatric rehabilitation practices for people with serious mental illness. Paradoxically, however, the USA lags behind most industrialized nations in providing access to high-quality mental health and psychiatric services. This essay examines several evidence-based practices developed in the USA, the spread of these practices, the barriers to ensuring availability to people who could benefit from these services, and some promising directions for overcoming the barriers. Factors influencing the growth and sustainment of effective client-centred practices include the availability of adequate and stable funding, committed leadership, and the influence of vested interests. Two strategies for promoting the spread and sustainment of well-implemented evidence-based practices are the adoption of fidelity scales and learning communities.

Implementation of Integrated Service Networks under the Quebec Mental Health Reform: Facilitators and Barriers associated with Different Territorial Profiles

INTRODUCTION

This study evaluates implementation of the Quebec Mental Health Reform (2005-2015), which promoted the development of integrated service networks, in 11 local service networks organized into four territorial groups according to socio-demographic characteristics and mental health services offered.

METHODS

Data were collected from documents concerning networks; structured questionnaires completed by 90 managers and by 16 respondent-psychiatrists; and semi-structured interviews with 102 network stakeholders. Factors associated with implementation and integration were organized according to: 1) reform characteristics; 2) implementation context; 3) organizational characteristics; and 4) integration strategies.

RESULTS

While local networks were in a process of development and expansion, none were fully integrated at the time of the study. Facilitators and barriers to implementation and integration were primarily associated with organizational characteristics. Integration was best achieved in larger networks including a general hospital with a psychiatric department, followed by networks with a psychiatric hospital. Formalized integration strategies such as service agreements, liaison officers, and joint training reduced some barriers to implementation in networks experiencing less favourable conditions.

CONCLUSION

Strategies for the implementation of healthcare reform and integrated service networks should include sustained support and training in best-practices, adequate performance indicators and resources, formalized integration strategies to improve network coordination and suitable initiatives to promote staff retention.

Bridging the Gap Between Research and Practice in Mental Health Service Settings: An Overview of Developments in Implementation Theory and Research

Twenty years after Sobell's (1996) influential call to integrate advances in behavior therapy into clinical settings, significant progress has been made in implementation science. In this narrative review, we provide an overview of implementation research findings and highlight recent findings that can inform efforts to bridge the gap between research and practice in mental health service settings. Key findings are summarized, organized according to levels of influence described in two implementation frameworks: The Exploration, Planning, Implementation, and Sustainment Framework (EPIS; Aarons et al., 2011) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). Important levels of influence to consider when implementing new treatments include the outer context, inner context, characteristics of the individual, and characteristics of the innovation. Research on strategies to prepare clinicians to deliver evidence-based psychosocial treatments (EBPTs) and to address contextual barriers to implementation at each level is described, with discussion of implications for the implementation of EBPTs and next steps for research.

Providers' attitudes toward evidence-based practices: is it just about providers, or do practices matter, too?

Evidence-based practice (EBP) attitudes were measured in a sample of Los Angeles County mental health service providers. Three types of data were collected: provider demographic characteristics, attitudes toward EBP in general, and attitudes toward specific EBPs being implemented in the county. Providers could reliably rate characteristics of specific EBPs, and these ratings differed across interventions. Preliminary implementation data indicate that appealing features of an EBP relate to the degree to which providers use it. These findings suggest that assessing EBP-specific attitudes is feasible and may offer implementation-relevant information beyond that gained solely from providers' general attitudes toward EBP.

Supporting shared decision making beyond consumer-prescriber interactions: Initial development of the CommonGround fidelity scale

Shared decision-making has become a central tenet of recovery-oriented, person-centered mental health care, yet the practice is not always transferred to the routine psychiatric visit. Supporting the practice at the system level, beyond the interactions of consumers and medication prescribers, is needed for successful adoption of shared decision-making. CommonGround is a systemic approach, intended to be part of a larger integration of shared decision-making tools and practices at the system level. We discuss the organizational components that CommonGround uses to facilitate shared decision-making, and we present a fidelity scale to assess how well the system is being implemented.

Implementation of a Substance Use Recovery Support Mobile Phone App in Community Settings: Qualitative Study of Clinician and Staff Perspectives of Facilitators and Barriers

BACKGROUND

Research supports the effectiveness of technology-based treatment approaches for substance use disorders. These approaches have the potential to broaden the reach of evidence-based care. Yet, there is limited understanding of factors associated with implementation of technology-based care approaches in different service settings.

OBJECTIVES

In this study, we explored provider and staff perceptions of facilitators and barriers to implementation of a mobile phone substance use recovery support app with clients in 4 service settings.

METHODS

Interviews were conducted with leadership and provider stakeholders (N=12) from 4 agencies in the first year of an implementation trial of the mobile phone app. We used the Consolidated Framework for Implementation Research as the conceptual foundation for identifying facilitators and barriers to implementation.

RESULTS

Implementation process facilitators included careful planning of all aspects of implementation before launch, engaging a dedicated team to implement and foster motivation, working collaboratively with the app development team to address technical barriers and adapt the app to meet client and agency needs, and consistently reviewing app usage data to inform progress. Implementation support strategies included training all staff to promote organization awareness about the recovery support app and emphasize its priority as a clinical care tool, encouraging clients to try the technology before committing to use, scaling rollout to clients, setting clear expectations with clients about use of the app, and using peer coaches and consistent client-centered messaging to promote engagement. Perceived compatibility of the mobile phone app with agency and client needs and readiness to implement emerged as salient agency-level implementation facilitators. Facilitating characteristics of the recovery support app itself included evidence of its impact for recovery support, perceived relative advantage of the app over usual care, the ability to adapt the app to improve client use, and its ease of use. The mobile phone itself was a strong motivation for clients to opt in to use the app in settings that provided phones. App access was limited in settings that did not provide phones owing to lack of mobile phone ownership or incompatibility of the app with clients' mobile phones. Individual differences in technology literacy and provider beliefs about substance use care either facilitated or challenged implementation. Awareness of patient needs and resources facilitated implementation, whereas external policies and regulations regarding technology use introduced barriers to implementation.

CONCLUSIONS

The conceptually grounded facilitators and barriers identified in this study can guide systematic targeting of strategies to improve implementation of mobile phone interventions in community treatment settings. Results also inform the design of technology-based therapeutic tools. This study highlights directions for research with regard to implementation of technology-based behavioral health care approaches.

Factors influencing the effectiveness of audit and feedback: nurses' perceptions

AIM

To explore the perceptions of nurses in an acute care setting on factors influencing the effectiveness of audit and feedback.

BACKGROUND

Audit and feedback is widely used and recommended in nursing to promote evidence-based practice and to improve care quality. Yet the literature has shown a limited to modest effect at most. Audit and feedback will continue to be unreliable until we learn what influences its effectiveness.

METHOD

A qualitative study was conducted using individual, semi-structured interviews with 14 registered nurses in an acute care teaching hospital in Montreal, Canada.

RESULTS

Three themes were identified: the relevance of audit and feedback, particularly understanding the purpose of audit and feedback and the prioritisation of audit criteria; the audit and feedback process, including its timing and feedback characteristics; and individual factors, such as personality and perceived accountability.

CONCLUSION

According to participants, they were likely to have a better response to audit and feedback when they perceived that it was relevant and that the process fitted their preferences.

IMPLICATIONS FOR NURSING MANAGEMENT

This study benefits nursing leaders and managers involved in quality improvement by providing a better understanding of nurses' perceptions on how best to use audit and feedback as a strategy to promote evidence-based practice.

Identifying and ranking implicit leadership strategies to promote evidence-based practice implementation in addiction health services

Background

Despite a solid research base supporting evidence-based practices (EBPs) for addiction treatment such as contingency management and medication-assisted treatment, these services are rarely implemented and delivered in community-based addiction treatment programs in the USA. As a result, many clients do not benefit from the most current and efficacious treatments, resulting in reduced quality of care and compromised treatment outcomes. Previous research indicates that addiction program leaders play a key role in supporting EBP adoption and use. The present study expanded on this previous work to identify strategies that addiction treatment program leaders report using to implement new practices.

Methods

We relied on a staged and iterative mixed-methods approach to achieve the following four goals: (a) collect data using focus groups and semistructured interviews and conduct analyses to identify implicit managerial strategies for implementation, (b) use surveys to quantitatively rank strategy effectiveness, (c) determine how strategies fit with existing theories of organizational management and change, and (d) use a consensus group to corroborate and expand on the results of the previous three stages. Each goal corresponded to a methodological phase, which included data collection and analytic approaches to identify and evaluate leadership interventions that facilitate EBP implementation in community-based addiction treatment programs.

Results

Findings show that the top-ranked strategies involved the recruitment and selection of staff members receptive to change, offering support and requesting feedback during the implementation process, and offering in vivo and hands-on training. Most strategies corresponded to emergent implementation leadership approaches that also utilize principles of transformational and transactional leadership styles. Leadership behaviors represented orientations such as being proactive to respond to implementation needs, supportive to assist staff members during the uptake of new practices, knowledgeable to properly guide the implementation process, and perseverant to address ongoing barriers that are likely to stall implementation efforts.

Conclusions

These findings emphasize how leadership approaches are leveraged to facilitate the implementation and delivery of EBPs in publicly funded addiction treatment programs. Findings have implications for the content and structure of leadership interventions needed in community-based addiction treatment programs and the development of leadership interventions in these and other service settings.

Challenges in the Implementation of Measurement Feedback Systems

This commentary on the articles published in the special section on the development and implementation of measurement feedback systems (MFSs) discusses three challenging themes in the process of MFS implementation: design and planning, organizational context, and sustainability and unintended consequences. It is argued that the implementation of MFSs is complex, but is an important step in improving outcomes in routine care for children and young persons.