A mixed methods multiple case study of implementation as usual in children's social service organizations: study protocol

Child Adult Relationship Enhancement in Primary Care (PriCARE): study design/protocol for a randomized trial of a primary care-based group parenting intervention to prevent child maltreatment

BACKGROUND

Child maltreatment (CM) is a pervasive public health problem and there is a critical need for brief, effective, scalable prevention programs. Problematic parent-child relationships lie at the heart of CM. Parents who maltreat their children are more likely to have punitive parenting styles characterized by high rates of negative interaction and ineffective discipline strategies with over-reliance on punishment. Thus, parenting interventions that strengthen parent-child relationships, teach positive discipline techniques, decrease harsh parenting, and decrease child behavioral problems hold promise as CM prevention strategies. Challenges in engaging parents, particularly low-income and minority parents, and a lack of knowledge regarding effective implementation strategies, however, have greatly limited the reach and impact of parenting interventions. Child Adult Relationship Enhancement in Primary Care (PriCARE)/Criando Niños con CARIÑO is a 6-session group parenting intervention that holds promise in addressing these challenges because PriCARE/CARIÑO was (1) developed and iteratively adapted with input from racially and ethnically diverse families, including low-income families and (2) designed specifically for implementation in primary care with inclusion of strategies to align with usual care workflow to increase uptake and retention.

METHODS

This study is a multicenter randomized controlled trial with two parallel arms. Children, 2-6 years old with Medicaid/CHIP/no insurance, and their English- and Spanish-speaking caregivers recruited from pediatric primary care clinics in Philadelphia and North Carolina will be enrolled. Caregivers assigned to the intervention regimen will attend PriCARE/CARIÑO and receive usual care. Caregivers assigned to the control regimen will receive usual care only. The primary outcome is occurrence of an investigation for CM by child protective services during the 48 months following completion of the intervention. In addition, scores for CM risk, child behavior problems, harsh and neglectful parenting behaviors, caregiver stress, and caregiver-child interactions will be assessed as secondary outcome measures and for investigation of possible mechanisms of intervention-induced change. We will also identify PriCARE/CARIÑO implementation factors that may be barriers and facilitators to intervention referrals, enrollment, and attendance.

DISCUSSION

By evaluating proximal outcomes in addition to the distal outcome of CM, this study, the largest CM prevention trial with individual randomization, will help elucidate mechanisms of change and advance the science of CM prevention. This study will also gather critical information on factors influencing successful implementation and how to optimize intervention referrals, enrollment, and attendance to inform future dissemination and practical applications.

TRIAL REGISTRATION

This trial was registered on ClinicalTrials.gov (NCT05233150) on February 1, 2022, prior to enrolling subjects.

A scoping review of community-based post-opioid overdose intervention programs: implications of program structure and outcomes

BACKGROUND

An emergent intervention to address the opioid epidemic is the use of multidisciplinary outreach teams which connect an individual in the community to healthcare resources after the experience of an opioid overdose. While these interventions are receiving federal funding, less is known empirically to inform future interventions. Understanding the process and outcomes of these interventions is advisable due to the novel partnerships of public health and law enforcement agencies who sometimes hold divergent goals. The objective of the present review was to describe program structure and evaluated outcomes of community-based post-overdose interventions.

RESULTS

A search of PubMed, PsycInfo, and Web of Science yielded 5 peer-reviewed articles that detail the implementation and outcomes of interventions delivered in the United States published from 2001 to July 2021. Most interventions used a multidisciplinary outreach team and referenced first responder data to contact individuals who recently experienced an overdose at their residence. Services offered often included referral to substance use treatment, recovery coaches, and social services. Method of outreach, evaluation measures, and outcomes varied. From the available literature, facilitators of program engagement included communication, information sharing, and leadership buy-in among multidisciplinary partners.

CONCLUSIONS

Future studies could benefit from exploration of service provision in rural areas, for family affected by overdose, and for minoritized populations. Community-based post-overdose interventions utilizing a law enforcement partnership are emergent with promising yet limited examples in empirical literature.

Implementation of Combined Lifestyle Interventions for Children with Overweight or Obesity: Experiences of Healthcare Professionals in Multiple Communities

BACKGROUND

To counteract children with obesity, different protocols for combined lifestyle interventions (CLIs) are implemented by healthcare providers (HCPs). To understand the effects of CLI, we studied the implementation process, facilitators and barriers experienced by HCPs.

METHODS

A multiple case study design in which community-based CLIs (n = 4), implemented in a total of ten different communities, are conceptualized as a "case". Qualitative data were collected via group interviews among HCPs (n = 48) regarding their implementation protocol, their network involvement and the adoption of the CLI in a community. Transcripts were coded and analysed using ATLAS.ti.

RESULTS

Barriers were the absence of a proper protocol, the low emphasis on the construction of the network and difficulty in embedding the CLI into the community. Funding for these activities was lacking. Facilitating factors were the involvement of a coordinator and to have everyone's role regarding signalling, diagnosis, guidance and treatment clearly defined and protocolled. HCPs suggested adding certain professions to their team because they lacked expertise in parenting advice and providing mental support to children.

CONCLUSIONS

Carrying out and adapting the content of the CLI to the community was experienced as easier compared to the management of the organizational aspects of the CLI. For these aspects, separate funding is essential. In the future, mapping the characteristics of a community will help to clarify this influence on the implementation even better.

Developing a co-production strategy to facilitate the adoption and implementation of evidence-based colorectal cancer screening interventions for rural health systems: a pilot study

BACKGROUND

Evidence-based colorectal cancer screening (CRCS) interventions have not been broadly adopted in rural primary care settings. Co-production of implementation strategies through a bundled approach may be promising in closing this gap by helping rural healthcare practitioners select and implement the best fitting CRCS interventions to the local context. This paper describes the process and outcomes of co-development and delivery of the bundled implementation strategy to improve adoption and implementation of CRCS interventions with two rural clinics.

METHODS

We used a bundle of implementation strategies with a core focus on academic-clinical partnership development (strategy 1) and Plan-Do-Study-Act cycles (strategy 2) to identify clinical partner interests/preferences on delivery methods and content needed to facilitate intervention identification and implementation that improves CRCS. We also developed an implementation blueprint for each clinic (strategy 3) through an online blueprinting process based on adapted "Putting Public Health Evidence in Action" (PPHEA) training curriculum. Clinic physicians and staff (n = 7) were asked to evaluate the bundled approach based on overall reactions and perceptions of innovation characteristics using 5-point Likert scale. After completing the bundled approach, we collected implementation outcomes and limited intervention effectiveness of the CRCS evidence-based interventions (EBIs) developed through the process.

RESULTS

Our co-production strategy yielded a prototype online blueprinting process consisting of 8 distance-learning PPHEA modules that guide selection and implementation of EBIs tailored to CRCS. Modules were delivered to clinic participants with minor adaptations, using PDSA cycle to improve quality of module contents and formats. Overall, participants in both clinics reported positive reactions toward the bundled approach. Both clinics reported improvements in how they perceived the characteristics of the innovation (the bundled approach) to tailor selected CRCS EBIs. As a result of the bundled strategies, each clinic selected and adopted specific EBI(s) with the varying degrees of implementation and CRCS outcomes.

CONCLUSIONS

The bundle of implementation strategies used were feasible and acceptable in rural primary care practices to facilitate the use of EBIs to improve CRCS.

Tracking, naming, specifying, and comparing implementation strategies for person-centred care in a real-world setting: a case study with seven embedded units

BACKGROUND

The implementation of person-centred care (PCC) is advocated worldwide. Stakeholders in charge of implementing PCC as a broad-scale change across the health care sector face two intertwined and complex challenges. First, making sense of PCC as an intervention with complex innovation characteristics and second, staging implementation of PCC by choosing appropriate implementation strategies. We aimed to explore one of these challenges by tracking, naming, specifying, and comparing which strategies and how strategies were enacted to support the implementation of more PCC in a real-world setting represented by one health care region in Sweden.

METHODS

A case study with seven embedded units at two organisational levels within a health care region was conducted from 2016 to 2019. Data were collected from three sources: activity logs, interviews, and written documents. Strategies were identified from all sources and triangulated deductively by name, definition, and cluster in line with the taxonomy Expert Recommendations for Implementing Change (ERIC) and specified according to recommendations by Proctor and colleagues as actor, action, action target, temporality, dose, outcome, and justification.

RESULTS

Four hundred thirteen activities were reported in logs, representing 43 discrete strategies identified in ERIC (n = 38), elsewhere (n = 1), or as emerging strategies (n = 4). The highest reported frequencies of discrete strategies were identified as belonging to two clusters: Train and educate stakeholders (40%) and Develop stakeholder interrelationships (38%). We identified a limited number of strategies belonging to the cluster Use evaluative and iterative strategies (4.6%) and an even smaller number of strategies targeting information to patients about the change initiative (0.8%). Most of the total dose of 11,076 person-hours in the 7 units was spent on strategies targeting health care professionals who provide PCC (81.5%) while the dose of strategies targeting support functions was 18.5%.

CONCLUSIONS

Our findings show both challenges and merits when strategies for implementation of PCC are conducted in a real-world setting. The results can be used to support and guide both scientists and practitioners in future implementation initiatives.

Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar

Background

Implementation science frameworks explore, interpret, and evaluate different components of the implementation process. By using a program logic approach, implementation frameworks with different purposes can be combined to detail complex interactions. The Implementation Research Logic Model (IRLM) facilitates the development of causal pathways and mechanisms that enable implementation. Critical elements of the IRLM vary across different study designs, and its applicability to synthesizing findings across settings is also under-explored. The dual purpose of this study is to develop an IRLM from an implementation research study that used case study methodology and to demonstrate the utility of the IRLM to synthesize findings across case sites.

Method

The method used in the exemplar project and the alignment of the IRLM to case study methodology are described. Cases were purposely selected using replication logic and represent organizations that have embedded exercise in routine care for people with cancer or mental illness. Four data sources were selected: semi-structured interviews with purposely selected staff, organizational document review, observations, and a survey using the Program Sustainability Assessment Tool (PSAT). Framework analysis was used, and an IRLM was produced at each case site. Similar elements within the individual IRLM were identified, extracted, and re-produced to synthesize findings across sites and represent the generalized, cross-case findings.

Results

The IRLM was embedded within multiple stages of the study, including data collection, analysis, and reporting transparency. Between 33-44 determinants and 36-44 implementation strategies were identified at sites that informed individual IRLMs. An example of generalized findings describing “intervention adaptability” demonstrated similarities in determinant detail and mechanisms of implementation strategies across sites. However, different strategies were applied to address similar determinants. Dependent and bi-directional relationships operated along the causal pathway that influenced implementation outcomes.

Conclusions

Case study methods help address implementation research priorities, including developing causal pathways and mechanisms. Embedding the IRLM within the case study approach provided structure and added to the transparency and replicability of the study. Identifying the similar elements across sites helped synthesize findings and give a general explanation of the implementation process. Detailing the methods provides an example for replication that can build generalizable knowledge in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00337-8.

A Modified Implementation Mapping Methodology for Evaluating and Learning From Existing Implementation

When empirically supported interventions are implemented in real-world practice settings, the process of how these interventions are implemented is highly relevant for their potential success. Implementation Mapping is a method that provides step-by-step guidance for systematically designing implementation processes that fit the respective intervention and context. It includes needs assessments among relevant stakeholders, the identification of implementation outcomes and determinants, the selection and design of appropriate implementation strategies, the production of implementation protocols and an implementation outcome evaluation. Implementation Mapping is generally conceptualized as a tool to prospectively guide implementation. However, many implementation efforts build on previous or ongoing implementation efforts, i.e., “existing implementation.” Learnings from existing implementation may offer insights critical to the success of further implementation activities. In this article, we present a modified Implementation Mapping methodology to be applied when evaluating existing implementation. We illustrate the methodology using the example of evaluating ongoing organized colorectal cancer screening programs in Switzerland. Through this example, we describe how we identify relevant stakeholders, implementation determinants and outcomes as well as currently employed implementation strategies. Moreover, we describe how we compare the types of strategies that are part of existing implementation efforts with those that implementation science would suggest as being suited to address identified implementation determinants. The results can be used for assessing the current state of implementation outcomes, refining ongoing implementation strategies, and informing future implementation efforts.

Positive school climate for school staff? The roles of administrators, staff beliefs, and school organization in high and low resource school districts

Despite continued calls for positive school climate, school staff members are rarely included in school climate research or theories. This qualitative case study explored how socioeconomic status (SES), school districts, leadership, and school contexts combine to create school climate. Four schools were selected from a sample of positive and negative climate schools in one high and one low-SES district. Data collection involved key informant interviews and observation. District and school SES did not directly dictate the quality of climate. Organizational variables such as the principal's decisions and vision were key influences on the quality of climate for staff members. Staff members often prioritize student experiences of climate above their own experiences. School organization and the influence of the principal are critical for the quality of school climate staff members experience; these along with staff voice are critical ongoing contributions to school climate research, theory, and practice.

Communication strategies to facilitate the implementation of new clinical practices: a qualitative study of community mental health therapists

Although communication is widely observed to be central to the implementation process, the field of implementation science has largely overlooked the details of how communication may best be utilized to facilitate implementation. This paper draws on relevant insights from Rogers' Diffusion of Innovations Theory, which laid the foundation for explicitly attending to the role of communication as a mechanism for implementation strategies to exert their effects. To offer empirically-derived and theory-informed recommendations regarding communication processes to support the effective introduction of new clinical practices. This investigation leverages data from 61 therapists poised to undergo implementation of measurement-based care (MBC) for depressed adults receiving psychotherapy in community mental health settings. Data were collected via focus groups across 12 sites. Themes emergent in the data analysis suggest five practices to facilitate effective communication in the introduction of new clinical practices like MBC: the communication of a clear rationale for the new practice; the provision of necessary procedural knowledge; communication about the change via multiple methods; sufficient lead time to prepare for the change; and the opportunity for bidirectional engagement. In addition to indicating several best practices to improve communication prior to implementation, our results suggest that the current conceptualization of implementation strategies may not yet be complete. Components and/or methods of effective communication about new practices should be included among the growing set of implementation strategies. Existing implementation strategies might also benefit from more temporal specificity, with more attention to the exploration and preparation phases. (Trial Registration: Clinicaltrials.gov NCT02266134. Registered 12 October 2014.).

“There’s no money in community dissemination”: A mixed methods analysis of researcher dissemination-as-usual

Background:

The field of dissemination and implementation science has the potential to narrow the translational research-to-practice gap and improve the use of evidence-based practices (EBPs) within community-based settings. Yet, foundational research related to dissemination efforts, such as understanding researcher attitudes, practices, and the determinants to sharing research findings, is lacking within extant literature.

Methods:

A sequential explanatory (QUAN $$ \to $$ qual) mixed methods design was used to examine 85 academic researchers’ perspectives and self-reported dissemination methods used to share research outcomes with community stakeholders to better understand researcher’s usual dissemination practices (referred to as dissemination-as-usual). Quantitative surveys collected researcher demographic data, attitudes toward dissemination efforts, and dissemination strategy use.

Results:

Multiple linear regression examined predictors of the quantity of dissemination strategies utilized by researchers, finding that years since earning their degree, time spent disseminating, and the number of reasons for engaging in dissemination efforts predicted greater numbers of dissemination strategies utilized by researchers. Individual, semi-structured interviews with a subset of researchers (n = 18) expanded upon quantitative findings, identifying barriers and facilitators to their dissemination efforts. Data strands were integrated using a joint display, and the Dissemination of Research model guided data interpretation. More established researchers experienced fewer barriers and more facilitators to support their use of a variety of dissemination strategies to share findings with community stakeholders. However, researchers reported needing specific training, institutional support, and/or dedicated time to plan and enact dissemination strategies.

Conclusion:

The necessary first step in research translation is the dissemination of research evidence, and understanding dissemination-as-usual can identify areas of need to advance translational science.

Implementation-as-Usual in Community-Based Organizations Providing Specialized Services to Individuals with Autism Spectrum Disorder: A Mixed Methods Study

Autism spectrum disorder (ASD) is a highly prevalent neurodevelopmental disorder. ASD community-based organizations (ASD-CBOs) underutilize or inconsistently utilize evidence-based practices (ASD-EBPs) despite numerous available EBPs to treat ASD. Nonetheless, ASD-CBOs implement changes to practices regularly. Understanding ASD-CBO's implementation-as-usual (IAU) processes may assist to develop strategies to facilitate ASD-EBP adoption, implementation and sustainment. A convergent mixed methods (quan + QUAL) design was utilized. Twenty ASD-CBO agency leaders (ALs) and 26 direct providers (DPs), from 21 ASD-CBOs, completed the Autism Model of Implementation Survey Battery, including demographic and agency IAU process questions. Surveys were analyzed through descriptive and content analyses. A subset of 10 ALs provided qualitative interview data that were analyzed using coding, consensus and comparison methods to allow for a more comprehensive understanding of the IAU process within their ASD-CBOs. Quantitative analyses and qualitative coding were merged utilizing a joint display and compared. Results suggest that the IAU process follows some phases identified in the Exploration, Preparation, Implementation, Sustainment (EPIS) framework but were conducted in an informal manner-lacking specificity, structure and consistency across and within ASD-CBOs. Moreover, data suggest adding a specific adoption decision phase to the framework. Nonetheless, most ALs felt previous implementation efforts were successful. IAU processes were explored to determine whether the implementation process may be an area for intervention to increase ASD-EBP utilization in ASD-CBOs. Developing a systematized implementation process may facilitate broader utilization of high quality ASD-EBPs within usual care settings, and ultimately improve the quality of life for individuals with ASD and their families.

Examining the implementation of the Icelandic model for primary prevention of substance use in a rural Canadian community: a study protocol

BACKGROUND

The Icelandic Prevention Model (IPM) is a collaborative upstream model that was designed to influence risk and protective factors related to substance use within the community, school, peer and family contexts. By engaging whole communities, the IPM has been found to be effective in reducing youth substance use behaviours across Iceland. As an extension to the IPM's participatory approach, this research will examine how youth involvement can enhance outcomes. In addition, this research will evaluate whether the IPM approach is beneficial for mental health promotion and general youth wellbeing.

METHODS

The present research protocol applies the bioecological model within a participatory mixed-method case study design to examine the implementation of the IPM in a rural community in Canada. This study was designed to identify whether the Icelandic substance use prevention model is effective in reducing substance use and promoting mental health and development for Canadian youth. It will also explore how to engage youth within the approach and how this adaptation influences implementation and outcomes.

DISCUSSION

The findings from this study will contribute to our understanding of upstream prevention of youth substance use and will be used to support scaling of the IPM across Canada.

Implementing Evidence Based Practices for Children's Mental Health: A Case Study in Implementing Modular Treatments in Community Mental Health

There is strong enthusiasm for utilizing implementation science in the implementation of evidence-based programs in children's community mental health, but there remains work to be done to improve the process. Despite the proliferation of implementation frameworks, there is limited literature providing case examples of overcoming implementation barriers. This article examines whether the use of three implementations strategies, a structured training and coaching program, the use of professional development portfolios for coaching, and a progress monitoring data system, help to overcome barriers to implementation by facilitating four implementation drivers at a community mental health agency. Results suggest that implementation is a process of recognizing and adapting to both predictable and unpredictable barriers. Furthermore, the use of these implementation strategies is important in improving implementation outcomes.

Improving the implementation and sustainment of evidence-based practices in community mental health organizations: a study protocol for a matched-pair cluster randomized pilot study of the Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS)

Background

Implementing and sustaining evidence-based programs with fidelity may require multiple implementation strategies tailored to address multi-level, context-specific barriers and facilitators. Ideally, selecting and tailoring implementation strategies should be guided by theory, evidence, and input from relevant stakeholders; however, methods to guide the selection and tailoring of strategies are not well-developed. There is a need for more rigorous methods for assessing and prioritizing implementation determinants (barriers and facilitators) and linking implementation strategies to determinants. The Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS) is an intervention designed to increase the effectiveness of evidence-based practice implementation and sustainment. COAST-IS will enable organizational leaders and clinicians to use Intervention Mapping to select and tailor implementation strategies to address their site-specific needs. Intervention Mapping is a multi-step process that incorporates theory, evidence, and stakeholder perspectives to ensure that implementation strategies effectively address key determinants of change.

Methods

COAST-IS will be piloted with community mental health organizations that are working to address the needs of children and youth who experience trauma-related emotional or behavioral difficulties by engaging in a learning collaborative to implement an evidence-based psychosocial intervention (trauma-focused cognitive behavioral therapy). Organizations will be matched and then randomized to participate in the learning collaborative only (control) or to receive additional support through COAST-IS. The primary aims of this study are to (1) assess the acceptability, appropriateness, feasibility, and perceived utility of COAST-IS; (2) evaluate the organizational stakeholders' fidelity to the core elements of COAST-IS; and (3) demonstrate the feasibility of testing COAST-IS in a larger effectiveness trial.

Discussion

COAST-IS is a systematic method that integrates theory, evidence, and stakeholder perspectives to improve the effectiveness and precision of implementation strategies. If effective, COAST-IS has the potential to improve the implementation and sustainment of a wide range of evidence-based practices in mental health and other sectors.

Trial registration

This study was registered in ClinicalTrials.gov (NCT03799432) on January 10, 2019 (last updated August 5, 2019).

Implementation Mapping: Using Intervention Mapping to Develop Implementation Strategies

Background: The ultimate impact of a health innovation depends not only on its effectiveness but also on its reach in the population and the extent to which it is implemented with high levels of completeness and fidelity. Implementation science has emerged as the potential solution to the failure to translate evidence from research into effective practice and policy evident in many fields. Implementation scientists have developed many frameworks, theories and models, which describe implementation determinants, processes, or outcomes; yet, there is little guidance about how these can inform the development or selection of implementation strategies (methods or techniques used to improve adoption, implementation, sustainment, and scale-up of interventions) (1, 2). To move the implementation science field forward and to provide a practical tool to apply the knowledge in this field, we describe a systematic process for planning or selecting implementation strategies: Implementation Mapping. Methods: Implementation Mapping is based on Intervention Mapping (a six-step protocol that guides the design of multi-level health promotion interventions and implementation strategies) and expands on Intervention Mapping step 5. It includes insights from both the implementation science field and Intervention Mapping. Implementation Mapping involves five tasks: (1) conduct an implementation needs assessment and identify program adopters and implementers; (2) state adoption and implementation outcomes and performance objectives, identify determinants, and create matrices of change objectives; (3) choose theoretical methods (mechanisms of change) and select or design implementation strategies; (4) produce implementation protocols and materials; and (5) evaluate implementation outcomes. The tasks are iterative with the planner circling back to previous steps throughout this process to ensure all adopters and implementers, outcomes, determinants, and objectives are addressed. Discussion: Implementation Mapping provides a systematic process for developing strategies to improve the adoption, implementation, and maintenance of evidence-based interventions in real-world settings.

Study protocol: a pragmatic, stepped-wedge trial of tailored support for implementing social determinants of health documentation/action in community health centers, with realist evaluation

BACKGROUND

National leaders recommend documenting social determinants of health and actions taken to address social determinants of health in electronic health records, and a growing body of evidence suggests the health benefits of doing so. However, little evidence exists to guide implementation of social determinants of health documentation/action.

METHODS

This paper describes a 5-year, mixed-methods, stepped-wedge trial with realist evaluation, designed to test the impact of providing 30 community health centers with step-by-step guidance on implementing electronic health record-based social determinants of health documentation. This guidance will entail 6 months of tailored support from an interdisciplinary team, including training and technical assistance. We will report on tailored support provided at each of five implementation steps; impact of tailored implementation support; a method for tracking such tailoring; and context-specific pathways through which these tailored strategies effect change. We will track the competencies and resources needed to support the study clinics' implementation efforts.

DISCUSSION

Results will inform how to tailor implementation strategies to meet local needs in real-world practice settings. Secondary analyses will assess impacts of social determinants of health documentation and referral-making on diabetes outcomes. By learning whether and how scalable, tailored implementation strategies help community health centers adopt social determinants of health documentation and action, this study will yield timely guidance to primary care providers. We are not aware of previous studies exploring implementation strategies that support adoption of social determinants of action using electronic health and interventions, despite the pressing need for such guidance.

TRIAL REGISTRATION

clinicaltrials.gov, NCT03607617 , registration date: 7/31/2018-retrospectively registered.

Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda

The field of implementation science was developed to better understand the factors that facilitate or impede implementation and generate evidence for implementation strategies. In this article, we briefly review progress in implementation science, and suggest five priorities for enhancing the impact of implementation strategies. Specifically, we suggest the need to: (1) enhance methods for designing and tailoring implementation strategies; (2) specify and test mechanisms of change; (3) conduct more effectiveness research on discrete, multi-faceted, and tailored implementation strategies; (4) increase economic evaluations of implementation strategies; and (5) improve the tracking and reporting of implementation strategies. We believe that pursuing these priorities will advance implementation science by helping us to understand when, where, why, and how implementation strategies improve implementation effectiveness and subsequent health outcomes.

Facilitators and barriers of sociodemographic data collection in Canadian health care settings: a multisite case study evaluation

Background

Despite growing awareness of the importance of social determinants of health, research remains limited about the implementation of sociodemographic data collection in Canadian health care settings. Little is known about the salient contextual factors that enable or hinder collection and use of social information to improve quality of care in clinical settings. This study examines the perceptions and experiences of managers and care providers to better understand how to support organizational efforts to collect and use sociodemographic data to provide equity-oriented care.

Methods

Case studies of three diverse urban health care settings employed semi-structured individual and group interviews with managers and care providers respectively to explore their experiences with implementation. Data was analyzed separately and in context for each site as part of an individual case study. A thematic analysis of interview transcripts was performed with an inductive approach to coding of segments of the text. Constructs of the Consolidated Framework for Implementation Research (CFIR) were used as an analytical framework to structure the data to support cross case comparisons of facilitators and barriers to implementation across settings.

Results

Several perceived facilitators and barriers to implementation were identified that clustered around three CFIR domains: intervention, inner setting and characteristics of individuals. Macro level (outer setting) factors were relatively unexplored. Sites were motivated by their recognition of need for social information to improve quality of care. Organizational readiness for implementation was demonstrated by priorities that reflected concern for equity in care, leadership support and commitment to an inclusive process for stakeholder engagement. Barriers included perceived relevance of only a subset of sociodemographic questions to service delivery, staff capacity and comfort with data collection as well as adequate resources (funding and time).

Conclusion

Although system level mandates were underexplored, they may accelerate adoption and implementation of sociodemographic data collection in the presence of organizational readiness. Standardized tools integrated into information systems and workflows would support adequately trained personnel. More research is needed to understand important factors in rural health settings and with clinical application to inform care delivery pathways.

Electronic supplementary material

The online version of this article (10.1186/s12939-018-0903-0) contains supplementary material, which is available to authorized users.

'Around the edges': using behaviour change techniques to characterise a multilevel implementation strategy for a fall prevention programme

Background

Implementation strategies are needed to ensure that evidence-based healthcare interventions are adopted successfully. However, strategies are generally poorly described and those used in everyday practice are seldom reported formally or fully understood. Characterising the active ingredients of existing strategies is necessary to test and refine implementation. We examined whether an implementation strategy, delivered across multiple settings targeting different stakeholders to support a fall prevention programme, could be characterised using the Behaviour Change Technique (BCT) Taxonomy.

Methods

Data sources included project plans, promotional material, interviews with a purposive sample of stakeholders involved in the strategy’s design and delivery and observations of staff training and information meetings. Data were analysed using TIDieR to describe the strategy and determine the levels at which it operated (organisational, professional, patient). The BCT Taxonomy identified BCTs which were mapped to intervention functions. Data were coded by three researchers and finalised through consensus.

Results

We analysed 22 documents, 6 interviews and 4 observation sessions. Overall, 21 out a possible 93 BCTs were identified across the three levels. At an organisational level, identifiable techniques tended to be broadly defined; the most common BCT was restructuring the social environment. While some activities were intended to encourage implementation, they did not have an immediate behavioural target and could not be coded using BCTs.

The largest number and variety of BCTs were used at the professional level to target the multidisciplinary teams delivering the programme and professionals referring to the programme. The main BCTs targeting the multidisciplinary team were instruction on how to perform the (assessment) behaviour and demonstration of (assessment) behaviour; the main BCT targeting referrers was adding objects to the environment. At the patient level, few BCTs were used to target attendance.

Conclusion

In this study, several behaviour change techniques were evident at the individual professional level; however, fewer techniques were identifiable at an organisational level. The BCT Taxonomy was useful for describing components of a multilevel implementation strategy that specifically target behaviour change. To fully and completely describe an implementation strategy, including components that involve organisational or systems level change, other frameworks may be needed.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0798-6) contains supplementary material, which is available to authorized users.

Fundamentals of Implementation Science: an intensive course on an emerging field of research

Abstract Objective: Report the participation in the Intensive Course on Fundamentals of Implementation Science (IS) in Global Health and present the theoretical scope of IS methods and the potential applicability perspectives for improving global health. Method: Experience report on participation in the course, promoted by the University of Washington, USA, September, 2017. Results: The course introduced IS methodologies and selected case studies, focusing on opportunities and challenges in applying IS in practice. Conclusion: The IS offers tools for the selection of published scientific material and its use through plans, programs, intervention projects, models and protocols that assume diminishing inconsistencies and leverage results on a large scale in health science. Implications for practice: Multidisciplinary teams can benefit from IS training to address in a related way the gaps consistent with progress in the health sciences.

Psychometric assessment of three newly developed implementation outcome measures

BACKGROUND

Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).

METHODS

Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change.

RESULTS

All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (α's from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions.

CONCLUSIONS

The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study

BACKGROUND

Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices).

METHODS

Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables.

RESULTS

Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs.

CONCLUSIONS

This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.

[Family vulnerability index in the municipality of Pasto, Colombia, 2012]

This study's objective was to measure the vulnerability index of families in the municipality of Pasto, Colombia. In a sample of 270 families of all socioeconomic strata (239 urban and 31 rural), a confidential survey was conducted using the vulnerability index, consisting of five dimensions: demographic, social, economic, environmental, and geography, forecasting, and prevention. The families in strata 1 and 2 and the houses in the center, west, northwest, and northeast of the urban area showed high vulnerability, even higher than those in rural areas. Fifty percent of the families showed low vulnerability and 42% medium vulnerability. The poorest families were the most vulnerable, but poor families were also vulnerable in the municipality of Pasto.

Evaluating the implementation of a national disclosure policy for large-scale adverse events in an integrated health care system: identification of gaps and successes

BACKGROUND

Many healthcare organizations have developed disclosure policies for large-scale adverse events, including the Veterans Health Administration (VA). This study evaluated VA's national large-scale disclosure policy and identifies gaps and successes in its implementation.

METHODS

Semi-structured qualitative interviews were conducted with leaders, hospital employees, and patients at nine sites to elicit their perceptions of recent large-scale adverse events notifications and the national disclosure policy. Data were coded using the constructs of the Consolidated Framework for Implementation Research (CFIR).

RESULTS

We conducted 97 interviews. Insights included how to handle the communication of large-scale disclosures through multiple levels of a large healthcare organization and manage ongoing communications about the event with employees. Of the 5 CFIR constructs and 26 sub-constructs assessed, seven were prominent in interviews. Leaders and employees specifically mentioned key problem areas involving 1) networks and communications during disclosure, 2) organizational culture, 3) engagement of external change agents during disclosure, and 4) a need for reflecting on and evaluating the policy implementation and disclosure itself. Patients shared 5) preferences for personal outreach by phone in place of the current use of certified letters. All interviewees discussed 6) issues with execution and 7) costs of the disclosure.

CONCLUSIONS

CFIR analysis reveals key problem areas that need to be addresses during disclosure, including: timely communication patterns throughout the organization, establishing a supportive culture prior to implementation, using patient-approved, effective communications strategies during disclosures; providing follow-up support for employees and patients, and sharing lessons learned.

Study protocol for "Study of Practices Enabling Implementation and Adaptation in the Safety Net (SPREAD-NET)": a pragmatic trial comparing implementation strategies

BACKGROUND

Little research has directly compared the effectiveness of implementation strategies in any setting, and we know of no prior trials directly comparing how effectively different combinations of strategies support implementation in community health centers. This paper outlines the protocol of the Study of Practices Enabling Implementation and Adaptation in the Safety Net (SPREAD-NET), a trial designed to compare the effectiveness of several common strategies for supporting implementation of an intervention and explore contextual factors that impact the strategies' effectiveness in the community health center setting.

METHODS/DESIGN

This cluster-randomized trial compares how three increasingly hands-on implementation strategies support adoption of an evidence-based diabetes quality improvement intervention in 29 community health centers, managed by 12 healthcare organizations. The strategies are as follows: (arm 1) a toolkit, presented in paper and electronic form, which includes a training webinar; (arm 2) toolkit plus in-person training with a focus on practice change and change management strategies; and (arm 3) toolkit, in-person training, plus practice facilitation with on-site visits. We use a mixed methods approach to data collection and analysis: (i) baseline surveys on study clinic characteristics, to explore how these characteristics impact the clinics' ability to implement the tools and the effectiveness of each implementation strategy; (ii) quantitative data on change in rates of guideline-concordant prescribing; and (iii) qualitative data on the "how" and "why" underlying the quantitative results. The outcomes of interest are clinic-level results, categorized using the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework, within an interrupted time-series design with segmented regression models. This pragmatic trial will compare how well each implementation strategy works in "real-world" practices.

DISCUSSION

Having a better understanding of how different strategies support implementation efforts could positively impact the field of implementation science, by comparing practical, generalizable methods for implementing clinical innovations in community health centers. Bridging this gap in the literature is a critical step towards the national long-term goal of effectively disseminating and implementing effective interventions into community health centers.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02325531.

Ethnographic process evaluation in primary care: explaining the complexity of implementation

Background

The recent growth of implementation research in care delivery systems has led to a renewed interest in methodological approaches that deliver not only intervention outcome data but also deep understanding of the complex dynamics underlying the implementation process. We suggest that an ethnographic approach to process evaluation, when informed by and integrated with quantitative data, can provide this nuanced insight into intervention outcomes. The specific methods used in such ethnographic process evaluations are rarely presented in detail; our objective is to stimulate a conversation around the successes and challenges of specific data collection methods in health care settings. We use the example of a translational clinical trial among 11 community clinics in Portland, OR that are implementing an evidence-based, health-information technology (HIT)-based intervention focused on patients with diabetes.

Discussion

Our ethnographic process evaluation employed weekly diaries by clinic-based study employees, observation, informal and formal interviews, document review, surveys, and group discussions to identify barriers and facilitators to implementation success, provide insight into the quantitative study outcomes, and uncover lessons potentially transferable to other implementation projects. These methods captured the depth and breadth of factors contributing to intervention uptake, while minimizing disruption to clinic work and supporting mid-stream shifts in implementation strategies. A major challenge is the amount of dedicated researcher time required.

Summary

The deep understanding of the ‘how’ and ‘why’ behind intervention outcomes that can be gained through an ethnographic approach improves the credibility and transferability of study findings. We encourage others to share their own experiences with ethnography in implementation evaluation and health services research, and to consider adapting the methods and tools described here for their own research.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-014-0607-0) contains supplementary material, which is available to authorized users.

A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions

OBJECTIVE

This systematic review examines experimental studies that test the effectiveness of strategies intended to integrate empirically supported mental health interventions into routine care settings. Our goal was to characterize the state of the literature and to provide direction for future implementation studies.

METHODS

A literature search was conducted using electronic databases and a manual search.

RESULTS

Eleven studies were identified that tested implementation strategies with a randomized (n = 10) or controlled clinical trial design (n = 1). The wide range of clinical interventions, implementation strategies, and outcomes evaluated precluded meta-analysis. However, the majority of studies (n = 7; 64%) found a statistically significant effect in the hypothesized direction for at least one implementation or clinical outcome.

CONCLUSIONS

There is a clear need for more rigorous research on the effectiveness of implementation strategies, and we provide several suggestions that could improve this research area.

Qualitative and mixed methods in mental health services and implementation research

Qualitative and mixed methods play a prominent role in mental health services research. However, the standards for their use are not always evident, especially for those not trained in such methods. This article reviews the rationale and common approaches to using qualitative and mixed methods in mental health services and implementation research based on a review of the articles included in this special series along with representative examples from the literature. Qualitative methods are used to provide a "thick description" or depth of understanding to complement breadth of understanding afforded by quantitative methods, elicit the perspective of those being studied, explore issues that have not been well studied, develop conceptual theories or test hypotheses, or evaluate the process of a phenomenon or intervention. Qualitative methods adhere to many of the same principles of scientific rigor as quantitative methods but often differ with respect to study design, data collection, and data analysis strategies. For instance, participants for qualitative studies are usually sampled purposefully rather than at random and the design usually reflects an iterative process alternating between data collection and analysis. The most common techniques for data collection are individual semistructured interviews, focus groups, document reviews, and participant observation. Strategies for analysis are usually inductive, based on principles of grounded theory or phenomenology. Qualitative methods are also used in combination with quantitative methods in mixed-method designs for convergence, complementarity, expansion, development, and sampling. Rigorously applied qualitative methods offer great potential in contributing to the scientific foundation of mental health services research.