Defining the external implementation context: an integrative systematic literature review

How does the external context affect an implementation processes? A qualitative study investigating the impact of macro-level variables on the implementation of goal-oriented primary care

BACKGROUND

Although the importance of context in implementation science is not disputed, knowledge about the actual impact of external context variables on implementation processes remains rather fragmented. Current frameworks, models, and studies merely describe macro-level barriers and facilitators, without acknowledging their dynamic character and how they impact and steer implementation. Including organizational theories in implementation frameworks could be a way of tackling this problem. In this study, we therefore investigate how organizational theories can contribute to our understanding of the ways in which external context variables shape implementation processes. We use the implementation process of goal-oriented primary care in Belgium as a case.

METHODS

A qualitative study using in-depth semi-structured interviews was conducted with actors from a variety of primary care organizations. Data was collected and analyzed with an iterative approach. We assessed the potential of four organizational theories to enrich our understanding of the impact of external context variables on implementation processes. The organizational theories assessed are as follows: institutional theory, resource dependency theory, network theory, and contingency theory. Data analysis was based on a combination of inductive and deductive thematic analysis techniques using NVivo 12.

RESULTS

Institutional theory helps to understand mechanisms that steer and facilitate the implementation of goal-oriented care through regulatory and policy measures. For example, the Flemish government issued policy for facilitating more integrated, person-centered care by means of newly created institutions, incentives, expectations, and other regulatory factors. The three other organizational theories describe both counteracting or reinforcing mechanisms. The financial system hampers interprofessional collaboration, which is key for GOC. Networks between primary care providers and health and/or social care organizations on the one hand facilitate GOC, while on the other hand, technology to support interprofessional collaboration is lacking. Contingent variables such as the aging population and increasing workload and complexity within primary care create circumstances in which GOC is presented as a possible answer.

CONCLUSIONS

Insights and propositions that derive from organizational theories can be utilized to expand our knowledge on how external context variables affect implementation processes. These insights can be combined with or integrated into existing implementation frameworks and models to increase their explanatory power.

External context in individual placement and support implementation: a scoping review with abductive thematic analysis

BACKGROUND

Context including the external context may considerably affect the adoption, implementation, sustainment, and scale-up of evidence-based practices. We investigated external contextual features by conducting a scoping review of empirical research regarding the implementation of an evidence-based psychiatric or mental health vocational rehabilitation service called Individual Placement and Support (IPS).

METHODS

The protocol for the scoping review was registered with the Open Science Framework. We used the methodology by Joanna Briggs Institute for conducting the scoping review and reported it according to the PRISMA-ScR checklist. We searched 12 databases for research regarding 'Individual Placement and Support' or 'Evidence-Based Supported Employment'. We retained peer-reviewed empirical studies investigating external contextual factors and their impact on IPS implementation outcomes. We extracted data from the eligible articles and conducted descriptive and thematic analyses.

RESULTS

Fifty-nine original research papers met our eligibility requirements and were retained after reviewing 1124 titles and abstracts and 119 full texts. The analysis generated two main themes: (1) external contextual determinants of service delivery and (2) external systems influencing the evidence-to-practice process. The first main theme encompassed policies and laws, financing, and administratively instituted support resources, and organizational arrangements associated with external stakeholders that may facilitate or hinder the local implementation. The second main theme comprised strategies and actions used by different stakeholders to facilitate implementation locally or scale-up efforts at a system level.

DISCUSSION

Our scoping review illustrates the important role that external contextual factors play and how they may facilitate or hinder the implementation and scale-up of the IPS model across mental health services in different countries. Consideration of these factors by decision-makers in mental health and welfare services, planners, providers, and practitioners is likely to facilitate the development of effective strategies for bridging the evidence-practice gap in implementing the EBPs. Finally, the scoping review identified gaps in knowledge and offered suggestions for future research.

TRIAL REGISTRATION

Open Science Framework.

Study protocol for a multi-level cross-sectional study on the equitable reach and implementation of coordinated specialty care for early psychosis

BACKGROUND

Approximately 115,000 young adults will experience their first episode of psychosis (FEP) each year in the USA. Coordinated specialty care (CSC) for early psychosis is an evidence-based early intervention model that has demonstrated effectiveness by improving quality of life and reducing psychiatric symptoms for many individuals. Over the last decade, there has significant increase in the implementation of CSC programs throughout the USA. However, prior research has revealed difficulties among individuals and their family members accessing CSC. Research has also shown that CSC programs often report the limited reach of their program to underserved populations and communities (e.g., ethnoracial minorities, rural and low socioeconomic neighborhoods). Dissemination and implementation research focused on the equitable reach and implementation of CSC is needed to address disparities at the individual level.

METHODS

The proposed study will create a novel integrative multi-level geospatial database of CSC programs implemented throughout the USA that will include program-level data (e.g., geocoded location, capacity, setting, role availability), provider-level data (race, ethnicity, professional credentials), and neighborhood-level census data (e.g., residential segregation, ethnic density, area deprivation, rural-urban continua, public transit time). This database will be used to characterize variations in CSC programs by geographical location and examine the overall reach CSC programs to specific communities. The quantitative data will be combined with qualitative data from state administrators, providers, and service users that will inform the development of dissemination tools, such as an interactive dashboard, that can aid decision making.

DISCUSSION

Findings from this study will highlight the impact of outer contextual determinants on implementation and reach of mental health services, and will serve to inform the future implementation of CSC programs with a primary focus on equity.

Local implementation of public health policies revealed by the COVID-19 crisis: the French case

BACKGROUND

Improving health system performance depends on the quality of health policy implementation at the local level. However, in general, the attention of researchers is mainly directed towards issues of health policy design and evaluation rather than implementation at the local level. The management of the COVID-19 crisis, especially in Europe, has particularly highlighted the complexity of implementing health policies, decided at the national or supranational level, at the local level.

METHODS

We conducted 23 semi-structured interviews with the main stakeholders in the management of the COVID-19 crisis in the second largest French region in order to identify the different actors and modes of coordination of the local implementation of health policies that this crisis management illustrates in a very visible way. Our methodology is complemented by a content analysis of the main guidelines and decisions related to this implementation.

RESULTS

The analysis of these data allows us to identify three levels of implementation of health policies at the local level (administrative, organizational and operational). Interviews also reveal the existence of different types of coordination specific to each of these levels of local implementation of health policies. These results then make it possible to identify important managerial avenues for promoting global coordination of these three levels of implementation.

CONCLUSIONS

Although research on health services emphasizes the existence of several levels of local implementation of health policies, it offers little in the way of definition or characterization of these levels. The identification in this study of the three levels of local implementation of health policies and their specific forms of coordination contribute to a more precise characterization of this implementation in order to promote, in practice, its global coordination.

Chronic Disease in Low-Resource Settings: Prevention and Management Throughout the Continuum of Care-A Call for Papers

Multimorbidity, defined as the presence of two or more chronic conditions in an individual, has become a global public health challenge [...].

Strategies to Implement Pet Robots in Long-Term Care Facilities for Dementia Care: A Modified Delphi Study

OBJECTIVES

Pet robots are technology-based substitutes for live animals that have demonstrated psychosocial benefits for people living with dementia in long-term care. However, little research has been conducted to understand how pet robots should be implemented in routine care. This study aims to identify, contextualize, and achieve expert consensus on strategies to implement pet robots as part of dementia care in long-term care facilities.

DESIGN

A 2-round modified Delphi study.

SETTINGS AND PARTICIPANTS

An international panel of 56 experts from 14 countries, involving care professionals, organizational leaders, and researchers.

METHODS

A list of potentially relevant strategies was identified, contextualized, and revised using empirical data and through stakeholder consultations. These strategies constituted statements for Round 1. Experts rated the relative importance of each statement on a 9-point scale, and free-text fields allowed them to provide justifications. Consensus was predefined as ≥75% agreement. Statements not reaching an agreement were brought forward to Round 2. Quantitative data were analyzed using descriptive statistics, and textual data were analyzed using inductive content analysis.

RESULTS

Thirteen strategies reached consensus; 11 were established as critical: (1) assess readiness and identify barriers and facilitators, (2) purposely reexamine the implementation, (3) obtain and use residents' and their family's feedback, (4) involve residents and their family, (5) promote adaptability, (6) conduct ongoing training, (7) conduct educational meetings, (8) conduct local consensus discussions, (9) organize clinician implementation team meetings, (10) provide local technical assistance, and (11) access new funding. Other strategies received differing extents of agreement. Reasons for variations included contextual differences, such as resource availability, organizational structures, and staff turnover.

CONCLUSIONS AND IMPLICATIONS

This study identified the most relevant strategies that can be used by technology developers, care providers, and researchers to implement pet robots in long-term care facilities for dementia care. Further development, specification, and testing in real-world settings are needed.

Methodological approaches to study context in intervention implementation studies: an evidence gap map

BACKGROUND

Within implementation science studies, contextual analysis is increasingly recognized as foundational to interventions' successful and sustainable implementation. However, inconsistencies between methodological approaches currently limit progress in studying context and guidance to standardize the use of those approaches is scant. Therefore, this study's objective was to systematically review and map current methodological approaches to contextual analysis in intervention implementation studies. The results would help us both to systematize the process of contextual analysis and identify gaps in the current evidence.

METHODS

We conducted an evidence gap map (EGM) based on literature data via a stepwise approach. First, using an empirically developed search string, we randomly sampled 20% of all intervention implementation studies available from PubMed per year (2015-2020). Second, we assessed included studies that conducted a contextual analysis. Data extraction and evaluation followed the Basel Approach for CoNtextual ANAlysis (BANANA), using a color-coded rating scheme. Also based on BANANA and on the Context and Implementation of Complex Interventions (CICI) framework-an implementation framework that pays ample attention to context- we created visual maps of various approaches to contextual analysis.

RESULTS

Of 15, 286 identified intervention implementation studies and study protocols, 3017 were screened for inclusion. Of those, 110 warranted close examination, revealing 22% that reported on contextual analysis. Only one study explicitly applied a framework for contextual analysis. Data were most commonly collected via surveys (n = 15) and individual interviews (n = 13). Ten studies reported mixed-methods analyses. Twenty-two assessed meso-level contextual and setting factors, with socio-cultural aspects most commonly studied. Eighteen described the use of contextual information for subsequent project phases (e.g., intervention development/adaption, selecting implementation strategies). Nine reported contextual factors' influences on implementation and/or effectiveness outcomes.

CONCLUSIONS

This study describes current approaches to contextual analysis in implementation science and provides a novel framework for evaluating and mapping it. By synthesizing our findings graphically in figures, we provide an initial evidence base framework that can incorporate new findings as necessary. We strongly recommend further development of methodological approaches both to conduct contextual analysis and to systematize the reporting of it. These actions will increase the quality and consistency of implementation science research.

Conceptual and relational advances of the PARIHS and i-PARIHS frameworks over the last decade: a critical interpretive synthesis

BACKGROUND

The number of research publications reporting the use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework and the integrated PARIHS (i-PARIHS) framework has grown steadily. We asked how the last decade of implementation research, predicated on the (i-)PARIHS framework (referring to the PARIHS or i-PARIHS framework), has contributed to our understanding of the conceptualizations of, relationships between, and dynamics among the core framework elements/sub-elements. Building on the Helfrich et al. (2010) review of research on the PARIHS framework, we undertook a critical interpretive synthesis to: (1) identify conceptual and relational advances in the (i-)PARIHS framework and (2) identify conceptual and relational aspects of the (i-)PARIHS framework that warrant further work.

METHODS

We performed a systematic search in PubMed/PubMed Central, Ovid MEDLINE, CINAHL, JSTOR, SCOPUS, Web of Science, and PsycInfo. Articles were eligible for synthesis if they (a) were peer-reviewed articles, written in English, and published between January 2009 and December 2021, (b) applied the (i-)PARIHS framework explicitly to guide implementation research, and (c) made conceptual (expanding the conceptualization of core elements) and/or relational contributions (elaborating relationships among elements/sub-elements, or theorizing the relationships using empirical data). We used a critical interpretive synthesis approach to synthesize conceptual-relational advances of the (i-)PARIHS framework.

RESULTS

Thirty-seven articles were eligible for synthesis. Twenty-four offered conceptual contributions, and 18 offered relational contributions (5 articles contributed in both ways). We found conceptual expansion of all core (i-)PARIHS elements, with most emphasis on context (particularly outer context and leadership), facilitation, and implementation success. Articles also gave insights into the complex relationships and relational dynamism among these elements, characterized as contingent, interactive, multilevel, and temporal effects.

CONCLUSIONS

We observed developmental advances of the (i-)PARIHS framework and proposed several directions to further advance the framework. Conceptualization of (i-)PARIHS elements (particularly evidence/innovation and recipients) need to be further developed by specifying conceptual and operational definitions of underlying sub-elements. Relationships among (i-)PARIHS elements/sub-elements need to be further elaborated through empirical studies that consider situational contingencies and causal complexities. This will require examining necessity and sufficiency of (i-)PARIHS elements/sub-elements in relation to implementation outcomes, interactions among elements, and mechanism-based explanations.

Too much theory and not enough practice? The challenge of implementation science application in healthcare practice

BACKGROUND

Implementation science (IS) should contribute to maintaining high standards of care across healthcare systems and enhancing care practices. However, despite the evident need for greater and more rapid uptake and integration of evidence in practice, IS design and methodology fall short of the needs of effective translation.

AIM

In this paper we examine what it is about IS that makes it so appealing for effective uptake of interventions in routine practice, and yet so difficult to achieve. We propose a number of ways that implementation scientists could build mutual relationships with healthcare practitioners and other stakeholders including public members to ensure greater shared care practices, and highlight the value of IS training, collaborative educational events, and co-designed research.

DISCUSSION

More consideration should be given to IS applications in healthcare contexts. Implementation scientists can make a valuable contribution by mobilizing theory and improving practice. However, goals for an evidence-based system may be more appropriately achieved through greater outreach and collaboration, with methods that are flexible to support rapid implementation in complex adaptive systems. Collective learning and mutual trust can be cultivated by embedding researchers into healthcare services while offering greater opportunities for practitioners to learn about, and engage in, implementation research.

CONCLUSION

To bridge the worlds of healthcare practice and IS, researchers could be more consistent in the relationships they build with professionals and the public, communicating through a shared language and co-joining practical approaches to effective implementation. This will build capacity for improved collaboration and foster respectful, interdisciplinary relationships.

'A priori' external contextual factors and relationships with process indicators: a mixed methods study of the pre-implementation phase of 'Communities in Charge of Alcohol'

BACKGROUND

It is widely recognised that complex public health interventions roll out in distinct phases, within which external contextual factors influence implementation. Less is known about relationships with external contextual factors identified a priori in the pre-implementation phase. We investigated which external contextual factors, prior to the implementation of a community-centred approach to reducing alcohol harm called 'Communities in Charge of Alcohol' (CICA), were related to one of the process indicators: numbers of Alcohol Health Champions (AHCs) trained.

METHODS

A mixed methods design was used in the pre-implementation phase of CICA. We studied ten geographic communities experiencing both high levels of deprivation and alcohol-related harm in the North West of England. Qualitative secondary data were extracted from pre-implementation meeting notes, recorded two to three months before roll-out. Items were coded into 12 content categories using content analysis. To create a baseline 'infrastructure score', the number of external contextual factors documented was counted per area to a maximum score of 12. Descriptive data were collected from training registers detailing training numbers in the first 12 months. The relationship between the baseline infrastructure score, external contextual factors, and the number of AHCs trained was assessed using non-parametric univariable statistics.

RESULTS

There was a positive correlation between baseline infrastructure score and total numbers of AHCs trained (Rs = 0.77, p = 0.01). Four external contextual factors were associated with significantly higher numbers of lay people recruited and trained: having a health care provider to coordinate the intervention (p = 0.02); a pool of other volunteers to recruit from (p = 0.02); a contract in place with a commissioned service (p = 0.02), and; formal volunteer arrangements (p = 0.03).

CONCLUSIONS

Data suggest that there were four key components that significantly influenced establishing an Alcohol Health Champion programme in areas experiencing both high levels of deprivation and alcohol-related harm. There is added value of capturing external contextual factors a priori and then testing relationships with process indicators to inform the effective roll-out of complex interventions. Future research could explore a wider range of process indicators and outcomes, incorporating methods to rate individual factors to derive a mean score.

TRIAL REGISTRATION

ISRCTN81942890, date of registration 12/09/2017.

Unraveling implementation context: the Basel Approach for coNtextual ANAlysis (BANANA) in implementation science and its application in the SMILe project

BACKGROUND

Designing intervention and implementation strategies with careful consideration of context is essential for successful implementation science projects. Although the importance of context has been emphasized and methodology for its analysis is emerging, researchers have little guidance on how to plan, perform, and report contextual analysis. Therefore, our aim was to describe the Basel Approach for coNtextual ANAlysis (BANANA) and to demonstrate its application on an ongoing multi-site, multiphase implementation science project to develop/adapt, implement, and evaluate an integrated care model in allogeneic SteM cell transplantatIon facILitated by eHealth (the SMILe project).

METHODS

BANANA builds on guidance for assessing context by Stange and Glasgow (Contextual factors: the importance of considering and reporting on context in research on the patient-centered medical home, 2013). Based on a literature review, BANANA was developed in ten discussion sessions with implementation science experts and a medical anthropologist to guide the SMILe project's contextual analysis. BANANA's theoretical basis is the Context and Implementation of Complex Interventions (CICI) framework. Working from an ecological perspective, CICI acknowledges contextual dynamics and distinguishes between context and setting (the implementation's physical location).

RESULTS

BANANA entails six components: (1) choose a theory, model, or framework (TMF) to guide the contextual analysis; (2) use empirical evidence derived from primary and/or secondary data to identify relevant contextual factors; (3) involve stakeholders throughout contextual analysis; (4) choose a study design to assess context; (5) determine contextual factors' relevance to implementation strategies/outcomes and intervention co-design; and (6) report findings of contextual analysis following appropriate reporting guidelines. Partly run simultaneously, the first three components form a basis both for the identification of relevant contextual factors and for the next components of the BANANA approach.

DISCUSSION

Understanding of context is indispensable for a successful implementation science project. BANANA provides much-needed methodological guidance for contextual analysis. In subsequent phases, it helps researchers apply the results to intervention development/adaption and choices of contextually tailored implementation strategies. For future implementation science projects, BANANA's principles will guide researchers first to gather relevant information on their target context, then to inform all subsequent phases of their implementation science project to strengthen every part of their work and fulfill their implementation goals.

A citation analysis and scoping systematic review of the operationalization of the Practical, Robust Implementation and Sustainability Model (PRISM)

Background

The Practical, Robust Implementation and Sustainability Model (PRISM) was developed in 2008 as a contextually expanded version of the broadly used Reach, Adoption, Effectiveness, Implementation, and Maintenance (RE-AIM) framework. PRISM provides researchers a pragmatic and intuitive model to improve translation of research interventions into clinical and community practice. Since 2008, the use of PRISM increased across diverse topics, populations, and settings. This citation analysis and scoping systematic review aimed to assess the use of the PRISM framework and to make recommendations for future research.

Methods

A literature search was conducted using three databases (PubMed, Web of Science, Scopus) for the period of 2008 and September 2020. After exclusion, reverse citation searches and invitations to experts in the field were used to identify and obtain recommendations for additional articles not identified in the original search. Studies that integrated PRISM into their study design were selected for full abstraction. Unique research studies were abstracted for information on study characteristics (e.g., setting/population, design), PRISM contextual domains, and RE-AIM outcomes.

Results

A total of 180 articles were identified to include PRISM to some degree. Thirty-two articles representing 23 unique studies integrated PRISM within their study design. Study characteristics varied widely and included studies conducted in diverse contexts, but predominately in high-income countries and in clinical out-patient settings. With regards to use, 19 used PRISM for evaluation, 10 for planning/development, 10 for implementation, four for sustainment, and one for dissemination. There was substantial variation across studies in how and to what degree PRISM contextual domains and RE-AIM outcomes were operationalized and connected. Only two studies directly connected individual PRISM context domains with RE-AIM outcomes, and another four included RE-AIM outcomes without direct connection to PRISM domains.

Conclusions

This is the first systematic review of the use of PRISM in various contexts. While there were low levels of ‘integrated’ use of PRISM and few reports on linkage to RE-AIM outcomes, most studies included important context domains of implementation and sustainability infrastructure and external environment. Recommendations are provided for more consistent and comprehensive use of and reporting on PRISM to inform both research and practice on contextual factors in implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01234-3.

The time is ripe for robopsychology

As robotic applications become increasingly diverse, more domains of human lives are being involved, now also extending to educational, therapeutic, and social situations, with a trend to even more complex interactions. This diversity generates new research questions that need to be met with an adequate infrastructure of psychological methods and theory. In this review, we illustrate the current lack of a sub-discipline in psychology to systematically study the psychological corollaries of living in societies where the application of robotic and artificial intelligence (AI) technologies is becoming increasingly common. We thus propose that organized efforts be made toward recognition of robopsychology as a sub-discipline so that the field of psychology moves away from isolated publications of robot- and AI-related topics to a body of knowledge that is able to meet the demands for change, as the world is preparing for the Fourth Industrial Revolution. We propose a definition of robopsychology that not only covers the study of the effects of robots on human behavior, but also of robots and AI themselves, as well as acknowledging how this sub-discipline may eventually be fundamentally changed through robots and AI. In this sense, our definition mirrors an already existing definition of the field of robophilosophy.

A mixed methods analysis evaluating an alcohol health champion community intervention: How do newly trained champions perceive and understand their training and role?

Globally, alcohol harm is recognised as one of the greatest population risks and reducing alcohol harm is a key priority for the UK Government. The Communities in Charge of Alcohol (CICA) programme took an asset-based approach in training community members across nine areas to become alcohol health champions (AHCs); trained in how to have informal conversations about alcohol and get involved with alcohol licensing. This paper reports on the experiences of AHCs taking part in the training through the analysis of: questionnaires completed pre- and post-training (n = 93) and semi-structured interviews with a purposive sample of five AHCs who had started their role. Questionnaires explored: characteristics of AHCs, perceived importance of community action around alcohol and health, and confidence in undertaking their role. Following training AHCs felt more confident to talk about alcohol harms, give brief advice and get involved in licensing decisions. Interviews explored: AHCs' experiences of the training, barriers and facilitators to the adoption of their role, and how they made sense of their role. Four overarching themes were identified through thematic analysis taking a framework approach: (a) perceptions of AHC training; (b) applying knowledge and skills in the AHC role; (c) barriers and facilitators to undertaking the AHC role; and (d) sustaining the AHC role. Findings highlight the challenges in establishing AHC roles can be overcome by combining the motivation of volunteers with environmental assets in a community setting: the most important personal asset being the confidence to have conversations with people about a sensitive topic, such as alcohol.

Integrating home treatment into the Austrian child and adolescent mental health care system: an implementation science perspective

BACKGROUND

In Austria, similar to other countries, health care planners have considered implementing home treatment (HT) programs for children and adolescents with mental health problems. However, introducing any new service innovation into the complex environment of the child and adolescent mental health care system requires context and implementation issues to be taken into account in advance.

METHODS

We identified implementation themes from international HT models and developed a questionnaire with open-ended implementation-related questions for Austrian professionals. We analysed the qualitative results alongside the domains in the Context and Implementation of Complex Interventions (CICI) framework to detect key implementation challenges for HT in Austria.

RESULTS

We found six implementation themes in international HT models, which formed the basis for development of the questionnaire. Within the qualitative data from ten Austrian professionals who responded to the questionnaire, we identified implementation challenges in all eight context and all four implementation domains of the CICI framework.

CONCLUSIONS

Key implementation challenges detected highlight the need to address the fit of HT within existing sociocultural norms and limited available human resources. Regional context-specific solutions need to be found on how to embed this new element best into the existing provider and financing structures. This calls for monitoring and evaluation alongside implementation to support decision-making.

Evaluation of a Large-Scale School Wellness Intervention Through the Consolidated Framework for Implementation Research (CFIR): Implications for Dissemination and Sustainability

Background

Numerous studies have tested school-based interventions promoting healthy behaviors in youth, but few have integrated dissemination and implementation (D&I) frameworks. Using D&I frameworks can inform if and how an evidence-based intervention is implemented and maintained and provide strategies to address contextual barriers. Such application is necessary to understand how and why interventions are sustained over time. We evaluated a school wellness initiative called SWITCH® (School Wellness Integration Targeting Child Health) to (1) assess implementation outcomes of adoption, fidelity, and penetration, (2) discern implementation determinants through the Consolidated Framework for Implementation Research (CFIR), and (3) examine differences among inexperienced and experienced schools and influential factors to sustainment.

Methods

A total of 52 schools from Iowa, United States enrolled in the 2019-2020 iteration of SWITCH (22 inexperienced; 30 experienced). The CFIR guided the adaptation of mixed methods data collection and analysis protocols for school settings. Specific attention was focused on (1) fidelity to core elements; (2) adoption of best practices; and (3) penetration of behavior change practices. Determinants were investigated through in-depth qualitative interviews and readiness surveys with implementation leaders. A systematic process was used to score CFIR domains (between -2 and +2) indicating positive or negative influence. Independent t-tests were conducted to capture differences between samples, followed by a cross-case analysis to compare determinants data. Inductive coding yielded themes related to sustainment of SWITCH beyond formal implementation support.

Results

Experienced schools had higher scores on fidelity/compliance (t = -1.86 p = 0.07) and adoption (t = -2.03 p = 0.04). CFIR determinants of innovation source, culture, relative priority, and leadership engagement were positive implementation determinants, whereas tension for change and networks and communications were negative determinants. Distinguishing factors between experienced and inexperienced schools were Readiness for Implementation and Self-efficacy (experienced significantly higher; p < 0.05). Strategies to enhance sustainability were increasing student awareness/advocacy, keeping it simple, and integrating into school culture.

Conclusions

Findings provide specific insights related to SWITCH implementation and sustainability but more generalized insights about the type of support needed to help schools implement and sustain school wellness programming. Tailoring implementation support to both inexperienced and experienced settings will ultimately enhance dissemination and sustainability of evidence-based interventions.

Context-specific adaptation of an eHealth-facilitated, integrated care model and tailoring its implementation strategies-A mixed-methods study as a part of the SMILe implementation science project

Background

Contextually adapting complex interventions and tailoring their implementation strategies is key to a successful and sustainable implementation. While reporting guidelines for adaptations and tailoring exist, less is known about how to conduct context-specific adaptations of complex health care interventions.

Aims

To describe in methodological terms how the merging of contextual analysis results (step 1) with stakeholder involvement, and considering overarching regulations (step 2) informed our adaptation of an Integrated Care Model (ICM) for SteM cell transplantatIon faciLitated by eHealth (SMILe) and the tailoring of its implementation strategies (step 3).

Methods

Step 1: We used a mixed-methods design at University Hospital Basel, guided by the Basel Approach for coNtextual ANAlysis (BANANA). Step 2: Adaptations of the SMILe-ICM and tailoring of implementation strategies were discussed with an interdisciplinary team (n = 28) by considering setting specific and higher-level regulatory scenarios. Usability tests were conducted with patients (n = 5) and clinicians (n = 4). Step 3: Adaptations were conducted by merging our results from steps 1 and 2 using the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME). We tailored implementation strategies according to the Expert Recommendations for Implementing Change (ERIC) compilation.

Results

Step 1: Current clinical practice was mostly acute-care-driven. Patients and clinicians valued eHealth-facilitated ICMs to support trustful patient-clinician relationships and the fitting of eHealth components to context-specific needs. Step 2: Based on information from project group meetings, adaptations were necessary on the organizational level (e.g., delivery of self-management information). Regulations informed the tailoring of SMILe-ICM`s visit timepoints and content; data protection management was adapted following Swiss regulations; and steering group meetings supported infrastructure access. The usability tests informed further adaptation of technology components. Step 3: Following FRAME and ERIC, SMILe-ICM and its implementation strategies were contextually adapted and tailored to setting-specific needs.

Discussion

This study provides a context-driven methodological approach on how to conduct intervention adaptation including the tailoring of its implementation strategies. The revealed meso-, and macro-level differences of the contextual analysis suggest a more targeted approach to enable an in-depth adaptation process. A theory-guided adaptation phase is an important first step and should be sufficiently incorporated and budgeted in implementation science projects.

Identifying unique barriers to implementing rural emergency department-based peer services for opioid use disorder through qualitative comparison with urban sites

BACKGROUND

In an effort to address the current opioid epidemic, a number of hospitals across the United States have implemented emergency department-based interventions for engaging patients presenting with opioid use disorder. The current study seeks to address gaps in knowledge regarding implementation of a sub-type of such interventions, emergency department-based peer support services, in rural areas by comparing implementation of rural and urban programs that participated in Indiana's Recovery Coach and Peer Support Initiative (RCPSI).

METHODS

We conducted a secondary analysis of qualitative semi-structured implementation interviews collected as part of an evaluation of 10 programs (4 rural and 6 urban) participating in the RCPSI. We conducted interviews with representatives from each program at 3 time points over the course of the first year of implementation. Our deductive coding process was guided by the Consolidated Framework for Implementation Research (CFIR) and an external context taxonomy.

RESULTS

We identified key differences for rural programs corresponding to each of the 5 primary constructs in the coding scheme. (1) Intervention characteristics: rural sites questioned intervention fit with their context, required more adaptations, and encountered unexpected costs. (2) External context: rural sites were not appropriately staffed to meet patient needs, encountered logistical and legal barriers regarding patient privacy, and had limited patient transportation options. (3) Inner setting: rural sites lacked strong mechanisms for internal communication and difficulties integrating with pre-existing culture and climate. (4) Characteristics of individuals: some rural providers resisted working with peers due to pre-existing attitudes and beliefs. (5) Implementation process: rural sites spent more time identifying external partners and abandoned more components of their initial implementation plans.

CONCLUSIONS

Findings demonstrate how rural programs faced greater challenges implementing emergency department-based peer services over time. These challenges required flexible adaptations to originally intended plans. Rural programs likely require flexibility to adapt interventions that were developed in urban settings to ensure success considering local contextual constraints that were identified by our analysis.

Quantitative measures used in empirical evaluations of mental health policy implementation: A systematic review

Background

Mental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures.

Method

Guided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995-2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale.

Results

We identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n  =  20, 57%) and other resources (n  =  12, 34%), actor relationships/networks (n  =  15, 43%), and organizational culture and climate (n  =  11, 31%). Fidelity was the most prevalent implementation outcome (n  =  9, 26%), followed by penetration (n  =  8, 23%) and acceptability (n  =  7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools.

Conclusions

This work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap.

Plain Language Summary

Mental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.

Implementation of mandatory opioid prescribing limits in North Carolina: healthcare administrator and prescriber perspectives

BACKGROUND

Recent increases in state laws to reduce opioid prescribing have demonstrated a need to understand how they are interpreted and implemented in healthcare systems. The purpose of this study was to explore the systems, strategies, and resources that hospital administrators and prescribers used to implement the 2017 North Carolina Strengthen Opioid Prevention (STOP) Act opioid prescribing limits, which limited initial prescriptions to a five (for acute) or seven (for post-surgical) days' supply.

METHODS

We interviewed 14 hospital administrators and 38 prescribers with degrees in medicine, nursing, pharmacy, business administration and public health working across North Carolina. Interview guides, informed by the Consolidated Framework for Implementation Research, explored barriers and facilitators to implementation. Interview topics included communication, resources, and hospital system support. Interviews were recorded and transcribed, then analyzed using flexible coding, integrating inductive and deductive coding, to inform analytic code development and identify themes.

RESULTS

We identified three main themes around implementation of STOP act mandated prescribing limits: organizational communication, prescriber education, and changes in the electronic medical record (EMR) systems. Administrators reflected on implementation in the context of raising awareness and providing reminders to facilitate changes in prescriber behavior, operationalized through email and in-person communications as well as dedicated resources to EMR changes. Prescribers noted administrative communications about prescribing limits often focused on legality, suggesting a directive of the organization's policy rather than a passive reminder. Prescribers expressed a desire for more spaces to have their questions answered and resources for patient communications. While hospital administrators viewed compliance with the law as a priority, prescribers reflected on concerns for adequately managing their patients' pain and limited time for clinical care.

CONCLUSIONS

Hospital administrators and prescribers approached implementation of the STOP act prescribing limits with different mindsets. While administrators were focused on policy compliance, prescribers were focused on their patients' needs. Strategies to implement the mandate then had to balance patient needs with policy compliance. As states continue to legislate to prevent opioid overdose deaths, understanding how laws are implemented by healthcare systems and prescribers will improve their effectiveness through tailoring and maximizing available resources.

Influence of commissioning arrangements on implementing and sustaining a complex healthcare intervention (ESCAPE-pain) for osteoarthritis: a qualitative case study

OBJECTIVES

Funding in health care has a critical impact on the implementation and sustainability of evidence-based interventions. This study explored the perspectives of physiotherapists on the influence of commissioning arrangements on the implementation and sustainability of a group rehabilitation programme for osteoarthritis (ESCAPE-pain).

DESIGN

A qualitative case study approach using in-depth interviews.

SETTING

National Health Service (NHS) musculoskeletal (MSK) outpatient departments in England.

PARTICIPANTS

Thirty physiotherapists in clinical and senior management roles from 11 NHS MSK providers.

RESULTS

Five themes were identified: (1) clinical perspectives of ESCAPE-pain - MSK services wanted to implement and sustain ESCAPE-pain because it provided evidence-based, quality care; (2) focusing on clinical activity over outcomes - commissioners were perceived as prioritising activity-based performance over delivering clinical outcomes; (3) rationing availability - patient access to ESCAPE-pain could be limited due to rationing resources; (4) absorbing costs - contracts did not always cover the activities associated with delivering ESCAPE-pain meaning that providers bore the costs; and (5) relationship between commissioners and providers - physiotherapists perceived a disconnect with commissioners and had little power to influence decisions.

CONCLUSIONS

Commissioning arrangements for MSK physiotherapy services can impede providers from implementing and sustaining a clinically and cost-effective intervention. To be implemented and sustained, an intervention needs to integrate into clinical practice and the wider healthcare system. Commissioning arrangements for MSK physiotherapy need to allow providers the flexibility to deliver interventions that best meet the needs of their patients. The move to more strategic, integrated, outcome-based commissioning has the potential to facilitate the spread and sustainability of interventions.

How can communities influence alcohol licensing at a local level? Licensing officers' perspectives of the barriers and facilitators to sustaining engagement in a volunteer-led alcohol harm reduction approach

Despite the World Health Organization's assertion that communities need to become involved in reducing alcohol harm, evidence of community engagement in alcohol licensing decision-making in England remains limited. The evaluation of the Communities in Charge of Alcohol (CICA) programme offers policymakers, Licensing authorities and public health practitioners, evidence regarding a specific volunteer-led, place-based approach, designed to enable community engagement in licensing with the aim of reducing localised alcohol harm. This study explored factors affecting the sustainable involvement of volunteers in alcohol licensing decision-making from six licensing officers' perspectives, through semi-structured interviews. Routinely collected crime, disorder, and hospital admissions data were reviewed for further context as proxy indicators for alcohol-related harm. Licensing officers perceived sustainable engagement to be impacted by: (i) the extent of alignment with statutory requirements and local political support; (ii) the ability of licensing officers to operationalise CICA and support local assets, and; (iii) the opportunity for, and ability of, volunteers to raise licensing issues. The perspectives of licensing officers indicate complexities inherent in seeking to empower residents to engage in licensing decision-making at a community level. These relate to statutory and political factors, funding, social norms regarding engagement in licensing decision-making, and the need for networks between critical actors including responsible authorities and communities. The evidence indicates that after increasing community capacity to influence alcohol availability decision-making at a local level, communities continue to struggle to influence statutory processes to affect alcohol availability where they live and work. More understanding of how to enable effective community engagement is required.

Sustainability of a biobehavioral intervention implemented by therapists and sustainment in community settings

The ultimate aim of dissemination and implementation of empirically supported treatments (ESTs) in behavioral medicine is (a) sustainability of the therapist/provider's EST usage and (b) sustainment of EST delivery in the setting. Thus far, sustainability has been understudied, and the therapist and setting variables that may be influential are unclear. The purpose of the study was to test the therapists' sustainability of a cancer-specific EST using a prospective longitudinal design and examine its predictors. Oncology mental health therapists (N = 134) from diverse settings (N = 110) completed training in the biobehavioral intervention (BBI) and were provided with 6 months of support for implementation, with no support thereafter. BBI usage (percent of patients treated) was reported at 2, 4, 6, and 12 months. Using a generalized estimating equation with a logistic link function, 12-month sustainability (a nonsignificant change in usage from 6 to 12 months) was studied along with therapist, supervisor, and setting variables as predictors. BBI usage increased through 6 months and, importantly, usage was sustained from 6 (68.4% [95% CI = 62.2%-73.9%]) to 12 months (70.9% [95% CI = 63.6%-77.3%]), with sustainment in 66 settings (60.0%). Predictors of implementation-to-sustainability usage were therapists' early intentions to use the BBI (p < .001) and from the setting, supervisors' positive attitudes toward ESTs (p = .016). Adding to the DI literature, a health psychology intervention was disseminated, implemented, and found sustainable across diverse therapists and settings. Therapists and setting predictors of usage, if modified, might facilitate future sustainability/sustainment of ESTs.

The impact of the opioid crisis on U.S. state prison systems

BACKGROUND

Prior studies have documented limited use of medications to treat opioid use disorders (OUD) for people incarcerated within state prisons in the United States. Using the framework of the criminal justice OUD service cascade, this study interviewed representatives of prison systems in states most heavily impacted by opioid overdose regarding the provision of medications for OUD (MOUD).

METHODS

A stratified sampling strategy included states with high indicators of opioid-overdose deaths. Two sampling strata targeted states with: 1) OUD overdose rates significantly higher than the per capita national average; or 2) high absolute number of OUD overdose fatalities. Interviews were completed with representatives from 21 of the 23 (91%) targeted states in 2019, representing 583 prisons across these states. Interviews assessed service provision across the criminal justice OUD service cascade, including OUD screening, withdrawal management, MOUD availability and provision, overdose prevention, re-entry services, barriers, and needs for training and technical assistance.

RESULTS

MOUD (buprenorphine, methadone, or naltrexone) was available in at least one prison in approximately 90% of the state prison systems and all three medications were available in at least one prison in 62% of systems. However, MOUD provision was limited to subsets of prisons within these systems: 15% provided buprenorphine, 9% provided methadone, 36% provided naltrexone, and only 7% provided all three. Buprenorphine and methadone were most frequently provided to pregnant women or individuals already receiving these at admission, whereas naltrexone was primarily used at release. Funding was the most frequently cited barrier for all medications.

CONCLUSION

Study findings yield a complex picture of how, when, and to whom MOUD is provided across prisons within prison systems in states most heavily impacted by opioid overdose in the United States and have implications for expanding availability.

Ethnography and user-centered design to inform context-driven implementation

Despite pervasive findings pointing to its inextricable role in intervention implementation, context remains poorly understood in implementation science. Existing approaches for describing context (e.g., surveys, interviews) may be narrow in scope or superficial in their elicitation of contextual data. Thus, in-depth and multilevel approaches are needed to meaningfully describe the contexts into which interventions will be implemented. Moreover, many studies assess context without subsequently using contextual information to enhance implementation. To be useful for improving implementation, though, methods are needed to apply contextual information during implementation. In the case example presented in this paper, we embedded an ethnographic assessment of context within a user-centered design approach to describe implementation context and apply that information to promote implementation. We developed a patient-reported outcome measure-based clinical intervention to assess and address the pervasive unmet needs of young adults with cancer: the Needs Assessment & Service Bridge (NA-SB). In this paper, we describe the user-centered design process that we used to anticipate context modifications needed to deliver NA-SB and implementation strategies needed to facilitate its implementation. Our ethnographic contextual inquiry yielded a rich understanding of local implementation context and contextual variation across potential scale-up contexts. Other methods from user-centered design (i.e., translation tables and a design team prototyping workshop) allowed us to translate that information into specifications for NA-SB delivery and a plan for implementation. Embedding ethnographic methods within a user-centered design approach can help us to tailor interventions and implementation strategies to their contexts of use to promote implementation.

Inequities in the delivery of mental health care: a grounded theory study of the policy context of primary care

BACKGROUND

Strengthening capacity for mental health in primary care improves health outcomes by providing timely access to coordinated and integrated mental health care. The successful integration of mental health in primary care is highly dependent on the foundation of the surrounding policy context. In Ontario, Canada, policy reforms in the early 2000's led to the implementation of a new interprofessional team-model of primary care called Family Health Teams. It is unclear the extent to which the policy context in Ontario influenced the integration of mental health care in Family Health Teams emerging from this period of policy reform. The research question guiding this study was: what were key features of Ontario's policy context that influenced FHTs capacity to provide mental health services for mood and anxiety disorders?

METHODS

A qualitative study informed by constructivist grounded theory. Individual interviews were conducted with executive directors, family physicians, nurse practitioners, nurses, and the range of professionals who provide mental health services in interprofessional primary care teams; community mental health providers; and provincial policy and decision makers. We used an inductive approach to data analysis. The electronic data management programme NVivo11 helped organise the data analysis process.

RESULTS

We conducted 96 interviews with 82 participants. With respect to the contextual factors considered to be important features of Ontario's policy context that influenced primary care teams' capacity to provide mental health services, we identified four key themes: i) lack of strategic direction for mental health, ii) inadequate resourcing for mental health care, iii) rivalry and envy, and, iv) variations across primary care models.

CONCLUSIONS

As the first point of contact for individuals experiencing mental health difficulties, primary care plays an important role in addressing population mental health care needs. In Ontario, the successful integration of mental health in primary care has been hindered by the lack of strategic direction, and inconsistent resourcing for mental health care. Achieving health equity may be stunted by the structural variations for mental health care across Family Health Teams and across primary care models in Ontario.

A more practical guide to incorporating health equity domains in implementation determinant frameworks

BACKGROUND

Due to striking disparities in the implementation of healthcare innovations, it is imperative that researchers and practitioners can meaningfully use implementation determinant frameworks to understand why disparities exist in access, receipt, use, quality, or outcomes of healthcare. Our prior work documented and piloted the first published adaptation of an existing implementation determinant framework with health equity domains to create the Health Equity Implementation Framework. We recommended integrating these three health equity domains to existing implementation determinant frameworks: (1) culturally relevant factors of recipients, (2) clinical encounter or patient-provider interaction, and (3) societal context (including but not limited to social determinants of health). This framework was developed for healthcare and clinical practice settings. Some implementation teams have begun using the Health Equity Implementation Framework in their evaluations and asked for more guidance.

METHODS

We completed a consensus process with our authorship team to clarify steps to incorporate a health equity lens into an implementation determinant framework.

RESULTS

We describe steps to integrate health equity domains into implementation determinant frameworks for implementation research and practice. For each step, we compiled examples or practical tools to assist implementation researchers and practitioners in applying those steps. For each domain, we compiled definitions with supporting literature, showcased an illustrative example, and suggested sample quantitative and qualitative measures.

CONCLUSION

Incorporating health equity domains within implementation determinant frameworks may optimize the scientific yield and equity of implementation efforts by assessing and ideally addressing implementation and equity barriers simultaneously. These practical guidance and tools provided can assist implementation researchers and practitioners to concretely capture and understand barriers and facilitators to implementation disparities.

An asset-based community development approach to reducing alcohol harm: Exploring barriers and facilitators to community mobilisation at initial implementation stage

Globally there is a need to engage communities in actions to reduce alcohol harm. This paper reports on the initial implementation phase of an asset-based community development (ABCD) approach to reducing alcohol harm in ten pre-identified areas across Greater Manchester (UK). This qualitative study highlights the experiences of stakeholders responsible for, or engaged in, implementation. Findings show that it is challenging to recruit sufficient volunteers in a specific, small area/community, which may limit the ability to build health assets. Wider policy and organisational factors that should be understood prior to implementing a place-based volunteer-led health promotion programme are also identified.

Trial registration

https://www.isrctn.com/ISRCTN81942890.

•

Identifies factors affecting implementation of a place-based alcohol harm intervention.

•

Examines a locally driven asset-based community development volunteer approach.

•

Capacity was limited by small volunteer pool in the specific targeted intervention areas.

•

Demonstrates the need to cultivate a partnership-based approach at local level.

Barriers impacting the POINT pragmatic trial: the unavoidable overlap between research and intervention procedures in "real-world" research

Background

This manuscript provides a research update to the ongoing pragmatic trial of Project POINT (Planned Outreach, Intervention, Naloxone, and Treatment), an emergency department-based peer recovery coaching intervention for linking patients with opioid use disorder to evidence-based treatment. The research team has encountered a number of challenges related to the “real-world” study setting since the trial began. Using an implementation science lens, we sought to identify and describe barriers impacting both the intervention and research protocols of the POINT study, which are often intertwined in pragmatic trials due to the focus on external validity.

Method

Qualitative data were collected from 3 peer recovery coaches, 2 peer recovery coach supervisors, and 3 members of the research team. Questions and deductive qualitative analysis were guided by the Consolidated Framework for Implementation Research (CFIR).

Results

Nine unique barriers were noted, with 5 of these barriers impacting intervention and research protocol implementation simultaneously. These simultaneous barriers were timing of intervention delivery, ineffective communication with emergency department staff, lack of privacy in the emergency department, the fast-paced emergency department setting, and patient’s limited resources. Together, these barriers represent the intervention characteristics, inner setting, and outer setting domains of the CFIR.

Conclusion

Results highlight the utility of employing an implementation science framework to assess implementation issues in pragmatic trials and how this approach might be used as a quality assurance mechanism given the considerable overlap that exists between research and intervention protocols in real-world trial settings. Previously undocumented changes to the trial design that have been made as a result of the identified barriers are discussed.

What Happens When Training Goes Virtual? Adapting Training and Technical Assistance for the School Mental Health Workforce in Response to COVID-19

The Northwest Mental Health Technology Transfer Center (MHTTC) provides workforce training and technical assistance (TA) to support evidence-based school mental health practices. Historically, this support targeted school professionals through in-person and online trainings, workshops, and coaching. However, in response to COVID-19 restrictions, all support moved to online formats, and the Center introduced trainings for families and caregivers. The purpose of this article is to present preliminary process and outcome data that compare the reach and impact of support before and following COVID-19-related restrictions. Results suggest that transition to online support resulted in a wider reach and a more diverse audience, with no decrease in trainee satisfaction and perceived impact. Furthermore, families and caregivers reported positive gains in knowledge and behaviors following participation in a virtual youth suicide prevention training. Together, these findings suggest that online training and TA can provide tangible benefits to professionals and family members who support student mental health.

Enabling Transdisciplinary Collaboration: Stakeholder Views on Working With "Children With Mentally Ill Parents" Research Groups

Working collaboratively and openly together with stakeholders has become a common phenomenon in research. While previous studies have gathered a clear picture on researchers' attitudes, motivations, and barriers for actively involving stakeholders in transdisciplinary research, the stakeholder perspective is yet unknown. Therefore, this paper sets out to identify how stakeholders perceive transdisciplinary collaborations with researchers. This paper in particular reveals the enablers and barriers for such collaborations from the viewpoint of stakeholders. To do so, we look at how stakeholders, who were actively involved in the governance structure of two "children with mentally ill parents" research groups in Austria, perceived their collaboration with researchers. We used a mixed-method, quantitative-qualitative design. We conducted an online survey and interviews with the members of the advisory board and competence group. These stakeholders reported great satisfaction with the transdisciplinary collaboration and emphasized the value of different expertise. As the most important enablers for successful, transdisciplinary collaboration stakeholders emphasized researchers' open-mindedness toward new perspectives and approaches, flexibility to adapt to the research process along the way, and creativity dealing with diverse backgrounds and skills. Stakeholders further underlined the importance of a person facilitating the collaboration process between researchers and stakeholders to resolve any tensions and insecurities. Concluding, researchers' attitudes, and in particular their understanding of the value of stakeholder involvement in research are key enablers for successful transdisciplinary research collaborations.

A Longitudinal Mixed Method Approach for Assessing Implementation Context and Process Factors: Comparison of Three Sites from a Housing First Implementation Strategy Pilot

Background

Implementation science's focus on establishing implementation strategy effectiveness has overshadowed the need to understand differential performance of such strategies under various conditions. Methods allowing for assessment between implementation context and process can help address this gap. This paper provides a detailed description of a mixed method procedure for assessing factors related to the implementation context and process intersection, which was developed as part of the pilot study of the Housing First Technical Assistance and Training (HFTAT) Program, a multifaceted strategy designed to support Housing First model implementation.

Methods

The HFTAT was pilot tested among a sample of three organizations. Our mixed method approach combines two tools often used in implementation research-the Stages of Implementation Completion and the Consolidated Framework for Implementation Research-in a novel way. Several stages to analysis were completed, starting with a separate analysis of data pertaining to each measure and then two levels of mixed method analysis.

Results

The approach provided a better understanding of the issues that impacted the implementation guided by the HFTAT, suggesting: (1) individual determinants seemed to have a bigger impact based on the number of SIC phases they affected, (2) implementation context and process were connected through climate-related factors in the inner setting that made the sites more or less responsive to addressing identified barriers, and (3) there is a need to better assess context factors to identify areas where implementation drivers should be better targeted to facilitate change, and this is supported by prior research.

Conclusions

Understanding the underlying factors impacting a setting's performance related to a specific implementation strategy has potential to improve decision-making and optimize future implementation efforts. The approach likely be as successful combining the SIC with other determinant frameworks and should be utilized at the onset of an implementation project to maximize its usefulness.

Perceptions, Experiences, and Outcomes of Lactation Support in the Workplace: A Systematic Literature Review

BACKGROUND

Despite legislation requiring break time and a private space to express milk, variations exist in accommodations for breastfeeding employees in the United States.

RESEARCH AIMS

We aimed to describe employee and employer perceptions of and experiences with workplace lactation support in the United States and to identify research needed to inform workplace lactation support programs.

METHODS

We searched Academic Search Complete, Business Search Complete, CINAHL, MEDLINE, PubMed, and PsycInfo for peer-reviewed articles published from 2009 to 2019 (n = 1638). We included 27 articles. Studies were categorized into four non-exclusive themes: (a) employee perceptions of and experiences with workplace lactation support; (b) employer reports of workplace lactation support; (c) association between workplace lactation support and business outcomes; and (d) association between workplace lactation support and breastfeeding outcomes.

RESULTS

Analyses of associations between lactation support at work and employee breastfeeding outcomes (n = 14, 52%), and employee perceptions of and experiences with lactation support at work (n = 14, 52%) were most common, followed by employer reports of lactation support (n = 3, 11%) and associations between lactation support at work and job satisfaction (n = 3, 11%). Results indicated that workplace lactation support varied by employer, and that employee perceptions of and experiences with workplace lactation support varied by demographic and employment characteristics. The use of cross-sectional designs, unvalidated instruments, and limited representation from women with low incomes and minorities were common study limitations.

CONCLUSIONS

More research is needed to learn about experiences of employers and low-income and minority women with workplace lactation support and associations with business-relevant outcomes.

Quantitative measures of health policy implementation determinants and outcomes: a systematic review

BACKGROUND

Public policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures.

METHODS

Three frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures.

RESULTS

Database searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported.

CONCLUSIONS

Well-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health.

REGISTRATION

Not registered.

Supporting children who have a parent with a mental illness in Tyrol: a situational analysis for informing co-development and implementation of practice changes

BACKGROUND

A research project, which aims to improve the situation of children of parents with a mental illness (COPMI) is currently underway in the Austrian region of Tyrol. The project aims to strengthen formal and informal support structures around the child, through enhancing their village of collaborative support. Understanding the current situation in the region is vital for implementing practice change. This paper aims to gain knowledge regarding the Tyrolean societal and service provision context.

METHODS

We collected qualitative (17 interviews among stakeholder and people with lived experience) and quantitative data (e.g. health insurance data) regarding overall societal characteristics, epidemiology of mental illness, currently existing services, uptake of services, and current practices and challenges of identifying and supporting COPMIs. We analysed data along eight external context dimensions: 1) professional influences, 2) political support, 3) social climate, 4) local infrastructure, 5) policy and legal climate, 6) relational climate, 7) target population, and 8) funding and economic climate.

RESULTS

We identified that there is awareness of potential challenges related to COPMIs at both a professional and planning level. Additionally, there is a lack of installed support processes and standards to meet these children's needs across Tyrol. A variety of services are available both for unwell parents, as well as for families and individual family members. Yet, only one small service addresses COPMIs directly. Services fall into different sectors (education, health, social affairs) and are funded from different sources, making coordination difficult. Access varies from universal to rather restricted (i.e. through referral). The potential number of parents which could be reached in order to identify their children via adult mental health, differs considerably by setting. Societal structures indicate that the informal and voluntary sector may be a realistic source for supporting COPMIs.

CONCLUSIONS

The societal structures and the current services provide a rich resource for improving identification and support of COPMIs, however considerable coordination and behaviour change efforts will be required due to the fragmentation of the system and professional cultures. The insights into the context of supporting COPMIs have been of high value for developing and implementing practice changes in the local organizations.

Developing an Evidence-Based Technical Assistance Model: a Process Evaluation of the National Training and Technical Assistance Center for Child, Youth, and Family Mental Health

The National Training and Technical Assistance Center for Child, Youth, and Family Mental Health (NTTAC) supports the development and implementation of systems of care (SOC) for youth with serious emotional disorders (SED) and their families. This article presents results from a process evaluation of NTTAC, conducted to support the Center’s quality improvement and contribute to the knowledge base around provision of technical assistance (TA). The evaluation used a mixed methods approach with data collection focused on a defined subset of NTTAC TA recipients—recipients of federal Comprehensive Community Mental Health Services for Children SOC grants. Data sources included coded administrative records from SOC grant sites, administrative data from NTTAC, standardized measures of SOC development, and stakeholder survey data. Results indicate that TA dosage matched needs and goals of TA recipients (SOC sites), overall levels of satisfaction with TA were high, and TA content was generally aligned with need. TA recipients reported significant progress on indicators of SOC development over time. Together, these findings suggest that it is possible to develop TA methods that reflect the level and type of TA recipients’ goals and needs, and, in turn, positively impact SOC development and behavioral health service delivery.

The role of the outer setting in implementation: associations between state demographic, fiscal, and policy factors and use of evidence-based treatments in mental healthcare

Background

Despite consistent recognition of their influence, empirical study of how outer setting factors (e.g., policies, financing, stakeholder relationships) influence public systems’ investment in and adoption of evidence-based treatment (EBT) is limited. This study examined associations among unmodifiable (e.g., demographic, economic, political, structural factors) and modifiable (e.g., allocation of resources, social processes, policies, and regulations) outer setting factors and adoption of behavioral health EBT by US states.

Methods

Multilevel models examined relationships between state characteristics, an array of funding and policy variables, and state adoption of behavioral health EBTs for adults and children across years 2002–2012, using data from the National Association for State Mental Health Program Directors Research Institute and other sources.

Results

Several unmodifiable state factors, including per capita income, controlling political party, and Medicaid expansion, predicted level of state fiscal investments in EBT. By contrast, modifiable factors, such as interagency collaboration and investment in research centers, were more predictive of state policies supportive of EBT. Interestingly, level of adult EBT adoption was associated with state fiscal supports for EBT, while child EBT adoption was predicted more by supportive policies. State per capita debt and direct state operation of services (versus contracting for services) predicted both child and adult EBT adoption.

Conclusions

State-level EBT adoption and associated implementation support is associated with an interpretable array of policy, financing, and oversight factors. Such information expands our knowledge base of the role of the outer setting in implementation and may provide insight into how best to focus efforts to promote EBT for behavioral health disorders.

Inter-facility transfers for advanced critical care services: Impacts on rural families

AIMS AND OBJECTIVES

The aim of this article is to provide an integrative review of rural family members' experiences of a relative's inter-facility transfer to an urban tertiary care centre for advanced critical care services.

BACKGROUND

Although there is an increasing awareness of the negative consequences of a relative's critical illness for family members, there has been limited research conducted on the impacts of this experience for families of patients in rural settings who require an inter-facility transfer for advanced services.

SEARCH STRATEGY

The search strategy for this review was framed by the study purpose, research question, and a conceptual structure of key constructs. Databases and reference lists were electronically and manually searched.

INCLUSION AND EXCLUSION CRITERIA

Articles were included if they were research articles, available in English, and focused on family members' experiences of inter-facility transfer during critical illness. Articles focusing on critical care transfers within a single urban setting, elective transfers undertaken for patient or family preference, transfers not associated with critical illness, and systematic or scoping reviews were excluded.

CONCLUSION

Stress and anxiety are central to the rural family members' experience of this phenomenon. These constructs are influenced by modifiable factors, including the actions of health care providers, the financial burden associated with the transfer, the family members' physical proximity to the patient, and their access to information and support networks. Further research is required to explore potential short- and long-term consequences of this experience and the experience of family members who are left behind and to develop and test interventions to support families.

RELEVANCE TO CLINICAL PRACTICE

This review highlights the concern that health care providers frequently misunderstand the needs of family members who are experiencing this phenomenon. Through a better understanding of this phenomenon, health care providers will be better positioned to effectively support these individuals during this unique experience.

Influence of external contextual factors on the implementation of health and social care interventions into practice within or across countries-a protocol for a 'best fit' framework synthesis

BACKGROUND

The widespread implementation of interventions is often hindered by a decline and variability in effectiveness across implementation sites. It is anticipated that variations in the characteristics of the external context in different sites, such as the political and funding environment, socio-cultural context, physical environment or population demographics can influence implementation outcome. However, there is only a limited understanding about which and how external contextual factors influence implementation. We aim to develop a comprehensive framework conceptualising the influence of external contextual factors on implementation, particularly when spreading health and social care interventions within or across countries.

METHODS

The review will use the 'best fit' framework synthesis approach. In the first stage of the review, we will examine existing frameworks, models, concepts and theories on external contextual factors and their influence on implementation from a variety of sectors and disciplines including health and social care, education, environmental studies and international development fields. The resulting a priori meta-framework will be tested and refined in the second review stage by analysing evidence from empirical studies focusing on the implementation of health and social care interventions within or across countries. Searches will be conducted in bibliographic databases such as MEDLINE, ERIC, HMIC and IBSS, grey literature sources and on relevant websites. We will also search reference lists, relevant journals, perform citation searches and ask experts in the field. There is no restriction to study type, setting, intervention type or implementation strategy to enable obtaining a broad and in-depth knowledge from various sources of evidence.

DISCUSSION

The review will lead to a comprehensive framework for understanding the influence of external contextual factors on implementation, particularly when spreading health and social care interventions within or across countries. The framework is anticipated to help identify factors explaining the decline and variability in effectiveness of interventions and assessing the prospects of implementation effectiveness, when spreading interventions. We do not intend to only develop another stand-alone implementation framework but one that can be used in conjunction with existing frameworks. The framework can be honed and validated in future empirical research.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018084485.

Emergency department-based peer support for opioid use disorder: Emergent functions and forms

Emergency department (ED)-based peer support programs aimed at linking persons with opioid use disorder (OUD) to medication for addiction treatment and other recovery services are a promising approach to addressing the opioid crisis. This brief report draws on experiences from three states' experience with such programs funded by the SAMHSA Opioid State Targeted Repose (STR) grants. Core functions of such programs include: Integration of peer supports in EDs; Alerting peers of eligible patients and making the patient aware of peer services; and connecting patients with recovery services. Qualitative data were analyzed using a general inductive approach conducted in 3 steps in order to identify forms utilized to fulfill these functions. Peer integration differed in terms of peer's physical location and who hired and supervised peers. Peers often depend on ED staff to alert them to potential patients while people other than the peers often first introduce potential patients to programming. Programs generally schedule initial appointments for recovery services for patients, but some programs provide a range of other services aimed at supporting participation in recovery services. Future effectiveness evaluations of ED-based peer support programs for OUD should consistently report on forms used to fulfill core functions.

Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework

Background

Effective implementation of evidence-based practices (EBPs) remains a significant challenge. Numerous existing models and frameworks identify key factors and processes to facilitate implementation. However, there is a need to better understand how individual models and frameworks are applied in research projects, how they can support the implementation process, and how they might advance implementation science. This systematic review examines and describes the research application of a widely used implementation framework, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework.

Methods

A systematic literature review was performed to identify and evaluate the use of the EPIS framework in implementation efforts. Citation searches in PubMed, Scopus, PsycINFO, ERIC, Web of Science, Social Sciences Index, and Google Scholar databases were undertaken. Data extraction included the objective, language, country, setting, sector, EBP, study design, methodology, level(s) of data collection, unit(s) of analysis, use of EPIS (i.e., purpose), implementation factors and processes, EPIS stages, implementation strategy, implementation outcomes, and overall depth of EPIS use (rated on a 1–5 scale).

Results

In total, 762 full-text articles were screened by four reviewers, resulting in inclusion of 67 articles, representing 49 unique research projects. All included projects were conducted in public sector settings. The majority of projects (73%) investigated the implementation of a specific EBP. The majority of projects (90%) examined inner context factors, 57% examined outer context factors, 37% examined innovation factors, and 31% bridging factors (i.e., factors that cross or link the outer system and inner organizational context). On average, projects measured EPIS factors across two of the EPIS phases (M = 2.02), with the most frequent phase being Implementation (73%). On average, the overall depth of EPIS inclusion was moderate (2.8 out of 5).

Conclusion

This systematic review enumerated multiple settings and ways the EPIS framework has been applied in implementation research projects, and summarized promising characteristics and strengths of the framework, illustrated with examples. Recommendations for future use include more precise operationalization of factors, increased depth and breadth of application, development of aligned measures, and broadening of user networks. Additional resources supporting the operationalization of EPIS are available.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0842-6) contains supplementary material, which is available to authorized users.

The housing first technical assistance and training (HFTAT) implementation strategy: outcomes from a mixed methods study of three programs

BACKGROUND

This paper discusses the initial testing of the Housing First Training and Technical Assistance (HFTAT) Program, a multifaceted, distance-based strategy for the implementation of the Housing First (HF) supportive housing model. HF is a complex housing intervention for serving people living with serious mental illness and a substance use disorder that requires significant individual- and structural-level changes to implement. As such, the HFTAT employs a combined training and consultation approach to target different levels of the organization. Training delivered to all organizational staff focuses on building individual knowledge and uses narrative storytelling to overcome attitudinal implementation barriers. Consultation seeks to build skills through technical assistance and fidelity audit and feedback.

METHOD

We employed a mixed method design to understand both individual-level (e.g., satisfaction with the HFTAT, HF knowledge acquisition and retention, and HF acceptability and appropriateness) and structural-level (e.g., fidelity) outcomes. Quantitative data were collected at various time points, and qualitative data were collected at the end of HFTAT activities. Staff and administrators (n = 113) from three programs across three states participated in the study.

RESULTS

Satisfaction with both training and consultation was high, and discussions demonstrated both activities were necessary. Flexibility of training modality and narrative storytelling were particular strengths, while digital badging and the community of practice were perceived as less valuable because of incompatibilities with the work context. HF knowledge was high post training and retained after 3-month follow-up. Participants reported training helped them better understand the model. Attitudes toward evidence-based interventions improved over 6 months, with qualitative data supporting this but demonstrating some minor concerns related to acceptability and appropriateness. Fidelity scores for all programs improved over 9 months.

CONCLUSION

The HFTAT was a well-liked and generally useful implementation strategy. Results support prior research pointing to the value of both (a) multifaceted strategies and (b) combined training and consultation approaches. The study also provides evidence for narrative storytelling as an approach for changing attitudinal implementation barriers. The need for compatibility between specific elements of an implementation strategy and the work environment was also observed.