Toward a policy ecology of implementation of evidence-based practices in public mental health settings

"Allowing space for voice…all our voices:" Understanding Ho'ouna Pono implementation through educational leadership perspectives in rural Hawai'i schools

Epidemiological research over the past two decades has highlighted substance use disparities that affect Native Hawaiian and Pacific Islander youth, and the lack of effective approaches to address such disparities (Okamoto et al., 2019). The Ho'ouna Pono curriculum is a culturally grounded, teacher-implemented, video-enhanced substance use prevention program that has demonstrated efficacy in rural Hawai'i in a large-scale trial (Okamoto et al., 2019). Despite its potential to ameliorate health disparities and address youth substance use, prevention programs such as Ho'ouna Pono have been poorly disseminated and implemented across Hawai'i, raising the question: Why are effective prevention programs not used in communities that most need them? The present study used concept mapping to understand previously identified implementation barriers and develop implementation strategies for Ho'ouna Pono. Seven Hawai'i Department of Education (HIDOE) educational leaders and administrators sorted Ho'ouna Pono implementation barriers (e.g., "There is a lack of HIDOE funding to support prevention curricula"), named concepts, and rated barriers' perceived impact and difficulty. Multidimensional scaling and cluster analysis yielded a five-cluster solution: (1) Kumu (Hawaiian word for teacher) Controlled, (2) School Level Buy-in, (3) Curriculum, (4) Student Attitudes + Mindsets (Family + Community), and (5) Policy. Participant ratings identified eight high-impact and low-difficulty barriers. Discussion revealed important intersections among barriers indicating the need for coordinated and cross-level implementation strategies to support Ho'ouna Pono sustainment. Brainstormed implementation strategies using participants' own language highlighted a need for participatory methods in school settings to bidirectionally share ways to best sustain substance use prevention programs.

Interagency Coordination to Manage Co-Occurring Intellectual and Developmental Disabilities and Mental Health Conditions

OBJECTIVE

The authors aimed to identify barriers to and strategies for supporting coordination between state agencies for intellectual and developmental disability (IDD) or mental health to meet the mental health needs of people with co-occurring IDD and mental health conditions.

METHODS

Forty-nine employees of state agencies as well as advocacy and service delivery organizations across 11 U.S. states with separate IDD and mental health agencies were interviewed between April 2022 and April 2023. Data were analyzed with a thematic analysis approach.

RESULTS

Interviewees reported that relationships between the IDD and mental health agencies have elements of both competition and coordination and that coordination primarily takes place in response to crisis events. Barriers to interagency coordination included a narrow focus on the populations targeted by each agency, within-state variation in agency structures, and a lack of knowledge about co-occurring IDD and mental health conditions. Interviewees also described both administrative (e.g., memorandums of understanding) and agency culture (e.g., focusing on whole-person care) strategies that are or could be used to improve coordination to provide mental health services for people with both IDD and a mental health condition.

CONCLUSIONS

Strategies that support state agencies in moving away from crisis response toward a focus on whole-person care should be prioritized to support coordination of mental health services for individuals with co-occurring IDD and mental health conditions.

Use of concept mapping to inform a participatory engagement approach for implementation of evidence-based HPV vaccination strategies in safety-net clinics

BACKGROUND

Multiple evidence-based strategies (EBS) for promoting HPV vaccination exist. However, adolescent HPV vaccination rates remain below target levels in communities at high risk for HPV-associated cancers and served by safety-net clinics. Participatory engaged approaches are needed to leverage the expertise of community and clinical partners in selecting EBS relevant to their local context. We engaged concept mapping as a method to inform the adoption and adaptation of EBS that seeks to empower implementation partners to prioritize, select, and ultimately implement context-relevant EBS for HPV vaccination.

METHODS

Using 38 EBS statements generated from qualitative interviews and national HPV vaccine advocacy sources, we conducted a modified concept mapping activity with partners internal to safety-net clinics and external community members in two study sites of a larger implementation study (Greater Los Angeles and New Jersey), to sort EBS into clusters and rate each EBS by importance and feasibility for increasing HPV vaccination within safety-net clinics. Concept mapping findings (EBS statement ratings, ladder graphs and go-zones) were shared with leaders from a large federally qualified health center (FQHC) system (focusing on three clinic sites), to select and implement EBS over 12 months.

RESULTS

Concept mapping participants (n=23) sorted and rated statements, resulting in an eight-cluster solution: 1) Community education and outreach; 2) Advocacy and policy; 3) Data access/quality improvement monitoring; 4) Provider tracking/audit and feedback; 5) Provider recommendation/communication; 6) Expanding vaccine access; 7) Reducing missed opportunities; and 8) Nurse/staff workflow and training. The FQHC partner then selected to intervene on eight of 17 EBS statements in the "go-zone" for action, with three from "reducing missed opportunities," two from "nurse/staff workflow and training," and one each from "provider tracking/audit and feedback," "provider recommendation/communication," and "expanding vaccine access," which the research team addressed through the implementation of three multi-level intervention strategies (e.g., physician communication training, staff training and workflow assessment, audit and feedback of clinic processes).

CONCLUSIONS

Concept mapping provided a powerful participatory approach to identify multilevel EBS for HPV vaccination relevant to the local safety-net clinic context, particularly when several strategies exist, and prioritization is necessary. This study demonstrates how a clinic system benefited directly from the ratings and prioritization of EBS by multilevel clinic and community partners within the broader safety-net clinic context to identify and adapt prioritized solutions needed to advance HPV vaccine equity.

Using policy codesign to achieve multi-sector alignment in adolescent behavioral health: a study protocol

BACKGROUND

Policymaking is quickly gaining focus in the field of implementation science as a potential opportunity for aligning cross-sector systems and introducing incentives to promote population health, including substance use disorders (SUD) and their prevention in adolescents. Policymakers are seen as holding the necessary levers for realigning service infrastructure to more rapidly and effectively address adolescent behavioral health across the continuum of need (prevention through crisis care, mental health, and SUD) and in multiple locations (schools, primary care, community settings). The difficulty of aligning policy intent, policy design, and successful policy implementation is a well-known challenge in the broader public policy and public administration literature that also affects local behavioral health policymaking. This study will examine a blended approach of coproduction and codesign (i.e., Policy Codesign), iteratively developed over multiple years to address problems in policy formation that often lead to poor implementation outcomes. The current study evaluates this scalable approach using reproducible measures to grow the knowledge base in this field of study.

METHODS

This is a single-arm, longitudinal, staggered implementation study to examine the acceptability and short-term impacts of Policy Codesign in resolving critical challenges in behavioral health policy formation. The aims are to (1) examine the acceptability, feasibility, and reach of Policy Codesign within two geographically distinct counties in Washington state, USA; (2) examine the impact of Policy Codesign on multisector policy development within these counties using social network analysis; and (3) assess the perceived replicability of Policy Codesign among leaders and other staff of policy-oriented state behavioral health intermediary organizations across the USA.

DISCUSSION

This study will assess the feasibility of a specific approach to collaborative policy development, Policy Codesign, in two diverse regions. Results will inform a subsequent multi-state study measuring the impact and effectiveness of this approach for achieving multi-sector and evidence informed policy development in adolescent SUD prevention and treatment.

A Tale of Two Taxes: Implementation of Earmarked Taxes for Behavioral Health Services in California and Washington State

OBJECTIVE

The authors sought to characterize perceptions of the impacts, attributes, and support for taxes earmarked for behavioral health services and to compare perceptions of the taxes among professionals in California and Washington, two states differing in earmarked tax designs.

METHODS

Surveys were completed by 155 public agency and community organization professionals involved in tax implementation in California (N=87) and Washington State (N=68) during 2022-2023 (29% response rate). Respondents indicated their perceptions of the taxes' impacts, attributes, and support. Responses were summed as aggregate scores and were also analyzed as individual items. Bivariate analyses were used to compare responses of professionals in California versus Washington State.

RESULTS

Earmarked taxes were generally regarded positively. Of the respondents, >80% strongly agreed that the taxes increased funding for services and were helpful, and only 10% strongly agreed that the taxes decreased behavioral health funding from other sources. Substantially more respondents in California than in Washington State strongly agreed that taxes' reporting requirements were complicated (45% vs. 5%, p<0.001) and that the taxes increased unjustified scrutiny of services or systems (33% vs. 2%, p<0.001). However, more respondents in California than in Washington State also strongly agreed that the taxes increased public awareness about behavioral health (56% vs. 15%, p<0.001) and decreased behavioral health stigma (47% vs. 14%, p<0.001).

CONCLUSIONS

Perceptions of the strengths and weaknesses of taxes earmarked for behavioral health services may vary by design features of the tax. Such features include stigma-reduction initiatives and tax spending and reporting requirements.

Exploring the Impact of External Facilitation Using Evidence-Based Implementation Strategies for Increasing Motivational Interviewing Capacity Among Outpatient Substance Use Disorder (SUD) Treatment Providers

The large majority of individuals who access substance use disorders (SUD) treatment do not receive evidence-based care. Little attention has been paid to the notion that the scale-up of evidence-based practices (EBPs) has been limited in large part due to a weakness in the "distribution system" for bringing new innovations to the attention of practitioners and into practice settings. This study explores the impact of the Training and Practice Implementation Institute (TPII; funded by the New York City Department of Health and Mental Hygiene), an intensive technical assistance initiative that offers external facilitation to outpatient SUD treatment providers via the incorporation of multiple evidence-based implementation strategies to enhance the practice of motivational interviewing (MI). Findings from this study show that staff completed a large majority (86%) of required training/technical assistance (TTA) activities across the 9-month implementation period, demonstrating a high level of engagement among staff and the feasibility of externally facilitated intensive TTA delivered to community-based organizations for the purpose of enhancing implementation of MI for SUDs. Results also show statistically significant improvements in the delivery of MI's technical components among staff, though did not reveal corresponding improvements in the delivery of MI's relational components. An understanding of the potential return on investment associated with externally facilitated TA, offers organizations an opportunity to inform the allocation of limited resources to areas where they will have the greatest impact and ultimately improve the quality and efficacy of services.

Mobile Crisis Services: A Clinician Survey of Current Suicide Prevention Practices and Barriers to Care Delivery

Mobile crisis teams (MCTs) deploy clinicians to assist individuals in acute crisis in the community. Little is known about the extent to which these teams provide evidence-based practices (EBPs) for suicide prevention nor the barriers they face. We surveyed 120 MCT clinicians across the United States about their: (1) use of suicide risk screening and assessment tools; (2) strategies used to address suicide risk (both EBPs and non-EBPs); and (3) perceived barriers to high-quality MCT services. Nearly all clinicians reported use of validated suicide screening tools and generic "safety planning." However, a sizeable minority also reported use of non-EBPs. Open-ended responses suggested many client/family-, clinician-, and systems-level barriers to MCT use of EBPs for suicide prevention. We identified several targets for future implementation efforts, including the need for de-implementation strategies to reduce use of ineffective and potentially harmful practices, and unique aspects of MCTs that require tailored implementation supports.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science

Policy implementation is a key component of scaling effective chronic disease prevention and management interventions. Policy can support scale-up by mandating or incentivizing intervention adoption, but enacting a policy is only the first step. Fully implementing a policy designed to facilitate implementation of health interventions often requires a range of accompanying implementation structures, like health IT systems, and implementation strategies, like training. Decision makers need to know what policies can support intervention adoption and how to implement those policies, but to date research on policy implementation is limited and innovative methodological approaches are needed. In December 2021, the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness and the Johns Hopkins Center for Mental Health and Addiction Policy convened a forum of research experts to discuss approaches for studying policy implementation. In this report, we summarize the ideas that came out of the forum. First, we describe a motivating example focused on an Affordable Care Act Medicaid health home waiver policy used by some US states to support scale-up of an evidence-based integrated care model shown in clinical trials to improve cardiovascular care for people with serious mental illness. Second, we define key policy implementation components including structures, strategies, and outcomes. Third, we provide an overview of descriptive, predictive and associational, and causal approaches that can be used to study policy implementation. We conclude with discussion of priorities for methodological innovations in policy implementation research, with three key areas identified by forum experts: effect modification methods for making causal inferences about how policies' effects on outcomes vary based on implementation structures/strategies; causal mediation approaches for studying policy implementation mechanisms; and characterizing uncertainty in systems science models. We conclude with discussion of overarching methods considerations for studying policy implementation, including measurement of policy implementation, strategies for studying the role of context in policy implementation, and the importance of considering when establishing causality is the goal of policy implementation research.

External context in individual placement and support implementation: a scoping review with abductive thematic analysis

BACKGROUND

Context including the external context may considerably affect the adoption, implementation, sustainment, and scale-up of evidence-based practices. We investigated external contextual features by conducting a scoping review of empirical research regarding the implementation of an evidence-based psychiatric or mental health vocational rehabilitation service called Individual Placement and Support (IPS).

METHODS

The protocol for the scoping review was registered with the Open Science Framework. We used the methodology by Joanna Briggs Institute for conducting the scoping review and reported it according to the PRISMA-ScR checklist. We searched 12 databases for research regarding 'Individual Placement and Support' or 'Evidence-Based Supported Employment'. We retained peer-reviewed empirical studies investigating external contextual factors and their impact on IPS implementation outcomes. We extracted data from the eligible articles and conducted descriptive and thematic analyses.

RESULTS

Fifty-nine original research papers met our eligibility requirements and were retained after reviewing 1124 titles and abstracts and 119 full texts. The analysis generated two main themes: (1) external contextual determinants of service delivery and (2) external systems influencing the evidence-to-practice process. The first main theme encompassed policies and laws, financing, and administratively instituted support resources, and organizational arrangements associated with external stakeholders that may facilitate or hinder the local implementation. The second main theme comprised strategies and actions used by different stakeholders to facilitate implementation locally or scale-up efforts at a system level.

DISCUSSION

Our scoping review illustrates the important role that external contextual factors play and how they may facilitate or hinder the implementation and scale-up of the IPS model across mental health services in different countries. Consideration of these factors by decision-makers in mental health and welfare services, planners, providers, and practitioners is likely to facilitate the development of effective strategies for bridging the evidence-practice gap in implementing the EBPs. Finally, the scoping review identified gaps in knowledge and offered suggestions for future research.

TRIAL REGISTRATION

Open Science Framework.

Revisiting the policy ecology framework for implementation of evidence-based practices in mental health settings

BACKGROUND

Over the past three decades, policy actors and actions have been highly influential in supporting the implementation of evidence-based practices (EBPs) in mental health settings. An early examination of these actions resulted in the Policy Ecology Framework (PEF), which was originally developed as a tactical primer for state and local mental health regulators in the field of child mental health. However, the policy landscape for implementation has evolved significantly since the original PEF was published. An interrogation of the strategies originally proposed in the PEF is necessary to provide an updated menu of strategies to improve our understanding of the mechanisms of policy action and promote system improvement.

OBJECTIVES

This paper builds upon the original PEF to address changes in the policy landscape for the implementation of mental health EBPs between 2009 and 2022. We review the current state of policy strategies that support the implementation of EBPs in mental health care and outline key areas for policy-oriented implementation research. Our review identifies policy strategies at federal, state, agency, and organizational levels, and highlights developments in the social context in which EBPs are implemented. Furthermore, our review is organized around some key changes that occurred across each PEF domain that span organizational, agency, political, and social contexts along with subdomains within each area.

DISCUSSION

We present an updated menu of policy strategies to support the implementation of EBPs in mental health settings. This updated menu of strategies considers the broad range of conceptual developments and changes in the policy landscape. These developments have occurred across the organizational, agency, political, and social contexts and are important for policymakers to consider in the context of supporting the implementation of EBPs. The updated PEF expands and enhances the specification of policy levers currently available, and identifies policy targets that are underdeveloped (e.g., de-implementation and sustainment) but are becoming visible opportunities for policy to support system improvement. The updated PEF clarifies current policy efforts within the field of implementation science in health to conceptualize and better operationalize the role of policy in the implementation of EBPs.

Comparing organization-focused and state-focused financing strategies on provider-level reach of a youth substance use treatment model: a mixed-method study

BACKGROUND

Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes.

METHOD

A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach.

RESULTS

The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities).

DISCUSSION

As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.

Strategies for enacting health policy codesign: a scoping review and direction for research

BACKGROUND

Strategies for supporting evidence-informed health policy are a recognized but understudied area of policy dissemination and implementation science. Codesign describes a set of strategies potentially well suited to address the complexity presented by policy formation and implementation. We examine the health policy literature describing the use of codesign in initiatives intended to combine diverse sources of knowledge and evidence in policymaking.

METHODS

The search included PubMed, MEDLINE, PsychInfo, CINAHL, Web of Science, and Google Scholar in November 2022 and included papers published between 1996 and 2022. Terms included codesign, health, policy, and system terminology. Title and abstracts were reviewed in duplicate and included if efforts informed policy or system-level decision-making. Extracted data followed scoping review guidelines for location, evaluation method, health focus, codesign definition, description, level of health system user input, sectors involved, and reported benefits and challenges.

RESULTS

From 550 titles, 23 citations describing 32 policy codesign studies were included from multiple continents (Australia/New Zealand, 32%; UK/Europe, 32%; South America, 14%; Africa, 9%; USA/Canada 23%). Document type was primarily case study (77%). The area of health focus was widely distributed. Policy type was more commonly little p policy (47%), followed by big p policy (25%), and service innovations that included policy-enabled funding (25%). Models and frameworks originated from formal design (e.g., human-centered or participatory design (44%), political science (38%), or health service research (16%). Reported outcomes included community mobilization (50%), policy feasibility (41%), improved multisector alignment (31%), and introduction of novel ideas and critical thinking (47%). Studies engaging policy users in full decision-making roles self-reported higher levels of community mobilization and community needs than other types of engagement.

DISCUSSION

Policy codesign is theoretically promising and is gaining interest among diverse health sectors for addressing the complexity of policy formation and implementation. The maturity of the science is just emerging. We observed trends in the association of codesign strategies and outcomes that suggests a research agenda in this area could provide practical insights for tailoring policy codesign to respond to local contextual factors including values, needs, and resources.

A scoping review of implementation science theories, models, and frameworks - an appraisal of purpose, characteristics, usability, applicability, and testability

BACKGROUND

A proliferation of theories, models, and frameworks (TMFs) have been developed in the implementation science field to facilitate the implementation process. The basic features of these TMFs have been identified by several reviews. However, systematic appraisals on the quality of these TMFs are inadequate. To fill this gap, this study aimed to assess the usability, applicability, and testability of the current TMFs in a structured way.

METHODS

A scoping review method was employed. Electronic databases were searched to locate English and Chinese articles published between January 2000 and April 2022. Search terms were specific to implementation science. Additionally, hand searches were administered to identify articles from related reviews. Purpose and characteristics such as the type of TMF, analytical level, and observation unit were extracted. Structured appraisal criteria were adapted from Birken et al.'s Theory Comparison and Selection Tool (T-CaST) to conduct an in-depth analysis of the TMFs' usability, applicability, and testability.

RESULTS

A total of 143 TMFs were included in this analysis. Among them, the most common purpose was to identify barriers and facilitators. Most TMFs applied the descriptive method to summarize the included constructs or the prescriptive method to propose courses of implementation actions. TMFs were mainly mid-range theories built on existing conceptual frameworks or demonstrated grand theories. The usability of the TMFs needs to be improved in terms of the provision of conceptually matched strategies to barriers and facilitators and instructions on the TMFs usage. Regarding the applicability, little attention was paid to the constructs of macro-level context, stages of scale-up and sustainability, and implementation outcomes like feasibility, cost, and penetration. Also, fewer TMFs could propose recommended research and measurement methods to apply the TMFs. Lastly, explicit hypotheses or propositions were lacking in most of the TMFs, and empirical evidence was lacking to support the claimed mechanisms between framework elements in testability.

CONCLUSIONS

Common limitations were found in the usability, application, and testability of the current TMFs. The findings of this review could provide insights for developers of TMFs for future theoretical advancements.

Who's "in the room where it happens"? A taxonomy and five-step methodology for identifying and characterizing policy actors

BACKGROUND

Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in "Big P" public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions-such as adopting evidence-based interventions at systems-scale-and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers.

METHODS

A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study-The Family First Prevention Services Act (FFPSA)-publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the "room" during policy decisions.

RESULTS

The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking "rooms," (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the "room," and (5) quantify (e.g., count actors across groups), summarize, and compare "rooms" to develop or select engagement approaches aligned with the "room" and actors. The use and outcomes of each step are exemplified through the FFPSA case study.

CONCLUSIONS

The pragmatic and transparent policy actor identification steps presented here can guide researchers' methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both "Big P" and "little p" (administrative guidelines, procedures) policymaking and implementation in global contexts.

Four very basic ways to think about policy in implementation science

BACKGROUND

Policy is receiving increasing attention in the field of implementation science. However, there remains a lack of clear, concise guidance about how policy can be conceptualized in implementation science research. Building on Curran's article "Implementation science made too simple"-which defines "the thing" as the intervention, practice, or innovation in need of implementation support-we offer a typology of four very basic ways to conceptualize policy in implementation science research. We provide examples of studies that have conceptualized policy in these different ways and connect aspects of the typology to established frameworks in the field. The typology simplifies and refines related typologies in the field. Four very basic ways to think about policy in implementation science research. 1) Policy as something to adopt: an evidence-supported policy proposal is conceptualized as "the thing" and the goal of research is to understand how policymaking processes can be modified to increase adoption, and thus reach, of the evidence-supported policy. Policy-focused dissemination research is well-suited to achieve this goal. 2) Policy as something to implement: a policy, evidence-supported or not, is conceptualized as "the thing" and the goal of research is to generate knowledge about how policy rollout (or policy de-implementation) can be optimized to maximize benefits for population health and health equity. Policy-focused implementation research is well-suited to achieve this goal. 3) Policy as context to understand: an evidence-supported intervention is "the thing" and policies are conceptualized as a fixed determinant of implementation outcomes. The goal of research is to understand the mechanisms through which policies affect implementation of the evidence-supported intervention. 4) Policy as strategy to use: an evidence-supported intervention is "the thing" and policy is conceptualized as a strategy to affect implementation outcomes. The goal of research is to understand, and ideally test, how policy strategies affect implementation outcomes related to the evidence-supported intervention.

CONCLUSION

Policy can be conceptualized in multiple, non-mutually exclusive ways in implementation science. Clear conceptualizations of these distinctions are important to advancing the field of policy-focused implementation science and promoting the integration of policy into the field more broadly.

Feasibility, Acceptability and Preliminary Implementation of the Cornerstone Program for Transition-Age Youth with Mental Health Conditions: A Mixed Methods Study

Transition-age youth with mental health conditions from low socio-economic backgrounds often drop out of mental health services and, as such, do not receive therapeutic doses of treatment. Cornerstone is an innovative team-based, multi-component intervention designed to address the clinical needs of this understudied population through coordination and extensive provision of services in vivo (in the community). The present study used a convergent parallel mixed-methods design. Researchers collected quantitative and qualitative data during a small developmental trial, analyzing the two data types independently and then exploring them side-by-side to evaluate feasibility, acceptability, and preliminary implementation. Semi-structured interviews and quantitative surveys were conducted with transition-age youth, clinic staff, and policy makers. Qualitative interview guides were developed using the Consolidated Framework for Implementation Research to build understanding on implementation determinants alongside feasibility and acceptability. A two-group preliminary randomized trial was conducted to assess feasibility outcomes, such as recruitment, randomization, measurement performance, and trends in pre- to post- outcomes. Using grounded theory coding techniques, transcripts were coded by multiple coders, and themes were identified on acceptability and implementation. The team recruited fifty-six transition-age youth. Randomization was used in the study and the intervention was provided without incident. Results suggest individual components with both the social worker and mentor were more acceptable to participants than group-based approaches. Thematic analyses revealed themes associated with the inner, outer, and policy contexts describing a range of critical implementation determinants. Findings suggest that Cornerstone is feasible, acceptable, and promising for transition-age youth. It represents an innovative multi-component intervention worth exploring for transition-age youth with mental health conditions in a larger efficacy trial.Trial registration: The trial was registered at ClinicalTrials.gov (NCT02696109) on 22 April 16, Protocol Record R34-MH102525-01A1, New York University, Cornerstone program for transition-age youth with serious mental illness: study protocol for a randomized controlled trial.

Developing a typology of network alteration strategies for implementation: a scoping review and iterative synthesis

BACKGROUND

Social networks transmit knowledge, influence, and resources. These relationships among patients, professionals, and organizations can shape how innovations are disseminated, adopted, implemented, and sustained. Network alteration interventions-interventions that change or rewire social networks-have the potential to be used as implementation strategies. Yet, the types, mechanisms, and effectiveness of these interventions for implementation are unclear. This scoping review and iterative synthesis identified and described network alteration strategies that could be tested for implementation.

METHODS

We used forward and backward citation tracking of influential articles on network interventions, bibliometric searches, and hand searches of peer-reviewed social network journals. At least two team members screened article titles/abstracts to identify studies that met inclusion criteria: empirical studies of an intervention, the intervention was designed to alter some element of a social network, and changes in social network metrics were measured at two or more time points. During full-text reviews, information about the network interventions, actors, ties, and main findings was extracted. Reporting was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA). To develop our typology, we synthesized these results using an iterative team-based and consensus-building process.

RESULTS

Fifty-three articles met the inclusion criteria. The interventions described were conducted in healthcare systems or behavioral health systems (34%), communities (26.4%), and schools (22.6%). The majority included records describing interventions designed to alter social support, information-sharing, or friendship networks (65%) among individual actors (84.9%), or to increase ties. Eight strategies emerged. Three strategies targeted the general context: (1) change the environment, (2) create groups, and (3) change the composition. Four strategies targeted individual actors: change (4) motivations, (5) skills for networking, (6) knowledge of one's social network, and (7) prominence/roles. One strategy (8) targeted specific ties within the network (targeting a particular pair-wise relationship or changing the nature of an existing tie).

CONCLUSION

The network alteration strategies in this typology provide further operational specificity for how implementation strategies target relationships. Advancing these strategies will require greater theoretical specification, the development of strategies that target professionals and organizations, and studies that examine the impact on implementation outcomes.

A policy implementation study of earmarked taxes for mental health services: study protocol

BACKGROUND

Insufficient funding is frequently identified as a critical barrier to the implementation and sustainment of evidence-based practices (EBPs). Thus, increasing access to funding is recognized as an implementation strategy. Policies that create earmarked taxes-defined as taxes for which revenue can only be spent on specific activities-are an increasingly common mental health financing strategy that could improve the reach of EBPs. This project's specific aims are to (1) identify all jurisdictions in the USA that have implemented earmarked taxes for mental health and catalogue information about tax design; (2) characterize experiences implementing earmarked taxes among local (e.g., county, city) mental health agency leaders and other government and community organization officials and assess their perceptions of the acceptability and feasibility of different types of policy implementation strategies; and (3) develop a framework to guide effect earmarked tax designs, inform the selection of implementation strategies, and disseminate the framework to policy audiences.

METHODS

The project uses the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to inform data collection about the determinants and processes of tax implementation and Leeman's typology of implementation strategies to examine the acceptability and feasibility strategies which could support earmarked tax policy implementation. A legal mapping will be conducted to achieve aim 1. To achieve aim 2, a survey will be conducted of 300 local mental health agency leaders and other government and community organization officials involved with the implementation of earmarked taxes for mental health. The survey will be followed by approximately 50 interviews with these officials. To achieve aim 3, quantitative and qualitative data will be integrated through a systematic framework development and dissemination process.

DISCUSSION

This exploratory policy implementation process study will build the evidence base for outer-context implementation determinants and strategies by focusing on policies that earmarked taxes for mental health services.

An Examination of Factors Affecting State Legislators' Support for Parity Laws for Different Mental Illnesses

Mental health parity legislation can improve mental health outcomes. U.S. state legislators determine whether state parity laws are adopted, making it critical to assess factors affecting policy support. This study examines the prevalence and demographic correlates of legislators' support for state parity laws for four mental illnesses- major depression disorder, post-traumatic stress disorder (PTSD), schizophrenia, and anorexia/bulimia. Using a 2017 cross-sectional survey of 475 U.S. legislators, we conducted bivariate analyses and multivariate logistic regression. Support for parity was highest for schizophrenia (57%), PTSD (55%), and major depression (53%) and lowest for anorexia/bulimia (40%). Support for parity was generally higher among females, more liberal legislators, legislators in the Northeast region of the country, and those who had previously sought treatment for mental illness. These findings highlight the importance of better disseminating evidence about anorexia/bulimia and can inform dissemination efforts about mental health parity laws to state legislators.

The Role of Formal Policy to Promote Informed Consent of Psychotropic Medications for Youth in Child Welfare Custody: A National Examination

Active participation of youth and surrogate decision-makers in providing informed consent and assent for mental health treatment is critical. However, the procedural elements of an informed consent process, particularly for youth in child welfare custody, are not well defined. Given calls for psychotropic medication oversight for youth in child welfare custody, this study proposes a taxonomy for the procedural elements of informed consent policies based upon formal and informal child welfare policies and then examines whether enacted state formal policies across the United States endorsed these elements. A sequential multi-method study design included: (1) semi-structured interviews with key informants (n = 58) primarily from state child welfare agencies to identify a taxonomy of procedural elements for informed consent of psychotropic medications and then (2) a legislative review of the 50 states and D.C. to characterize whether formal policies endorsed each procedural element through February 2022. Key informants reported five procedural elements in policy, including how to: (1) gather social and medical history, (2) prescribe the medication, (3) authorize its use through consent and youth assent, (4) notify relevant stakeholders, and (5) routinely review the consenting decision. Twenty-three states endorsed relevant legislation; however, only two states specified all five procedural elements. Additionally, the content of a procedural element, when included, varied substantively across policies. Further research and expert consensus are needed to set best practices and guide policymakers in setting policies to advance transparency and accountability for informed consent of mental health treatment among youth in child welfare custody.

The updated Consolidated Framework for Implementation Research based on user feedback

BACKGROUND

Many implementation efforts fail, even with highly developed plans for execution, because contextual factors can be powerful forces working against implementation in the real world. The Consolidated Framework for Implementation Research (CFIR) is one of the most commonly used determinant frameworks to assess these contextual factors; however, it has been over 10 years since publication and there is a need for updates. The purpose of this project was to elicit feedback from experienced CFIR users to inform updates to the framework.

METHODS

User feedback was obtained from two sources: (1) a literature review with a systematic search; and (2) a survey of authors who used the CFIR in a published study. Data were combined across both sources and reviewed to identify themes; a consensus approach was used to finalize all CFIR updates. The VA Ann Arbor Healthcare System IRB declared this study exempt from the requirements of 38 CFR 16 based on category 2.

RESULTS

The systematic search yielded 376 articles that contained the CFIR in the title and/or abstract and 334 unique authors with contact information; 59 articles included feedback on the CFIR. Forty percent (n = 134/334) of authors completed the survey. The CFIR received positive ratings on most framework sensibility items (e.g., applicability, usability), but respondents also provided recommendations for changes. Overall, updates to the CFIR include revisions to existing domains and constructs as well as the addition, removal, or relocation of constructs. These changes address important critiques of the CFIR, including better centering innovation recipients and adding determinants to equity in implementation.

CONCLUSION

The updates in the CFIR reflect feedback from a growing community of CFIR users. Although there are many updates, constructs can be mapped back to the original CFIR to ensure longitudinal consistency. We encourage users to continue critiquing the CFIR, facilitating the evolution of the framework as implementation science advances.

State Policies that Impact the Design of Children's Mental Health Services: A Modified Delphi Study

To identify the state-level policies and policy domains that state policymakers and advocates perceive as most important for positively impacting the use of children's mental health services (CMHS). We used a modified Delphi technique (i.e., two rounds of questionnaires and an interview) during Spring 2021 to elicit perceptions among state mental health agency officials and advocates (n = 28) from twelve states on state policies that impact the use of CMHS. Participants rated a list of pre-specified policies on a 7-point Likert scale (1 = not important, 7 = extremely important) in the following policy domains: insurance coverage and limits, mental health services, school and social. Participants added nine policies to the initial list of 24 policies. The "school" policy domain was perceived as the most important, while the "social" policy domain was perceived as the least important after the first questionnaire and the second most important policy domain after the second questionnaire. The individual policies perceived as most important were school-based mental health services, state mental health parity, and Medicaid reimbursement rates. Key stakeholders in CMHS should leverage this group of policies to understand the current policy landscape in their state and to identify gaps in policy domains and potential policy opportunities to create a more comprehensive system to address children's mental health from a holistic, evidence-based policymaking perspective.

Attitudes Towards Evidence-Based Practice of Professionals Working with Children and Adolescents with Autism Spectrum Disorder in Bangladesh

Like in many lower-middle-income countries (LMIC), progress in implementing evidence-based practices (EBPs) for children with autism spectrum disorder (ASD) has been slow in Bangladesh. This cross-sectional study examined professionals' attitudes towards evidence-based practice (EBP) for children and adolescents with ASD and explored how providers' demographic factors are related to attitudes to and adoption of EBPs in Bangladesh. The sample consisted of 150 mental health professionals and special teachers from the urban area of Dhaka. Attitudes were assessed by the Evidence-based Practice Attitude Scale-36. Findings indicated that professionals have favorable attitudes towards EBP. Their attitudes varied depending on service settings (public clinical, private clinical, and special school) and caseload per year. Professionals who work in private and special school settings claimed to be more willing to adopt an EBP when required and perceived a higher fit of EBPs and their work than those in public clinical settings. The number of different EBPs used also differed by service setting. Every type of intervention (except medication) was used by more professionals in special schools than in private and public clinical settings. Many professionals reported few barriers to the implementation of EBPs. These findings indicate conditions that are often conducive to the implementation of EBPs. However, these results do not reflect the situation in rural areas, in which poverty is more widespread and the number of specialized professionals is low.

Inter-agency collaboration is associated with increased frequency of research use in children's mental health policy making

OBJECTIVE

To determine whether the self-report frequency of inter-agency collaboration about children's mental health issues is associated with the self-report frequency of using research evidence in children's mental health policy and program decision making in mental health agencies (MHAs).

DATA SOURCES

Primary data were collected through web-based surveys of state (N = 221) and county (N = 117) MHA officials.

DESIGN

The primary independent variable was a composite score quantifying the frequency of collaboration about children's mental health issues between officials in MHAs and six other state agencies. The dependent variables were composite scores quantifying the frequency of research use in children's mental health policy and program decision making in general and for specific purposes (i.e., conceptual, instrumental, tactical, imposed). Covariates were composite scores quantifying well-established determinants of research use (e.g., agency leadership, research use skills) in agency policy and program decision making.

DATA METHODS

Separate multiple linear regression models estimated associations between frequency of inter-agency collaboration and research use scores, adjusting for other determinants of research use, respondent state, and other covariates. Data from state and county officials were analyzed separately.

PRINCIPAL FINDINGS

The frequency of inter-agency collaboration was positively and independently associated with the frequency of research use in children's mental health policy making among state (β = 0.22, p = 0.004) and county (β = 0.39, p < 0.0001) MHA officials. Inter-agency collaboration was also the only variable significantly associated with the frequency of research use for all four specific purposes among state MHA officials, and similar findings we observed among county MHA officials. The magnitudes of associations between inter-agency collaboration and frequency of research use were generally stronger than for more well-established determinants of research use in policy making.

CONCLUSIONS

Strategies that promote collaboration between MHA officials and external agencies could increase the use of research evidence in children's mental health policy and program decision making in MHAs.

Increasing Access to Organization Theories for Implementation Science

Background

Organization theories offer numerous existing, highly relevant, yet largely untapped explanations of the organizational dynamics underlying evidence-based intervention (EBI) implementation. Rooted in ideas regarding power, autonomy, and control, organization theories can explain how and why organizations adopt, implement, and sustain EBI use. Although they have gained visibility, organization theories remain underused in implementation research, perhaps due to their inaccessibility to implementation scientists. To improve access to organization theory among implementation scientists, we summarized organization theories with relevance to implementation science.

Methods

Led by the Cancer Prevention and Control Research Network (CPCRN) Organization Theory for Implementation Science workgroup, we employed a modified Delphi process to reach a consensus among 18 experts at the intersection of organization and implementation science regarding organization theories with relevance to implementation science. From texts that described the organization theories, using standardized abstraction forms, two investigators independently abstracted information regarding constructs, propositions regarding how or why constructs might influence implementation, the potential relevance of organization theories' propositions for implementation, and overviews of each theory. The investigators then reconciled discrepancies until reaching consensus. A third investigator reviewed reconciled abstraction forms for accuracy, coherence, and completeness.

Findings

We identified nine organization theories with relevance to implementation science: contingency, complexity, institutional, network, organizational learning, resource dependence, sociotechnical, and transaction cost economics. From the theories, we abstracted 70 constructs and 65 propositions. An example proposition from institutional theory is: "Coercive, mimetic, and normative pressures contribute to organizations…within an organizational field [becoming increasingly similar]." These propositions can be operationalized as levers to facilitate EBI implementation.

Conclusions

To increase use in the field, organization theories must be made more accessible to implementation scientists. The abstraction forms developed in this study are now publicly available on the CPCRN website with the goal of increasing access to organization theories among an interdisciplinary audience of implementation scientists through the CPCRN Scholars program and other venues. Next steps include consolidating organization theory constructs into domains and translating the resulting framework for use among researchers, policymakers and practitioners, aiding them in accounting for a comprehensive set of organization theory constructs thought to influence EBI implementation.

Substantiating Nexus Between Consumption Values and Sustainable Consumption Behavior: A Way Toward Sustainable Business

The unprecedented economic growth in recent decades has cultivated the exploitation of natural resources and over-consumption, leading to ecological deterioration and sustainability. The ever-increasing consumption in developing countries is creating a significant environmental strain. Thus, the industry and consumers' environmental issues and their harmful effects on human health have led to concerns among researchers, scientists, academic communities, and policymakers. The present work examines the impact of different consumption value factors on sustainable consumption behavior concerning consumer choice in Pakistan and China. A cross-sectional study is conducted, and data are collected through a primary source questionnaire. A sample of 431 respondents is chosen from different cities in Pakistan, and a sample of 342 respondents is selected from China. Estimation techniques like descriptive statistics, frequency distribution, multicollinearity, R square, independent sample t-test, the coefficient of correlation, and regression analysis are used for the data analysis. The comparative results show that knowledge values (KVs) and emotional values (EMVs) significantly influence the choice behavior of respondents toward environmentally friendly products both in Pakistan and China. In contrast, social values (SVs) and conditional values (CVs) show insignificant influence. Furthermore, functional values (FVs) are significant in Pakistan while insignificant in the context of China, and environmental values (EVs) are significant in China although insignificant in Pakistan with regard to sustainable consumption behavior.

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study

BACKGROUND

Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures.

METHOD

We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment.

DISCUSSION

The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.

Policy Makers' Priorities for Addressing Youth Substance Use and Factors That Influence Priorities

OBJECTIVE

Understanding public policy makers' priorities for addressing youth substance use and the factors that influence these priorities can inform the dissemination and implementation of strategies that promote evidence-based decision making. This study characterized the priorities of policy makers in substance use agencies of U.S. states and counties for addressing youth substance use, the factors that influenced these priorities, and the differences in priorities and influences between state and county policy makers.

METHODS

In 2020, a total of 122 substance use agency policy makers from 35 states completed a Web-based survey (response rate=22%). Respondents rated the priority of 14 issues related to youth substance use and the extent to which nine factors influenced these priorities. Data were analyzed as dichotomous and continuous variables and for state and county policy makers together and separately.

RESULTS

The highest priorities for youth substance use were social determinants of substance use (87%), adverse childhood experiences and childhood trauma (85%), and increasing access to school-based substance use programs (82%). The lowest priorities were increasing access to naloxone for youths (49%), increasing access to medications for opioid use disorder among youths (49%), and deimplementing non-evidence-based youth substance use programs (41%). The factors that most influenced priorities were budget issues (80%) and state legislature (69%), federal (67%), and governor priorities (65%). Issues related to program implementation and deimplementation were significantly higher priorities for state than for county policy makers.

CONCLUSIONS

These findings can inform the tailoring of dissemination and implementation strategies to account for the inner- and outer-setting contexts of substance use agencies.

Cocreating evidence-informed health equity policy with community

Objective

To explore the feasibility of a rapid, community‐engaged strategy to prioritize health equity policy options as informed by research evidence, community‐voiced needs, and public health priorities.

Data Sources

Data came from residents in a midsized, demographically, and geographically diverse county over a period of 8 months in 2020 and an evidence review of the health equity policy literature during the same time period.

Study Design

A descriptive case study is used to explore the feasibility and potential value of a community codesigned approach to establish community priorities for health equity policy.

Data Collection/Extraction Methods

Evidence synthesis of health equity policy was conducted parallel to 15 community listening sessions across the county to elicit information on health needs. We used scoping review methods to obtain literature from academic databases and scholarly public health and policy organizations. This information was cross walked with themes from the listening sessions to identify 10 priority policy areas, which were taken back to the community for 15 participatory discussion and ranking sessions.

Principal Findings

The process appeared to authentically engage the input of 200 community members representative of minoritized groups while identifying 99 evidence‐informed policy levers to promote health equity. Discussion and ranking activities were successful in facilitating community discussion and policy decision making. Remote platforms may have limited the engagement of some residents while promoting the participation of others. Conducting information integration within the research team prior to community policy ranking sessions limited the community ownership over how policies were interpreted and communicated.

Conclusions

A combination of information integration and community ranking activities can be used to achieve community‐engaged policy prioritization of options in a fairly rapid period of time. While this process provides an example of authentic community ownership of policy prioritization, the compressed timeline limited the community's engagement in the information integration phase.

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process

BACKGROUND

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services.

METHOD

Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool.

DISCUSSION

This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.

Expanding the reach of evidence-based mental health interventions to private practice: Qualitative assessment using a policy ecology framework

Background

Evidence-based interventions (EBIs) for mental health disorders are underutilized in routine clinical practice. Exposure therapy for anxiety disorders is one particularly difficult-to-implement EBI that has robust empirical support. Previous research has examined EBI implementation determinants in publicly funded mental health settings, but few studies have examined EBI implementation determinants in private practice settings. Private practice clinicians likely face unique barriers to implementation, including setting-specific contextual barriers to EBI use. The policy ecology framework considers broad systemic determinants, including organizational, regulatory, social, and political contexts, which are likely relevant to EBI implementation in private practice settings but have not been examined in prior research.

Methods

Qualitative interviews were conducted to assess private practice clinicians' perceptions of EBI implementation determinants using the policy ecology framework. Clinicians were asked about implementing mental health EBIs broadly and exposure therapy specifically. Mixed methods analyses compared responses from clinicians working in solo vs. group private practice and clinicians who reported high vs. low organizational support for exposure therapy.

Results

Responses highlight several barriers and facilitators to EBI implementation in private practice. Examples include determinants related to organizational support (e.g., colleagues using EBIs), payer restrictions (e.g., lack of reimbursement for longer sessions), fiscal incentives (e.g., payment for attending training), and consumer demand for EBIs. There were notable differences in barriers faced by clinicians who work in group private practices compared to those working in solo practices. Solo private practice clinicians described ways in which their practice setting limits their degree of colleague support (e.g., for consultation or exposure therapy planning), while also allowing for flexibility (e.g., in their schedules and practice location) that may not be available to clinicians in group practice.

Conclusions

Using the policy ecology framework provides a broad understanding of contextual factors that impact private practice clinicians' use of EBIs, including exposure therapy. Findings point to potential implementation strategies that may address barriers that are unique to clinicians working in private practice.

Quantitative measures used in empirical evaluations of mental health policy implementation: A systematic review

Background

Mental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures.

Method

Guided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995-2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale.

Results

We identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n  =  20, 57%) and other resources (n  =  12, 34%), actor relationships/networks (n  =  15, 43%), and organizational culture and climate (n  =  11, 31%). Fidelity was the most prevalent implementation outcome (n  =  9, 26%), followed by penetration (n  =  8, 23%) and acceptability (n  =  7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools.

Conclusions

This work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap.

Plain Language Summary

Mental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.

Value of family involvement in substance use disorder treatment: Aligning clinical and financing priorities

BACKGROUND

Family members' support (e.g., informational, tangible, emotional) has important and lasting impacts on individuals' recovery from substance use disorders (SUDs). Unfortunately, SUD services in the United States do not consistently incorporate patients' family members effectively. One barrier to family involvement in SUD services is the mechanisms through which SUD services in the United States are commonly financed.

METHOD

Using our recent experiences with developing a group intervention for support persons of patients with opioid use disorder, we illustrate how gaps in feasible financing models limit SUD service systems from effectively implementing and sustaining family services for individuals receiving SUD treatment.

DISCUSSION

Long-term availability of family-inclusive interventions will require collaboration with payors and health systems. We offer two sets of recommendations for funding family involvement in SUD services; one set of immediately implementable recommendations and other longer-term goals requiring structural changes in SUD service delivery and financing.

Rapid-cycle systems modeling to support evidence-informed decision-making during system-wide implementation

Background

To “model and simulate change” is an accepted strategy to support implementation at scale. Much like a power analysis can inform decisions about study design, simulation models offer an analytic strategy to synthesize evidence that informs decisions regarding implementation of evidence-based interventions. However, simulation modeling is under-utilized in implementation science. To realize the potential of simulation modeling as an implementation strategy, additional methods are required to assist stakeholders to use models to examine underlying assumptions, consider alternative strategies, and anticipate downstream consequences of implementation. To this end, we propose Rapid-cycle Systems Modeling (RCSM)—a form of group modeling designed to promote engagement with evidence to support implementation. To demonstrate its utility, we provide an illustrative case study with mid-level administrators developing system-wide interventions that aim to identify and treat trauma among children entering foster care.

Methods

RCSM is an iterative method that includes three steps per cycle: (1) identify and prioritize stakeholder questions, (2) develop or refine a simulation model, and (3) engage in dialogue regarding model relevance, insights, and utility for implementation. For the case study, 31 key informants were engaged in step 1, a prior simulation model was adapted for step 2, and six member-checking group interviews (n = 16) were conducted for step 3.

Results

Step 1 engaged qualitative methods to identify and prioritize stakeholder questions, specifically identifying a set of inter-related decisions to promote implementing trauma-informed screening. In step 2, the research team created a presentation to communicate key findings from the simulation model that addressed decisions about programmatic reach, optimal screening thresholds to balance demand for treatment with supply, capacity to start-up and sustain screening, and availability of downstream capacity to provide treatment for those with indicated need. In step 3, member-checking group interviews with stakeholders documented the relevance of the model results to implementation decisions, insight regarding opportunities to improve system performance, and potential to inform conversations regarding anticipated implications of implementation choices.

Conclusions

By embedding simulation modeling in a process of stakeholder engagement, RCSM offers guidance to realize the potential of modeling not only as an analytic strategy, but also as an implementation strategy.

Lessons From Maslow: Prioritizing Funding to Improve the Quality of Community Mental Health and Substance Use Services

Evidence-based practices (EBPs) are frequently proposed as an approach to close the quality chasm in behavioral health treatment, and many U.S. municipalities are investing in EBPs as a primary way to improve the quality of care delivered to individuals most in need. In this Open Forum, the authors argue that EBPs often cannot be successfully implemented because basic organizational needs are not met in the current fiscal environment. The authors summarize research that supports why EBPs, along with other approaches to improve quality, are likely to fail until there is adequate financing. They also propose a policy and research agenda to ameliorate and address the fiscal challenges inherent in community mental health and substance use services.

Study protocol for a type III hybrid effectiveness-implementation trial of strategies to implement firearm safety promotion as a universal suicide prevention strategy in pediatric primary care

Background

Insights from behavioral economics, or how individuals’ decisions and behaviors are shaped by finite cognitive resources (e.g., time, attention) and mental heuristics, have been underutilized in efforts to increase the use of evidence-based practices in implementation science. Using the example of firearm safety promotion in pediatric primary care, which addresses an evidence-to-practice gap in universal suicide prevention, we aim to determine: is a less costly and more scalable behavioral economic-informed implementation strategy (i.e., “Nudge”) powerful enough to change clinician behavior or is a more intensive and expensive facilitation strategy needed to overcome implementation barriers?

Methods

The Adolescent and child Suicide Prevention in Routine clinical Encounters (ASPIRE) hybrid type III effectiveness-implementation trial uses a longitudinal cluster randomized design. We will test the comparative effectiveness of two implementation strategies to support clinicians’ use of an evidence-based firearm safety practice, S.A.F.E. Firearm, in 32 pediatric practices across two health systems. All pediatric practices in the two health systems will receive S.A.F.E. Firearm materials, including training and cable locks. Half of the practices (k = 16) will be randomized to receive Nudge; the other half (k = 16) will be randomized to receive Nudge plus 1 year of facilitation to target additional practice and clinician implementation barriers (Nudge+). The primary implementation outcome is parent-reported clinician fidelity to the S.A.F.E Firearm program. Secondary implementation outcomes include reach and cost. To understand how the implementation strategies work, the primary mechanism to be tested is practice adaptive reserve, a self-report practice-level measure that includes relationship infrastructure, facilitative leadership, sense-making, teamwork, work environment, and culture of learning.

Discussion

The ASPIRE trial will integrate implementation science and behavioral economic approaches to advance our understanding of methods for implementing evidence-based firearm safety promotion practices in pediatric primary care. The study answers a question at the heart of many practice change efforts: which strategies are sufficient to support change, and why? Results of the trial will offer valuable insights into how best to implement evidence-based practices that address sensitive health matters in pediatric primary care.

Trial registration

ClinicalTrials.gov, NCT04844021. Registered 14 April 2021.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01154-8.

Youth Mental Health First Aid: Examining the Influence of Pre-Existing Attitudes and Knowledge on Training Effectiveness

Youth Mental Health First Aid (YMHFA) is a gatekeeper training designed to assist professionals in their identification and response to youth mental illness. Despite widespread implementation, few studies have examined how individual-level trainee characteristics may impact the effectiveness of this training. In this study, we examined how pre-existing knowledge and attitudes about youth mental illness (measured prior to participation in the program) influenced training outcomes. We used a sample of 987 child-serving professionals who completed the YMHFA training program to identify demographic and work-related factors that predicted pre-existing attitudes and knowledge. The sample was then divided into two groups based on pre-existing attitudes and knowledge. We examined training effectiveness across four constructs: knowledge, confidence, intentions to intervene, and stigma. Findings indicate that gender and work-related characteristics were associated with pre-existing knowledge and attitudes about youth mental illness. The magnitude of training effectiveness varied across pre-existing attitudes and knowledge. Those participants with above average attitudes and knowledge benefited little from the program, suggesting that universal implementation of YMHFA may not be an efficient use of resources. Suggestions for a tiered implementation plan that prioritizes professionals most in need of the training are discussed.

The Organizational Financial Context of Publicly-Funded Mental Health Clinics: Development and Preliminary Psychometric Evaluation of the Agency Financial Status Scales

Funding is a major barrier to implementation of evidence-based practices (EBPs) in publicly-funded community mental health clinics (CMHCs). Understanding how best to deploy implementation strategies that address this barrier requires greater clarity on the financial context within agencies. We developed the Agency Financial Status Scales (AFSS) to assess employee perceptions of the level of three hypothesized and theoretical funding related constructs in organizations: (a) perceptions of financial health, (b) financial attitudes toward EBPs, and (c) strategic financial climate. This investigation serves as a preliminary evaluation of this measure. Participants were 239 therapists and 40 supervisors from 25 publicly-funded CMHCs providing outpatient mental health services for young people. Confirmatory factor analysis was used to investigate the latent trait structure of the items. Internal consistency, interrater agreement, concordance between therapists and supervisors, and convergent validity were also examined. A two-factor model measuring perceptions of financial health and strategic financial climate best fit the data. For both of these scales, alpha reliability was acceptable and agreement statistics provided moderate support for aggregation at the organizational level. Analyses supported the convergent validity of the scales. The development and preliminary evaluation of the AFSS is an important first step in understanding the financial context of publicly-funded CMHCs. Though findings from this investigation are promising, additional development and testing are needed to develop a more thorough understanding of the constructs and to improve the validity and reliability of this measure.

Advancing health equity through a theoretically critical implementation science

While implementation science is driven by theory, most implementation science theories, models, and frameworks (TMF) do not address issues of power, inequality, and reflexivity that are pivotal to achieving health equity. Theories used in anthropology address these issues effectively and could complement prevailing implementation science theories and constructs. We propose three broad areas of theory that complement and extend existing TMF in implementation science to advance health equity. First, theories of postcoloniality and reflexivity foreground attention to the role of power in knowledge production and to the ways that researchers and interventionists may perpetuate the inequalities shaping health. Second, theories of structural violence and intersectionality can help us to better understand the unequal burden of health disparities in the population, thereby encouraging researchers to think beyond single interventions to initiate partnerships that can impact overlapping health vulnerabilities and influence the upstream causes of vulnerability. Finally, theories of policy and governance encourage us to examine the social-political forces of the "outer context" crucial for implementation and sustainability. The incorporation of critical theories could enhance implementation science and foster necessary reflexivity among implementation scientists. We contend that a theoretically critical implementation science will promote better science and, more importantly, support progress toward health equity.

Exploring the heterogeneity of factors that may influence implementation of PrEP in family planning clinics: a latent profile analysis

BACKGROUND

Title X-funded family planning clinics have been identified as optimal sites for delivery of pre-exposure prophylaxis (PrEP) for HIV prevention. However, PrEP has not been widely integrated into family planning services, especially in the Southern US, and data suggest there may be significant implementation challenges in this setting. Because Title X clinics vary greatly in provider-, organizational-, and systems-level characteristics, there is likely variation in capacity to implement PrEP across clinics.

METHODS

We conducted a survey from February to June 2018 among providers and administrators of non-PrEP-providing Title X-funded clinics across 18 southern states. Survey items were designed using the Consolidated Framework for Implementation Research (CFIR) to assess constructs relevant to PrEP implementation. To explore the heterogeneity of CFIR-related implementation determinants and identify distinct sub-groups of Title X clinics, a latent profile analysis was conducted using nine CFIR constructs: complexity, relative advantage, cost, attitudes, implementation climate, compatibility, leadership engagement, available resources, and cosmopolitanism. We then conducted a multi-level analysis (accounting for nesting of participants within clinics) to test whether group membership was associated with readiness for implementation of PrEP, controlling for key sociodemographic characteristics.

RESULTS

Four hundred and fourteen healthcare providers/administrators from 227 non-PrEP-providing Title X clinics participated in the study. We identified six sub-groups of clinics that each had distinct patterns of PrEP implementation determinants. Clinic sub-groups included "Highest Capacity for Implementation", "Favorable Conditions for Implementation", "Mixed Implementation Context", "Neutral Implementation Context", "Incompatible Setting for Implementation", and "Resource-Strained Setting". Group membership was related to numerous provider-level (i.e., ability to prescribe medication) and clinic-level (i.e., provision of primary care) characteristics. In comparison to the "Neutral" group (which held neutral perceptions across the implementation determinants), the "Highest Capacity" and "Favorable Conditions" groups had significantly higher levels of implementation readiness, and the "Resource-Strained" group had a significantly lower level of implementation readiness.

CONCLUSIONS

Latent profile analyses can help researchers understand how implementation readiness varies across healthcare settings, promoting tailoring of implementation strategies to unique contexts.

Dissemination Science in School Mental Health: A Framework for Future Research

There has been an increase in school mental health research aimed at producing generalizable knowledge to address longstanding science-to-practice gaps to increase children's access to evidence-based mental health services. Successful dissemination and implementation are both important pieces to address science-to-practice gaps, but there is conceptual and semantic imprecision that creates confusion regarding where dissemination ends and implementation begins, as well as an imbalanced focus in research on implementation relative to dissemination. In this paper, we provide an enhanced operational definition of dissemination; offer a conceptual model that outlines elements of effective dissemination that can produce changes in awareness, knowledge, perceptions, and motivation across different stakeholder groups; and delineate guiding principles that can inform dissemination science and practice. The overarching goal of this paper is to stimulate future research that aims to advance dissemination science and practice in school mental health.

Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment

BACKGROUND

Bridging factors are relational ties, formal arrangements, and processes that connect outer system and inner organizational contexts. They may be critical drivers of evidence-based practice (EBP) implementation and sustainment. Yet, the complex interplay between outer and inner contexts is often not considered. Bridging factors were recently defined in the updated Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Further identification and specification of this construct will advance implementation models, measures, and methods. Our goal is to advance bridging factor research by identifying relevant dimensions and exemplifying these dimensions through illustrative case studies.

METHODS

We used a multiple case study design. Each case (n = 10) represented different contexts, EBPs, and bridging factor types. Inclusion criteria were the presence of clearly distinguishable outer and inner contexts, identifiable bridging factor, sufficient information to describe how the bridging factor affected implementation, and variation from other cases. We used an iterative qualitative inquiry process to develop and refine a list of dimensions. Case data were entered into a matrix. Dimensions comprised the rows and case details comprised the columns. After a review of all cases, we collectively considered and independently coded each dimension as function or form.

RESULTS

We drew upon the concepts of functions and forms, a distinction originally proposed in the complex health intervention literature. Function dimensions help define the bridging factor and illustrate its purpose as it relates to EBP implementation. Form dimensions describe the specific structures and activities that illustrate why and how the bridging factor has been customized to a local implementation experience. Function dimensions can help researchers and practitioners identify the presence and purpose of bridging factors, whereas form dimensions can help us understand how the bridging factor may be designed or modified to support EBP implementation in a specific context. We propose five function and three form bridging factor dimensions.

CONCLUSIONS

Bridging factors are described in many implementation models and studies, but without explicit reference or investigation. Bridging factors are an understudied and critical construct that requires further attention to facilitate implementation research and practice. We present specific recommendations for a bridging factors research agenda.

Funding Diversity: A Case Study of A State-initiated, Funding-driven Program to Diversify Mental Health Service Provision in Minnesota

Human service organizations are reconfiguring to address diversifying populations and widening inequality. However, institutional change is challenging to implement and fund; resource scarcity and stakeholder buy-in are barriers. In this case study, we analyze a funding-driven, state-initiated program that supports mental health professionals who are people of color in order to decrease health disparities. Analyses of interviews and documents depict how the program struggled with high turnover and uninspired, halfhearted messaging, but was nevertheless well loved. Findings illustrate how the 'pitch' and leadership matter in programming for institutional change and its contested nature, a contestation that funding alone cannot temper.

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

BACKGROUND

The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policymakers, researchers, and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts.

METHODS

We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: How is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as gray literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature.

RESULTS

A total of 11,434 documents were retrieved and assessed for eligibility and 35 additional documents were identified through other sources. Eighty-six unique documents were ultimately included in the analysis. Our findings indicate that policy is described as (1) the context, (2) a focusing lens, (3) the innovation itself, (4) a lever of influence, (5) an enabler/facilitator or barrier, or (6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components.

CONCLUSIONS

This framework begins to bridge the divide between disciplines and provides a new perspective about implementation processes at the systems level. It offers researchers, policymakers, and implementers a new way of thinking about implementation that better integrates policy considerations and can be used for planning or evaluating implementation efforts.

Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis

BACKGROUND

The fields of implementation science and knowledge translation have evolved somewhat independently from the field of policy implementation research, despite calls for better integration. As a result, implementation theory and empirical work do not often reflect the implementation experience from a policy lens nor benefit from the scholarship in all three fields. This means policymakers, researchers, and practitioners may find it challenging to draw from theory that adequately reflects their implementation efforts.

METHODS

We developed an integrated theoretical framework of the implementation process from a policy perspective by combining findings from these fields using the critical interpretive synthesis method. We began with the compass question: How is policy currently described in implementation theory and processes and what aspects of policy are important for implementation success? We then searched 12 databases as well as gray literature and supplemented these documents with other sources to fill conceptual gaps. Using a grounded and interpretive approach to analysis, we built the framework constructs, drawing largely from the theoretical literature and then tested and refined the framework using empirical literature.

RESULTS

A total of 11,434 documents were retrieved and assessed for eligibility and 35 additional documents were identified through other sources. Eighty-six unique documents were ultimately included in the analysis. Our findings indicate that policy is described as (1) the context, (2) a focusing lens, (3) the innovation itself, (4) a lever of influence, (5) an enabler/facilitator or barrier, or (6) an outcome. Policy actors were also identified as important participants or leaders of implementation. Our analysis led to the development of a two-part conceptual framework, including process and determinant components.

CONCLUSIONS

This framework begins to bridge the divide between disciplines and provides a new perspective about implementation processes at the systems level. It offers researchers, policymakers, and implementers a new way of thinking about implementation that better integrates policy considerations and can be used for planning or evaluating implementation efforts.

Accelerating the conceptual use of behavioral health research in juvenile court decision-making: study protocol

BACKGROUND

The youth criminal-legal system is under heavy political scrutiny with multiple calls for significant transformation. Leaders within the system are faced with rethinking traditional models and are likely to benefit from behavioral health research evidence as they redesign systems. Little is known about how juvenile court systems access and use behavioral health research evidence; further, the field lacks a validated survey measure of behavioral health research use that can be used to evaluate the effectiveness of evidence dissemination interventions for policy and system leaders. Conceptual research use is a particularly salient construct for system reform as it describes the process of shifting awareness and the consideration of new frameworks for action. A tool designed to measure the conceptual use of behavioral health research would advance the field's ability to develop effective models of research evidence dissemination, including collaborative planning models to support the use of behavioral health research in reforms of the criminal-legal system.

METHODS

The ARC Study is a longitudinal, cohort and measurement validation study. It will proceed in two phases. The first phase will focus on measure development using established methods of construct validity (theoretical review, Delphi methods for expert review, cognitive interviewing). The second phase will involve gathering responses from the developed survey to examine scale psychometrics using Rasch analyses, change sensitivity analyses, and associations between research use exposure and conceptual research use among juvenile court leaders. We will recruit juvenile court leaders (judges, administrators, managers, supervisors) from 80 juvenile court jurisdictions with an anticipated sample size of n = 520 respondents.

DISCUSSION

The study will introduce a new measurement tool for the field that will advance implementation science methods for the study of behavioral health research evidence use in complex policy and decision-making interventions. To date, there are few validated survey measures of conceptual research use and no measures that are validated for measuring change in conceptual frameworks over time among agency leaders. While the study is most directly related to leaders in the youth criminal-legal system, the findings are expected to be informative for research focused on leadership and decision-making in diverse fields.

Using behavioral insights to design implementation strategies in public mental health settings: a qualitative study of clinical decision-making

BACKGROUND

Trauma-focused cognitive behavioral therapy (TF-CBT) is an evidence-based intervention for youth with posttraumatic stress disorder. An important component of TF-CBT is the trauma narrative (TN), a phase in the intervention in which youth are guided to process the memories, thoughts, and feelings associated with their traumatic experience(s). Previous work has shown that TF-CBT clinicians complete TNs with only half of their clients, yet little is known about what determines TF-CBT clinicians' use of TNs. The behavioral insights literature-an interdisciplinary field studying judgment and decision-making-offers theoretical and empirical tools to conceptualize what drives complex human behaviors and decisions. Drawing from the behavioral insights literature, the present study seeks to understand what determines clinician use of TNs and to generate strategies that target these determinants.

METHODS

Through semi-structured qualitative interviews, we sought the perspectives of trained TF-CBT clinicians working in public mental health settings across the city of Philadelphia (N = 17) to understand their decisions to use TNs with clients. We analyzed the qualitative data using a coding approach informed by the behavioral insights literature. We used an iterative process of structured hypothesis generation, aided by a behavioral insights guide, and rapid validation informed by behavioral insights to uncover the determinants of TN use. We then generated implementation strategies that targeted these determinants using the "Easy Attractive Social Timely" framework, a behavioral insights design approach.

RESULTS

We generated and validated three broad themes about what determines clinician implementation of TNs: decision complexity, clinician affective experience, and agency norms. We hypothesized behavioral insights that underlie these implementation determinants and designed a list of nine corresponding behavioral insights strategies that may facilitate TN implementation.

CONCLUSIONS

Our study investigated why an effective component of an evidence-based intervention is difficult to implement. We leveraged robust scientific theories and empirical regularities from the behavioral insights literature to understand clinician perspectives on TN implementation. These factors were theoretically linked to implementation strategies. Our work revealed the potential for using behavioral insights in the diagnosis of evidence-based intervention determinants and the design of implementation strategies.