Aligning dissemination and implementation science with health policies to improve children's mental health

Systematic Review and Meta-Analysis of Policies and Interventions that Improve Health, Psychosocial, and Economic Outcomes for Young People Leaving the Out-of-Home Care System

Young people who transition to adulthood from out-of-home care (OOHC) are more likely to experience a range of poorer outcomes relative to their same-age peers in the community. This systematic review assessed the effectiveness of policies or interventions (hereafter "interventions") aimed at improving housing, health, education, economic, and psychosocial outcomes for youth leaving OOHC (hereafter "care leavers"). Eleven databases of published literature were reviewed along with gray literature. Eligible studies used randomized or quasi-experimental designs and assessed interventions that provided support to care leavers prior to, during, or after they left OOHC. Primary outcomes were housing and homelessness, health and well-being, education, economic and employment, criminal and delinquent behavior, and risky behavior, while secondary outcomes were supportive relationships and life skills. Where possible, results were pooled in a meta-analysis. Certainty of evidence was assessed using Grading of Recommendations Assessment, Development and Evaluation. Fourteen studies published in 27 reports were identified that examined independent living programs (ILPs) (n = 5), intensive support services (n = 2), coaching and peer support (C&PSP) (n = 2), transitional housing (n = 1), health information or coaching (n = 2), and extended care (n = 2). All but one study was conducted in the United States. Twenty small meta-analyses were undertaken encompassing ILPs and C&PSP, with two showing results that favored the intervention with certainty. The level of confidence in each meta-analysis was considered very low. A significant risk of bias was identified in each of the included studies. While some interventions showed promise, particularly extended care, the scope and strength of included evidence is insufficient to recommend any included approach.

Using policy codesign to achieve multi-sector alignment in adolescent behavioral health: a study protocol

BACKGROUND

Policymaking is quickly gaining focus in the field of implementation science as a potential opportunity for aligning cross-sector systems and introducing incentives to promote population health, including substance use disorders (SUD) and their prevention in adolescents. Policymakers are seen as holding the necessary levers for realigning service infrastructure to more rapidly and effectively address adolescent behavioral health across the continuum of need (prevention through crisis care, mental health, and SUD) and in multiple locations (schools, primary care, community settings). The difficulty of aligning policy intent, policy design, and successful policy implementation is a well-known challenge in the broader public policy and public administration literature that also affects local behavioral health policymaking. This study will examine a blended approach of coproduction and codesign (i.e., Policy Codesign), iteratively developed over multiple years to address problems in policy formation that often lead to poor implementation outcomes. The current study evaluates this scalable approach using reproducible measures to grow the knowledge base in this field of study.

METHODS

This is a single-arm, longitudinal, staggered implementation study to examine the acceptability and short-term impacts of Policy Codesign in resolving critical challenges in behavioral health policy formation. The aims are to (1) examine the acceptability, feasibility, and reach of Policy Codesign within two geographically distinct counties in Washington state, USA; (2) examine the impact of Policy Codesign on multisector policy development within these counties using social network analysis; and (3) assess the perceived replicability of Policy Codesign among leaders and other staff of policy-oriented state behavioral health intermediary organizations across the USA.

DISCUSSION

This study will assess the feasibility of a specific approach to collaborative policy development, Policy Codesign, in two diverse regions. Results will inform a subsequent multi-state study measuring the impact and effectiveness of this approach for achieving multi-sector and evidence informed policy development in adolescent SUD prevention and treatment.

Acceptability and feasibility of policy implementation strategies for taxes earmarked for behavioral health services

Background

This study's aims are to: (1) Compare the acceptability and feasibility of five types of implementation strategies that could be deployed to increase the reach of evidence-based practices (EBPs) with revenue from policies that earmark taxes for behavioral health services, and (2) Illustrate how definitions of implementation strategies and measures of acceptability and feasibility can be used in policy-focused implementation science research.

Methods

Web-based surveys of public agency and community organization professionals involved with earmarked tax policy implementation were completed in 2022-2023 (N = 211, response rate = 24.9%). Respondents rated the acceptability and feasibility of five types of implementation strategies (dissemination, implementation process, integration, capacity-building, and scale-up). Aggregate acceptability and feasibility scores were calculated for each type of strategy (scoring range 4-20). Analyses of variance compared scores across strategies and between organizational actor types.

Findings

For acceptability, capacity-building strategies had the highest rating (M = 16.3, SD = 3.0), significantly higher than each of the four other strategies, p ≤ . 004), and scale-up strategies had the lowest rating (M = 15.6). For feasibility, dissemination strategies had the highest rating (M = 15.3, significantly higher than three of the other strategies, p ≤ .002) and scale-up strategies had the lowest rating (M = 14.4).

Conclusions

Capacity-building and dissemination strategies may be well-received and readily deployed by policy implementers to support EBPs implementation with revenue from taxes earmarked for behavioral health services. Adapting definitions of implementation strategies for policy-focused topics, and applying established measures of acceptability and feasibility to these strategies, demonstrates utility as an approach to advance research on policy-focused implementation strategies.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science

Policy implementation is a key component of scaling effective chronic disease prevention and management interventions. Policy can support scale-up by mandating or incentivizing intervention adoption, but enacting a policy is only the first step. Fully implementing a policy designed to facilitate implementation of health interventions often requires a range of accompanying implementation structures, like health IT systems, and implementation strategies, like training. Decision makers need to know what policies can support intervention adoption and how to implement those policies, but to date research on policy implementation is limited and innovative methodological approaches are needed. In December 2021, the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness and the Johns Hopkins Center for Mental Health and Addiction Policy convened a forum of research experts to discuss approaches for studying policy implementation. In this report, we summarize the ideas that came out of the forum. First, we describe a motivating example focused on an Affordable Care Act Medicaid health home waiver policy used by some US states to support scale-up of an evidence-based integrated care model shown in clinical trials to improve cardiovascular care for people with serious mental illness. Second, we define key policy implementation components including structures, strategies, and outcomes. Third, we provide an overview of descriptive, predictive and associational, and causal approaches that can be used to study policy implementation. We conclude with discussion of priorities for methodological innovations in policy implementation research, with three key areas identified by forum experts: effect modification methods for making causal inferences about how policies' effects on outcomes vary based on implementation structures/strategies; causal mediation approaches for studying policy implementation mechanisms; and characterizing uncertainty in systems science models. We conclude with discussion of overarching methods considerations for studying policy implementation, including measurement of policy implementation, strategies for studying the role of context in policy implementation, and the importance of considering when establishing causality is the goal of policy implementation research.

Enhancing Impact: A Call to Action for Equitable Implementation Science

Despite investments in evidence-based interventions and Implementation Science, most evidence-based interventions are not widely or routinely adopted, delivered, or sustained in many real-world community and healthcare settings. This gap is even greater in settings and populations experiencing numerous social and structural barriers to health, with important implications for persistent patterns in health inequities. In this Viewpoint, as part of a Special Issue on Advancing the Adaptability of Chronic Disease Prevention and Management through Implementation Science, we outline seven calls to action for the field of Implementation Science, with the goal of encouraging researchers, practitioners, and funders to be more intentional and accountable in applying Implementation Science to have greater impact on promoting health equity. Calls to action include (1) enhance public health, community, and multi-sectoral partnerships to promote health equity and equitable implementation; (2) revisit and build the evidence base needed to promote health equity and impact at multiple levels; (3) prioritize focus on policy development, dissemination, and implementation; (4) be agile and responsive in application of Implementation Science frameworks, processes, and methods; (5) identify and redefine meaningful metrics for equity and impact; (6) disseminate scientific evidence and research to a diverse range of partners and potential beneficiaries; and (7) extend focus on de-implementation, mis-implementation, and sustainability which are central to enhancing health equity. Additionally, we outline why a focus on prevention and public health is essential to making progress towards health equity in Implementation Science, summarize important advancements that the field has made towards making equity more foundational, and pose important research questions to enhance equitable impact of work in this area.

Behavioral Health Services Outcomes That Matter Most to Caregivers of Children, Youth, and Young Adults with Mental Health Needs

This project documents the service outcomes that caregivers value most. A diverse group of caregivers, representing six regions of the United States, participated in two rounds of virtual one-hour focus groups. In round 1, participants identified what they hoped to gain from using behavioral health services for themselves, their families, and their child and discussed what made services a positive experience for them. They then reported their top-three most-hoped-for outcomes. In round 2, groups validated and refined summary findings from round 1. Caregivers prioritized service quality outcomes, primarily. They expressed a desire for an accessible, respectful, and supportive treatment environment, underpinned by well-trained and culturally responsive professionals. Caregivers also desire seamless cross-sector provider collaboration and care transitions, which integrate the insights and preferences of families and children themselves to craft a customized care plan. Priority outcomes not related to service quality included hoping to gain increased knowledge, resources, and tools and techniques to support the mental health needs of their children, to see their children improve their daily functioning and for their child develop more effective interpersonal communication skills. Caregivers also reported hoping to experience less stigma related to the mental health needs of their children and to achieve personal fulfillment for themselves and their children. Research, policies, and mental health services should prioritize and be designed to address the outcomes that matter to youth and families.

Revisiting the policy ecology framework for implementation of evidence-based practices in mental health settings

BACKGROUND

Over the past three decades, policy actors and actions have been highly influential in supporting the implementation of evidence-based practices (EBPs) in mental health settings. An early examination of these actions resulted in the Policy Ecology Framework (PEF), which was originally developed as a tactical primer for state and local mental health regulators in the field of child mental health. However, the policy landscape for implementation has evolved significantly since the original PEF was published. An interrogation of the strategies originally proposed in the PEF is necessary to provide an updated menu of strategies to improve our understanding of the mechanisms of policy action and promote system improvement.

OBJECTIVES

This paper builds upon the original PEF to address changes in the policy landscape for the implementation of mental health EBPs between 2009 and 2022. We review the current state of policy strategies that support the implementation of EBPs in mental health care and outline key areas for policy-oriented implementation research. Our review identifies policy strategies at federal, state, agency, and organizational levels, and highlights developments in the social context in which EBPs are implemented. Furthermore, our review is organized around some key changes that occurred across each PEF domain that span organizational, agency, political, and social contexts along with subdomains within each area.

DISCUSSION

We present an updated menu of policy strategies to support the implementation of EBPs in mental health settings. This updated menu of strategies considers the broad range of conceptual developments and changes in the policy landscape. These developments have occurred across the organizational, agency, political, and social contexts and are important for policymakers to consider in the context of supporting the implementation of EBPs. The updated PEF expands and enhances the specification of policy levers currently available, and identifies policy targets that are underdeveloped (e.g., de-implementation and sustainment) but are becoming visible opportunities for policy to support system improvement. The updated PEF clarifies current policy efforts within the field of implementation science in health to conceptualize and better operationalize the role of policy in the implementation of EBPs.

Strategies for enacting health policy codesign: a scoping review and direction for research

BACKGROUND

Strategies for supporting evidence-informed health policy are a recognized but understudied area of policy dissemination and implementation science. Codesign describes a set of strategies potentially well suited to address the complexity presented by policy formation and implementation. We examine the health policy literature describing the use of codesign in initiatives intended to combine diverse sources of knowledge and evidence in policymaking.

METHODS

The search included PubMed, MEDLINE, PsychInfo, CINAHL, Web of Science, and Google Scholar in November 2022 and included papers published between 1996 and 2022. Terms included codesign, health, policy, and system terminology. Title and abstracts were reviewed in duplicate and included if efforts informed policy or system-level decision-making. Extracted data followed scoping review guidelines for location, evaluation method, health focus, codesign definition, description, level of health system user input, sectors involved, and reported benefits and challenges.

RESULTS

From 550 titles, 23 citations describing 32 policy codesign studies were included from multiple continents (Australia/New Zealand, 32%; UK/Europe, 32%; South America, 14%; Africa, 9%; USA/Canada 23%). Document type was primarily case study (77%). The area of health focus was widely distributed. Policy type was more commonly little p policy (47%), followed by big p policy (25%), and service innovations that included policy-enabled funding (25%). Models and frameworks originated from formal design (e.g., human-centered or participatory design (44%), political science (38%), or health service research (16%). Reported outcomes included community mobilization (50%), policy feasibility (41%), improved multisector alignment (31%), and introduction of novel ideas and critical thinking (47%). Studies engaging policy users in full decision-making roles self-reported higher levels of community mobilization and community needs than other types of engagement.

DISCUSSION

Policy codesign is theoretically promising and is gaining interest among diverse health sectors for addressing the complexity of policy formation and implementation. The maturity of the science is just emerging. We observed trends in the association of codesign strategies and outcomes that suggests a research agenda in this area could provide practical insights for tailoring policy codesign to respond to local contextual factors including values, needs, and resources.

Who's "in the room where it happens"? A taxonomy and five-step methodology for identifying and characterizing policy actors

BACKGROUND

Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in "Big P" public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions-such as adopting evidence-based interventions at systems-scale-and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers.

METHODS

A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study-The Family First Prevention Services Act (FFPSA)-publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the "room" during policy decisions.

RESULTS

The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking "rooms," (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the "room," and (5) quantify (e.g., count actors across groups), summarize, and compare "rooms" to develop or select engagement approaches aligned with the "room" and actors. The use and outcomes of each step are exemplified through the FFPSA case study.

CONCLUSIONS

The pragmatic and transparent policy actor identification steps presented here can guide researchers' methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both "Big P" and "little p" (administrative guidelines, procedures) policymaking and implementation in global contexts.

Four very basic ways to think about policy in implementation science

BACKGROUND

Policy is receiving increasing attention in the field of implementation science. However, there remains a lack of clear, concise guidance about how policy can be conceptualized in implementation science research. Building on Curran's article "Implementation science made too simple"-which defines "the thing" as the intervention, practice, or innovation in need of implementation support-we offer a typology of four very basic ways to conceptualize policy in implementation science research. We provide examples of studies that have conceptualized policy in these different ways and connect aspects of the typology to established frameworks in the field. The typology simplifies and refines related typologies in the field. Four very basic ways to think about policy in implementation science research. 1) Policy as something to adopt: an evidence-supported policy proposal is conceptualized as "the thing" and the goal of research is to understand how policymaking processes can be modified to increase adoption, and thus reach, of the evidence-supported policy. Policy-focused dissemination research is well-suited to achieve this goal. 2) Policy as something to implement: a policy, evidence-supported or not, is conceptualized as "the thing" and the goal of research is to generate knowledge about how policy rollout (or policy de-implementation) can be optimized to maximize benefits for population health and health equity. Policy-focused implementation research is well-suited to achieve this goal. 3) Policy as context to understand: an evidence-supported intervention is "the thing" and policies are conceptualized as a fixed determinant of implementation outcomes. The goal of research is to understand the mechanisms through which policies affect implementation of the evidence-supported intervention. 4) Policy as strategy to use: an evidence-supported intervention is "the thing" and policy is conceptualized as a strategy to affect implementation outcomes. The goal of research is to understand, and ideally test, how policy strategies affect implementation outcomes related to the evidence-supported intervention.

CONCLUSION

Policy can be conceptualized in multiple, non-mutually exclusive ways in implementation science. Clear conceptualizations of these distinctions are important to advancing the field of policy-focused implementation science and promoting the integration of policy into the field more broadly.

Study protocol for a multi-level cross-sectional study on the equitable reach and implementation of coordinated specialty care for early psychosis

BACKGROUND

Approximately 115,000 young adults will experience their first episode of psychosis (FEP) each year in the USA. Coordinated specialty care (CSC) for early psychosis is an evidence-based early intervention model that has demonstrated effectiveness by improving quality of life and reducing psychiatric symptoms for many individuals. Over the last decade, there has significant increase in the implementation of CSC programs throughout the USA. However, prior research has revealed difficulties among individuals and their family members accessing CSC. Research has also shown that CSC programs often report the limited reach of their program to underserved populations and communities (e.g., ethnoracial minorities, rural and low socioeconomic neighborhoods). Dissemination and implementation research focused on the equitable reach and implementation of CSC is needed to address disparities at the individual level.

METHODS

The proposed study will create a novel integrative multi-level geospatial database of CSC programs implemented throughout the USA that will include program-level data (e.g., geocoded location, capacity, setting, role availability), provider-level data (race, ethnicity, professional credentials), and neighborhood-level census data (e.g., residential segregation, ethnic density, area deprivation, rural-urban continua, public transit time). This database will be used to characterize variations in CSC programs by geographical location and examine the overall reach CSC programs to specific communities. The quantitative data will be combined with qualitative data from state administrators, providers, and service users that will inform the development of dissemination tools, such as an interactive dashboard, that can aid decision making.

DISCUSSION

Findings from this study will highlight the impact of outer contextual determinants on implementation and reach of mental health services, and will serve to inform the future implementation of CSC programs with a primary focus on equity.

A landscape assessment of the activities and capacities of evidence-to-policy intermediaries (EPI) in behavioral health

BACKGROUND

A significant gap exists between the production of research evidence and its use in behavioral health policymaking. Organizations providing consulting and support activities for improving policy represent a promising source for strengthening the infrastructure to address this gap. Understanding the characteristics and activities of these evidence-to-policy intermediary (EPI) organizations can inform the development of capacity-building activities, leading to strengthened evidence-to-policy infrastructure and more widespread evidence-based policymaking.

METHODS

Online surveys were sent to 51 organizations from English-speaking countries involved in evidence-to-policy activities in behavioral health. The survey was grounded in a rapid evidence review of the academic literature regarding strategies used to influence research use in policymaking. The review identified 17 strategies, which were classified into four activity categories. We administered the surveys via Qualtrics and calculated the descriptive statistics, scales, and internal consistency statistics using R.

RESULTS

A total of 31 individuals completed the surveys from 27 organizations (53% response rate) in four English-speaking countries. EPIs were evenly split between university (49%) and non-university (51%) settings. Nearly all EPIs conducted direct program support (mean = 4.19/5 [sd = 1.25]) and knowledge-building (4.03 [1.17]) activities. However, engagement with traditionally marginalized and non-traditional partners (2.84 [1.39]) and development of evidence reviews using formal critical appraisal methods (2.81 [1.70]) were uncommon. EPIs tend to be specialized, focusing on a group of highly related strategies rather than incorporating multiple evidence-to-policy strategies in their portfolios. Inter-item consistency was moderate to high, with scale α's ranging from 0.67 to 0.85. Ratings of respondents' willingness to pay for training in one of three evidence dissemination strategies revealed high interest in program and policy design.

CONCLUSIONS

Our results suggest that evidence-to-policy strategies are frequently used by existing EPIs; however, organizations tend to specialize rather than engage in a breadth of strategies. Furthermore, few organizations reported consistently engaging with non-traditional or community partners. Focusing on building capacity for a network of new and existing EPIs could be a promising strategy for growing the infrastructure needed for evidence-informed behavioral health policymaking.

Integrating pragmatic and implementation science randomized clinical trial approaches: a PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) analysis

BACKGROUND

Over the past two decades, pragmatic and implementation science clinical trial research methods have advanced substantially. Pragmatic and implementation studies have natural areas of overlap, particularly relating to the goal of using clinical trial data to leverage health care system policy changes. Few investigations have addressed pragmatic and implementation science randomized trial methods development while also considering policy impact.

METHODS

The investigation used the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2) and PRECIS-2-Provider Strategies (PRECIS-2-PS) tools to evaluate the design of two multisite randomized clinical trials that targeted patient-level effectiveness outcomes, provider-level practice changes and health care system policy. Seven raters received PRECIS-2 training and applied the tools in the coding of the two trials. Descriptive statistics were produced for both trials, and PRECIS-2 wheel diagrams were constructed. Interrater agreement was assessed with the Intraclass Correlation (ICC) and Kappa statistics. The Rapid Assessment Procedure Informed Clinical Ethnography (RAPICE) qualitative approach was applied to understanding integrative themes derived from the PRECIS-2 ratings and an end-of-study policy summit.

RESULTS

The ICCs for the composite ratings across the patient and provider-focused PRECIS-2 domains ranged from 0.77 to 0.87, and the Kappa values ranged from 0.25 to 0.37, reflecting overall fair-to-good interrater agreement for both trials. All four PRECIS-2 wheels were rated more pragmatic than explanatory, with composite mean and median scores ≥ 4. Across trials, the primary intent-to-treat analysis domain was consistently rated most pragmatic (mean = 5.0, SD = 0), while the follow-up/data collection domain was rated most explanatory (mean range = 3.14-3.43, SD range = 0.49-0.69). RAPICE field notes identified themes related to potential PRECIS-2 training improvements, as well as policy themes related to using trial data to inform US trauma care system practice change; the policy themes were not captured by the PRECIS-2 ratings.

CONCLUSIONS

The investigation documents that the PRECIS-2 and PRECIS-2-PS can be simultaneously used to feasibly and reliably characterize clinical trials with patient and provider-level targets. The integration of pragmatic and implementation science clinical trial research methods can be furthered by using common metrics such as the PRECIS-2 and PRECIS-2-PS. Future study could focus on clinical trial policy research methods development.

TRIAL REGISTRATION

DO-SBIS ClinicalTrials.gov NCT00607620. registered on January 29, 2008. TSOS ClinicalTrials.gov NCT02655354, registered on July 27, 2015.

A policy implementation study of earmarked taxes for mental health services: study protocol

BACKGROUND

Insufficient funding is frequently identified as a critical barrier to the implementation and sustainment of evidence-based practices (EBPs). Thus, increasing access to funding is recognized as an implementation strategy. Policies that create earmarked taxes-defined as taxes for which revenue can only be spent on specific activities-are an increasingly common mental health financing strategy that could improve the reach of EBPs. This project's specific aims are to (1) identify all jurisdictions in the USA that have implemented earmarked taxes for mental health and catalogue information about tax design; (2) characterize experiences implementing earmarked taxes among local (e.g., county, city) mental health agency leaders and other government and community organization officials and assess their perceptions of the acceptability and feasibility of different types of policy implementation strategies; and (3) develop a framework to guide effect earmarked tax designs, inform the selection of implementation strategies, and disseminate the framework to policy audiences.

METHODS

The project uses the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to inform data collection about the determinants and processes of tax implementation and Leeman's typology of implementation strategies to examine the acceptability and feasibility strategies which could support earmarked tax policy implementation. A legal mapping will be conducted to achieve aim 1. To achieve aim 2, a survey will be conducted of 300 local mental health agency leaders and other government and community organization officials involved with the implementation of earmarked taxes for mental health. The survey will be followed by approximately 50 interviews with these officials. To achieve aim 3, quantitative and qualitative data will be integrated through a systematic framework development and dissemination process.

DISCUSSION

This exploratory policy implementation process study will build the evidence base for outer-context implementation determinants and strategies by focusing on policies that earmarked taxes for mental health services.

Future Directions in Youth and Family Treatment Engagement: Finishing the Bridge Between Science and Service

The field has spent more than 50 years investing in the quality of youth mental healthcare, with intervention science yielding roughly 1,300 efficacious treatments. In the latter half of this period, concurrent efforts in implementation science have developed effective methods for supporting front-line service organizations and therapists to begin to bridge the science to service gap. However, many youths and families still do not benefit fully from these strategic investments due to low treatment engagement: nearly half of youths in need of services pursue them, and among those who do, roughly another half terminate prematurely. The negative impact of low engagement is substantial, and is disproportionally and inequitably so for many. We contend that to build a robust and "finished" bridge connecting science and service, the field must go beyond its two historical foci of designing interventions and preparing therapists to deliver them, to include an intentional focus on the youths and families who participate in these interventions and who work with those therapists. In this paper, we highlight the significance of treatment engagement in youth mental healthcare and discuss the current state of the literature related to four priorities: conceptualization, theory, measurement, and interventions. Next, we offer an example from our own program of research as one illustration for advancing these priorities. Finally, we propose recommendations to act on these priorities.

Tailoring dissemination strategies to increase evidence-informed policymaking for opioid use disorder treatment: study protocol

BACKGROUND

Policy is a powerful tool for systematically altering healthcare access and quality, but the research to policy gap impedes translating evidence-based practices into public policy and limits widespread improvements in service and population health outcomes. The US opioid epidemic disproportionately impacts Medicaid members who rely on publicly funded benefits to access evidence-based treatment including medications for opioid use disorder (MOUD). A myriad of misaligned policies and evidence-use behaviors by policymakers across federal agencies, state Medicaid agencies, and managed care organizations limit coverage of and access to MOUD for Medicaid members. Dissemination strategies that improve policymakers' use of current evidence are critical to improving MOUD benefits and reducing health disparities. However, no research describes key determinants of Medicaid policymakers' evidence use behaviors or preferences, and few studies have examined data-driven approaches to developing dissemination strategies to enhance evidence-informed policymaking. This study aims to identify determinants and intermediaries that influence policymakers' evidence use behaviors, then develop and test data-driven tailored dissemination strategies that promote MOUD coverage in benefit arrays.

METHODS

Guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, we will conduct a national survey of state Medicaid agency and managed care organization policymakers to identify determinants and intermediaries that influence how they seek, receive, and use research in their decision-making processes. We will use latent class methods to empirically identify subgroups of agencies with distinct evidence use behaviors. A 10-step dissemination strategy development and specification process will be used to tailor strategies to significant predictors identified for each latent class. Tailored dissemination strategies will be deployed to each class of policymakers and assessed for their acceptability, appropriateness, and feasibility for delivering evidence about MOUD benefit design.

DISCUSSION

This study will illuminate key determinants and intermediaries that influence policymakers' evidence use behaviors when designing benefits for MOUD. This study will produce a critically needed set of data-driven, tailored policy dissemination strategies. Study results will inform a subsequent multi-site trial measuring the effectiveness of tailored dissemination strategies on MOUD benefit design and implementation. Lessons from dissemination strategy development will inform future research about policymakers' evidence use preferences and offer a replicable process for tailoring dissemination strategies.

The greatest challenges and solutions to improve children's health and well-being worldwide in the next decade and beyond: Using complex systems and implementation science approaches

The health and well-being of children is paramount to health and well-being of society and is the foundation of health and well-being later in life. This paper presents the perspective that a complex systems approach that embeds implementation science is needed to address the rising challenges to child health and well-being in this decade (2020-2030) and beyond. Reflection on facilitators of the success of programs deemed promising to address child health and well-being in the past decade (2010-2020) is presented, to advance programs to address children's health and well-being. A priority that needs to be addressed is developing, testing and using theories of child and family health and well-being (and related initiatives) that can be used to build on existing successes to make progress. Understanding context including further elucidating the drivers of child health and well-being at multiple levels of relevant systems (e.g., health, education, community) across the life course, and considering implications for caregivers also require greater consideration. Methods to address future challenges to child health and well-being include co-designing initiatives that support child health and well-being with children and families themselves rather than using predesigned initiatives, thoughtful outcome selection, and reporting the challenges of implementing future programs to promote learning. The approaches, priorities and methods presented can be used to design or refine interventions, models or care or community-based initiatives and provide new direction to fields of child health enquiry.

Unintended consequences of disseminating behavioral health evidence to policymakers: Results from a survey-based experiment

Background

Communication research demonstrates that messages often have unintended consequences, but this work has received limited attention in implementation science. This dissemination experiment sought to determine whether state-tailored policy briefs about the behavioral health consequences of adverse childhood experiences (ACEs), compared to national policy briefs on the topic, increased state legislators'/staffers' perceptions of the policy brief relevance and parental blame for the consequences of ACEs, and whether effects differed between Democrats and Republicans.

Method

A preregistered, web-based survey experiment with U.S. state legislators/staffers was conducted in 2021 (n  =  133). Respondents were randomized to view a policy brief about the behavioral health consequences of ACEs that included state-tailored data (intervention condition) or national data (control condition) and then answered survey questions. Dependent variables were perceived policy brief relevance and parental blame for the consequences of ACEs.

Results

The mean policy brief relevance score was 4.1% higher in the intervention than in the control condition (p  =  .24), but the mean parental blame score was 16.5% higher (p  =  .02). When outcomes were dichotomized, 61.2% of respondents in the intervention condition rated parents as "very much to blame" for the consequences of ACEs compared to 37.1% in the control condition (p  =  .01). When the sample was stratified by political affiliation, the effect of the state-tailored policy brief on parental blame was larger in magnitude among Democrats and not significant among Republicans. The intervention policy brief increased the mean parental blame score by 22.8% among Democrats relative to the control policy brief (p  =  .007) and doubled the proportion rating parents as "very much to blame" (52.2% vs. 26.1%, p  =  .03).

Conclusions

Despite limited statistical power, state-tailored policy briefs significantly increased state legislators'/staffers' perceptions of parental blame for the behavioral health consequences of ACEs, relative to a policy brief with national data. Unintended messaging effects warrant greater attention in dissemination research and practice.

Implementation strategies and outcomes of school-based programs for adolescent suicide prevention: A scoping review protocol

OBJECTIVE

This scoping review aims to identify and map the empirical literature on the implementation strategies and outcomes of school-based programs for adolescent suicide prevention (SBASP).

INTRODUCTION

School-based programs are preferred interventions for preventing suicide in adolescents, and their effectiveness has been well-systematized in several reviews. Implementation research is a growing field for prevention programs, making it possible to understand the nature of success or failure outcomes and maximize intervention benefits. However, there is a knowledge gap in the implementation research applied to adolescent suicide prevention in the educational context. We conduct a scoping review to provide the first overview of the scope of implementation research applied to adolescent suicide prevention programs in the school setting to know what implementation strategies and outcomes are reported by these programs and how they are evaluated.

METHODS

The proposed scoping review will be conducted following six stages, including the definition of objectives. Studies must be empirical and address implementation strategies or implementation outcomes of school-based programs for adolescent suicide prevention. Studies that focused exclusively on clinical efficacy or effectiveness evaluation will be excluded. A preliminary search of PubMed was conducted to refine the initial search strings, followed by a final search of several other electronic databases. Finally, a gray literature search will identify unpublished literature and reduce location bias. There will be no limits to a specific date. Two independent reviewers will screen, select, and extract the retrieved records. The results will be presented using tabular forms and a narrative summary with attention to the review objectives and research questions and their implications for research and practice of school-based programs for adolescent suicide prevention.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

BACKGROUND

Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers.

MAIN BODY

Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches.

CONCLUSION

We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

Inter-agency collaboration is associated with increased frequency of research use in children's mental health policy making

OBJECTIVE

To determine whether the self-report frequency of inter-agency collaboration about children's mental health issues is associated with the self-report frequency of using research evidence in children's mental health policy and program decision making in mental health agencies (MHAs).

DATA SOURCES

Primary data were collected through web-based surveys of state (N = 221) and county (N = 117) MHA officials.

DESIGN

The primary independent variable was a composite score quantifying the frequency of collaboration about children's mental health issues between officials in MHAs and six other state agencies. The dependent variables were composite scores quantifying the frequency of research use in children's mental health policy and program decision making in general and for specific purposes (i.e., conceptual, instrumental, tactical, imposed). Covariates were composite scores quantifying well-established determinants of research use (e.g., agency leadership, research use skills) in agency policy and program decision making.

DATA METHODS

Separate multiple linear regression models estimated associations between frequency of inter-agency collaboration and research use scores, adjusting for other determinants of research use, respondent state, and other covariates. Data from state and county officials were analyzed separately.

PRINCIPAL FINDINGS

The frequency of inter-agency collaboration was positively and independently associated with the frequency of research use in children's mental health policy making among state (β = 0.22, p = 0.004) and county (β = 0.39, p < 0.0001) MHA officials. Inter-agency collaboration was also the only variable significantly associated with the frequency of research use for all four specific purposes among state MHA officials, and similar findings we observed among county MHA officials. The magnitudes of associations between inter-agency collaboration and frequency of research use were generally stronger than for more well-established determinants of research use in policy making.

CONCLUSIONS

Strategies that promote collaboration between MHA officials and external agencies could increase the use of research evidence in children's mental health policy and program decision making in MHAs.

Partisan differences in the effects of economic evidence and local data on legislator engagement with dissemination materials about behavioral health: a dissemination trial

BACKGROUND

State legislators make policy decisions that influence children's exposure to adverse childhood experiences (ACEs), such as child maltreatment, and their effects on behavioral health. Effective dissemination of scientific research can increase the likelihood that legislators' decisions are aligned with evidence to prevent ACEs and their consequences, and effective dissemination requires legislators to engage with dissemination materials. Informed by the elaboration likelihood model of persuasive communication and Brownson's Model of Dissemination Research, we tested the hypothesis that inclusion of economic evidence and local data would increase legislator engagement with dissemination materials about evidence-supported policies related to ACEs and behavioral health.

METHODS

A three-arm randomized dissemination trial was conducted. A university researcher e-mailed dissemination materials which contained evidence about ACEs and behavioral health problems to state legislators (two e-mails sent 2 weeks apart, 12,662 e-mails delivered to 6509 legislators). The e-mail subject lines, text, and policy brief content were manipulated across the study arms. The intervention condition received state-tailored data about rates of ACEs and state-tailored economic evidence about the costs of ACEs for public systems, the enhanced control condition received state-tailored data and not economic evidence, and the control condition received national data and not economic evidence. Outcomes were rates of e-mail views, policy brief link clicks, requests for researcher consultation, and mentions of child maltreatment terms in legislators' social media posts.

RESULTS

For the first e-mail, the e-mail view rate was 42.6% higher in the intervention than in the enhanced control condition (22.8% vs. 14.8%) and 20.8% higher than in the control condition (22.8% vs. 18.5%) (both p < .0001). Similar results were observed for the second e-mail. These differences remained significant after adjustment for demographic differences across study conditions in individual-level models, but not multilevel models. There was a significant interaction between the experimental condition and political party (p < .0001) in which the intervention increased e-mail view rates among Democrats but not Republicans. The intervention had no effect on policy brief link clicks or requests for consultation and a mixed effect on social media posts.

CONCLUSIONS

Inclusion of state-tailored economic evidence in dissemination materials can increase engagement with research evidence among Democrat, but not Republican, legislators. Dissemination strategies tailored for legislators' political party affiliation may be needed.

Grounding implementation science in health equity for cancer prevention and control

BACKGROUND

The past decade of research has seen theoretical and methodological advances in both implementation science and health equity research, opening a window of opportunity for facilitating and accelerating cross-disciplinary exchanges across these fields that have largely operated in siloes. In 2019 and 2020, the National Cancer Institute's Consortium for Cancer Implementation Science convened an action group focused on 'health equity and context' to identify opportunities to advance implementation science. In this paper, we present a narrative review and synthesis of the relevant literature at the intersection of health equity and implementation science, highlight identified opportunities (i.e., public goods) by the action group for advancing implementation science in cancer prevention and control, and integrate the two by providing key recommendations for future directions.

DISCUSSION

In the review and synthesis of the literature, we highlight recent advances in implementation science, relevant to promoting health equity (e.g., theories/models/frameworks, adaptations, implementation strategies, study designs, implementation determinants, and outcomes). We acknowledge the contributions from the broader field of health equity research and discuss opportunities for integration and synergy with implementation science, which include (1) articulating an explicit focus on health equity for conducting and reviewing implementation science; (2) promoting an explicit focus on health equity in the theories, models, and frameworks guiding implementation science; and (3) identifying methods for understanding and documenting influences on the context of implementation that incorporate a focus on equity. To advance the science of implementation with a focus on health equity, we reflect on the essential groundwork needed to promote bi-directional learning between the fields of implementation science and health equity research and recommend (1) building capacity among researchers and research institutions for health equity-focused and community-engaged implementation science; (2) incorporating health equity considerations across all key implementation focus areas (e.g., adaptations, implementation strategies, study design, determinants, and outcomes); and (3) continuing a focus on transdisciplinary opportunities in health equity research and implementation science. We believe that these recommendations can help advance implementation science by incorporating an explicit focus on health equity in the context of cancer prevention and control and beyond.

How Ethnic Minority Context Alters the Risk for Developing Mental Health Disorders and Psychological Distress for Latinx Young Adults

Policy Points In low-income communities in the South Bronx and Puerto Rico, Puerto Rican youth are exposed to many of the same risk and protective factors for developing depression, anxiety, or psychological distress; yet it is unclear how the ethnic minority context of the South Bronx and ethnic majority context of Puerto Rico influence risk. Results from our quasi-experimental, longitudinal study demonstrate the importance of addressing social factors (parent-child relationships, youth peer relationships) for youth living in the majority context, and neighborhood and cultural factors (residential mobility, perceived discrimination, perceived social position in the neighborhood) for youth living in the minority context. Our findings support the need for tailoring programs specific to the needs of youth who reside in an ethnic majority or a minority context, since some of the risk factors might operate differently depending on context. Housing and neighborhood environment policies that address discrimination and eliminate structural inequities for ethnic minority groups may protect against the harm of minoritization on young people's mental health.

CONTEXT

Puerto Rican youth growing up in low-income communities in the South Bronx and Puerto Rico are exposed to many of the same risk factors for major depressive disorder, generalized anxiety disorder, and psychological distress. One potentially powerful factor differs: Puerto Ricans have been socially marginalized as an ethnic minority group in the South Bronx, but are the ethnic majority of the population in Puerto Rico. A growing body of literature demonstrates the influence of neighborhood, cultural, and social factors and parental psychopathology in the development of mental health problems. An important unanswered question is whether these risk and protective factors have the same impact for youth raised as members of an ethnic majority versus minority group.

METHODS

Using a population-based cohort study, with four waves of assessment from early childhood into young adulthood, we investigated whether ethnic minority context alters risk and protective factors for depression, anxiety, and psychological distress. Our longitudinal data set includes 2,491 young children at baseline (82.8% retained at wave 4). Using a quasi-experimental design, we examine how ethnic minority context can alter the development of mental health disorders as Latinx children transition to late adolescence and young adulthood.

FINDINGS

Some risk and protective factors operated differently across minority and majority contexts. Higher discrimination and social position were more powerful risk and protective factors, respectively, in the minority context, whereas positive peer relationships mattered more in the majority context. Children of mothers with depression were significantly more likely to develop anxiety in late adolescence and young adulthood in the majority context (60.0%) compared to the minority context (4.5%).

CONCLUSIONS

Preventing depression and anxiety disorders in Latinx young adults may require targeting different childhood factors depending on whether they reside within the ethnic majority or minority context. People in the ethnic minority context may benefit more from policies aimed at reducing discrimination and improving economic opportunity, while people in the majority context may benefit more from opportunities for strengthening family and peer relationships.

Don't Get Lost in Translation: Integrating Developmental and Implementation Sciences to Accelerate Real-World Impact on Children's Development, Health, and Wellbeing

Translation of developmental science discoveries is impeded by numerous barriers at different stages of the research-to-practice pipeline. Actualization of the vast potential of the developmental sciences to improve children's health and development in the real world is imperative but has not yet been fully realized. In this commentary, we argue that an integrated developmental-implementation sciences framework will result in a translational mindset essential for accelerating real world impact. We delineate key principles and methods of implementation science of salience to the developmental science audience, lay out a potential synthesis between implementation and developmental sciences, provide an illustration of the Mental Health, Earlier Partnership (MHE-P), and set actionable steps for realization. Blending these approaches along with wide-spread adoption of the translational mindset has transformative potential for population-level impact of developmental science discovery.

Policy Makers' Priorities for Addressing Youth Substance Use and Factors That Influence Priorities

OBJECTIVE

Understanding public policy makers' priorities for addressing youth substance use and the factors that influence these priorities can inform the dissemination and implementation of strategies that promote evidence-based decision making. This study characterized the priorities of policy makers in substance use agencies of U.S. states and counties for addressing youth substance use, the factors that influenced these priorities, and the differences in priorities and influences between state and county policy makers.

METHODS

In 2020, a total of 122 substance use agency policy makers from 35 states completed a Web-based survey (response rate=22%). Respondents rated the priority of 14 issues related to youth substance use and the extent to which nine factors influenced these priorities. Data were analyzed as dichotomous and continuous variables and for state and county policy makers together and separately.

RESULTS

The highest priorities for youth substance use were social determinants of substance use (87%), adverse childhood experiences and childhood trauma (85%), and increasing access to school-based substance use programs (82%). The lowest priorities were increasing access to naloxone for youths (49%), increasing access to medications for opioid use disorder among youths (49%), and deimplementing non-evidence-based youth substance use programs (41%). The factors that most influenced priorities were budget issues (80%) and state legislature (69%), federal (67%), and governor priorities (65%). Issues related to program implementation and deimplementation were significantly higher priorities for state than for county policy makers.

CONCLUSIONS

These findings can inform the tailoring of dissemination and implementation strategies to account for the inner- and outer-setting contexts of substance use agencies.

Simulating the role of knowledge brokers in policy making in state agencies: An agent-based model

OBJECTIVE

To model children's mental health policy making dynamics and simulate the impacts of knowledge broker interventions.

DATA SOURCES

Primary data from surveys (n = 221) and interviews (n = 64) conducted in 2019-2021 with mental health agency (MHA) officials in state agencies.

STUDY DESIGN

A prototype agent-based model (ABM) was developed using the PARTE (Properties, Actions, Rules, Time, Environment) framework and informed through primary data collection. In each simulation, a policy is randomly generated (salience weights: cost, contextual alignment, and strength of evidence) and discussed among agents. Agents are MHA officials and heterogenous in their properties (policy making power and network influence) and policy preferences (based on salience weights). Knowledge broker interventions add agents to the MHA social network who primarily focus on the policy's research evidence.

DATA COLLECTION/EXTRACTION METHODS

A sequential explanatory mixed method approach was used. Descriptive and regression analyses were used for the survey data and directed content analysis was used to code interview data. Triangulated results informed ABM development. In the ABM, policy makers with various degrees of decision influence interact in a scale-free network before and after knowledge broker interventions. Over time, each decides to support or oppose a policy proposal based on policy salience weights and their own properties and interactions. The main outcome is an agency-level decision based on policy maker support. Each intervention and baseline simulation runs 250 times across 50 timesteps.

PRINCIPAL FINDINGS

Surveys and interviews revealed that barriers to research use could be addressed by knowledge brokers. Simulations indicated that policy decision outcomes varied by policy making context within agencies.

CONCLUSIONS

This is the first application of ABM to evidence-informed mental health policy making focusing. Results suggest that the presence of knowledge brokers can: (1) influence consensus formation in MHAs, (2) accelerate policy decisions, and (3) increase the likelihood of evidence-informed policy adoption.

Cocreating evidence-informed health equity policy with community

Objective

To explore the feasibility of a rapid, community‐engaged strategy to prioritize health equity policy options as informed by research evidence, community‐voiced needs, and public health priorities.

Data Sources

Data came from residents in a midsized, demographically, and geographically diverse county over a period of 8 months in 2020 and an evidence review of the health equity policy literature during the same time period.

Study Design

A descriptive case study is used to explore the feasibility and potential value of a community codesigned approach to establish community priorities for health equity policy.

Data Collection/Extraction Methods

Evidence synthesis of health equity policy was conducted parallel to 15 community listening sessions across the county to elicit information on health needs. We used scoping review methods to obtain literature from academic databases and scholarly public health and policy organizations. This information was cross walked with themes from the listening sessions to identify 10 priority policy areas, which were taken back to the community for 15 participatory discussion and ranking sessions.

Principal Findings

The process appeared to authentically engage the input of 200 community members representative of minoritized groups while identifying 99 evidence‐informed policy levers to promote health equity. Discussion and ranking activities were successful in facilitating community discussion and policy decision making. Remote platforms may have limited the engagement of some residents while promoting the participation of others. Conducting information integration within the research team prior to community policy ranking sessions limited the community ownership over how policies were interpreted and communicated.

Conclusions

A combination of information integration and community ranking activities can be used to achieve community‐engaged policy prioritization of options in a fairly rapid period of time. While this process provides an example of authentic community ownership of policy prioritization, the compressed timeline limited the community's engagement in the information integration phase.

Translating Medicaid policy into practice: policy implementation strategies from three US states' experiences enhancing substance use disorder treatment

BACKGROUND

Despite the important upstream impact policy has on population health outcomes, few studies in implementation science in health have examined implementation processes and strategies used to translate state and federal policies into accessible services in the community. This study examines the policy implementation strategies and experiences of Medicaid programs in three US states that responded to a federal prompt to improve access to evidence-based practice (EBP) substance use disorder (SUD) treatment.

METHODS

Three US state Medicaid programs implementing American Society of Addiction Medicine (ASAM) Criteria-driven SUD services under Section 1115 waiver authority were used as cases. We conducted 44 semi-structured interviews with Medicaid staff, providers and health systems partners in California, Virginia, and West Virginia. Interviews were triangulated with document review of state readiness and implementation plans. The Exploration, Preparation, Implementation, Sustainment Framework (EPIS) guided qualitative theme analysis. The Expert Recommendations for Implementing Change and Specify It criteria were used to create a taxonomy of policy implementation strategies used by policymakers to promote providers' uptake of statewide EBP SUD care continuums.

RESULTS

Four themes describe states' experiences and outcomes implementing a complex EBP SUD treatment policy directive: (1) Medicaid agencies adapted their inner/outer contexts to align with EBPs and adapted EBPs to fit their local context; (2) enhanced financial reimbursement arrangements were inadequate bridging factors to achieve statewide adoption of new SUD services; (3) despite trainings, service providers and managed care organizations demonstrated poor fidelity to the ASAM Criteria; and (4) successful policy adoption at the state level did not guarantee service providers' uptake of EBPs. States used 29 implementation strategies to implement EBP SUD care continuums. Implementation strategies were used in the Exploration (n=6), Preparation (n=10), Implementation (n=19), and Sustainment (n=6) phases, and primarily focused on developing stakeholder interrelationships, evaluative and iterative approaches, and financing.

CONCLUSIONS

This study enhances our understanding of statewide policy implementation outcomes in low-resource, public healthcare settings. Themes highlight the need for additional pre-implementation and sustainment focused implementation strategies. The taxonomy of detailed policy implementation strategies employed by policymakers across states should be tested in future policy implementation research.

Strategies for implementing music-based interventions for people with dementia in long-term care facilities: A systematic review

OBJECTIVES

Research has found that music-based interventions can decrease behavioral and psychological symptoms of dementia or behaviors that challenge (BPSD/BtC). However, how to effectively implement these interventions is unclear. This study synthesizes available evidence regarding implementation strategies and outcomes of music-based interventions for people with dementia at residential long-term care facilities.

METHODS

Study registered with PROSPERO (registration number: CRD42020194354). We searched the following databases: PsychInfo, PubMed, MEDLINE, CINAHL, and The Cochrane Library. Inclusion criteria included articles targeting music-based interventions conducted for people with dementia, studies conducted in residential long-term care facilities, and articles that reported implementation strategies and outcomes of the intervention.

RESULTS

Of the included eight studies, half were studies of music therapy and the other half were on individualized music. 49 implementation strategies were reported. The most frequently reported category of strategies was planning (34.7%), followed by education (24.5%), quality management (24.5%), restructuring (12.2%), and finance (4.1%). No strategies under the category of attending to the policy context were reported. The most frequently reported implementation outcomes were appropriateness (27.3%), followed by adoption (22.7%), fidelity (22.7%), acceptability (9.1%), sustainability (9.1%), and cost (9.1%). No studies measured feasibility or penetration.

CONCLUSIONS

Although various effective implementation strategies were identified, we were unable to examine the effectiveness of individual implementation strategies due to the designs of the selected studies. Less attention has been paid to strategies that aim at structural changes of intervention delivery systems. Future studies should investigate facilitators and barriers of implementing music-based interventions especially focusing on structural aspects.

Quantitative measures used in empirical evaluations of mental health policy implementation: A systematic review

Background

Mental health is a critical component of wellness. Public policies present an opportunity for large-scale mental health impact, but policy implementation is complex and can vary significantly across contexts, making it crucial to evaluate implementation. The objective of this study was to (1) identify quantitative measurement tools used to evaluate the implementation of public mental health policies; (2) describe implementation determinants and outcomes assessed in the measures; and (3) assess the pragmatic and psychometric quality of identified measures.

Method

Guided by the Consolidated Framework for Implementation Research, Policy Implementation Determinants Framework, and Implementation Outcomes Framework, we conducted a systematic review of peer-reviewed journal articles published in 1995-2020. Data extracted included study characteristics, measure development and testing, implementation determinants and outcomes, and measure quality using the Psychometric and Pragmatic Evidence Rating Scale.

Results

We identified 34 tools from 25 articles, which were designed for mental health policies or used to evaluate constructs that impact implementation. Many measures lacked information regarding measurement development and testing. The most assessed implementation determinants were readiness for implementation, which encompassed training (n  =  20, 57%) and other resources (n  =  12, 34%), actor relationships/networks (n  =  15, 43%), and organizational culture and climate (n  =  11, 31%). Fidelity was the most prevalent implementation outcome (n  =  9, 26%), followed by penetration (n  =  8, 23%) and acceptability (n  =  7, 20%). Apart from internal consistency and sample norms, psychometric properties were frequently unreported. Most measures were accessible and brief, though minimal information was provided regarding interpreting scores, handling missing data, or training needed to administer tools.

Conclusions

This work contributes to the nascent field of policy-focused implementation science by providing an overview of existing measurement tools used to evaluate mental health policy implementation and recommendations for measure development and refinement. To advance this field, more valid, reliable, and pragmatic measures are needed to evaluate policy implementation and close the policy-to-practice gap.

Plain Language Summary

Mental health is a critical component of wellness, and public policies present an opportunity to improve mental health on a large scale. Policy implementation is complex because it involves action by multiple entities at several levels of society. Policy implementation is also challenging because it can be impacted by many factors, such as political will, stakeholder relationships, and resources available for implementation. Because of these factors, implementation can vary between locations, such as states or countries. It is crucial to evaluate policy implementation, thus we conducted a systematic review to identify and evaluate the quality of measurement tools used in mental health policy implementation studies. Our search and screening procedures resulted in 34 measurement tools. We rated their quality to determine if these tools were practical to use and would yield consistent (i.e., reliable) and accurate (i.e., valid) data. These tools most frequently assessed whether implementing organizations complied with policy mandates and whether organizations had the training and other resources required to implement a policy. Though many were relatively brief and available at little-to-no cost, these findings highlight that more reliable, valid, and practical measurement tools are needed to assess and inform mental health policy implementation. Findings from this review can guide future efforts to select or develop policy implementation measures.

Embracing policy implementation science to ensure translation of evidence to cancer control policy

Although health-related policies are abundant, efforts to understand how to ensure that these policies serve as an effective vehicle for translating scientific evidence are relatively sparse. This paper explores how policy-focused implementation science (IS) may contribute to understanding the translation of scientific evidence to health-related policy in governmental and nongovernmental sectors. Expanding the focus of implementation science in cancer control could systematically address policy to both increase the use of scientific evidence in general and to address health equity. In this Commentary, we look to relevant work outside of IS that could be informative, most notably from the field of political science. We propose several ideas for future research that could help move the field of policy implementation science in cancer control in the USA forward. Although most efforts to increase uptake of the scientific evidence base reference translation to "practice and policy," there has been relatively little emphasis in the USA on implementation at the policy level, especially related to cancer control. If we are to achieve the full benefits of scientific discovery on population and public health, we will need to consider policy as a critical mechanism by which evidence can be translated to practice. We have a robust set of methods within implementation science that are increasing the pace of adoption and maintenance of evidence-based programs in a variety of settings. Building on these efforts, the time is right to expand our focus to include policy implementation.

Application of an antiracism lens in the field of implementation science (IS): Recommendations for reframing implementation research with a focus on justice and racial equity

Background

Despite the promise of implementation science (IS) to reduce health inequities, critical gaps and opportunities remain in the field to promote health equity. Prioritizing racial equity and antiracism approaches is critical in these efforts, so that IS does not inadvertently exacerbate disparities based on the selection of frameworks, methods, interventions, and strategies that do not reflect consideration of structural racism and its impacts.

Methods

Grounded in extant research on structural racism and antiracism, we discuss the importance of advancing understanding of how structural racism as a system shapes racial health inequities and inequitable implementation of evidence-based interventions among racially and ethnically diverse communities. We outline recommendations for explicitly applying an antiracism lens to address structural racism and its manifests through IS. An anti-racism lens provides a framework to guide efforts to confront, address, and eradicate racism and racial privilege by helping people identify racism as a root cause of health inequities and critically examine how it is embedded in policies, structures, and systems that differentially affect racially and ethnically diverse populations.

Results

We provide guidance for the application of an antiracism lens in the field of IS, focusing on select core elements in implementation research, including: (1) stakeholder engagement; (2) conceptual frameworks and models; (3) development, selection, adaptation of EBIs; (4) evaluation approaches; and (5) implementation strategies. We highlight the need for foundational grounding in antiracism frameworks among implementation scientists to facilitate ongoing self-reflection, accountability, and attention to racial equity, and provide questions to guide such reflection and consideration.

Conclusion

We conclude with a reflection on how this is a critical time for IS to prioritize focus on justice, racial equity, and real-world equitable impact. Moving IS towards making consideration of health equity and an antiracism lens foundational is central to strengthening the field and enhancing its impact.

Plain language abstract

There are important gaps and opportunities that exist in promoting health equity through implementation science. Historically, the commonly used frameworks, measures, interventions, strategies, and approaches in the field have not been explicitly focused on equity, nor do they consider the role of structural racism in shaping health and inequitable delivery of evidence-based practices/programs. This work seeks to build off of the long history of research on structural racism and health, and seeks to provide guidance on how to apply an antiracism lens to select core elements of implementation research. We highlight important opportunities for the field to reflect and consider applying an antiracism approach in: 1) stakeholder/community engagement; 2) use of conceptual frameworks; 3) development, selection and adaptation of evidence-based interventions; 4) evaluation approaches; 5) implementation strategies (e.g., how to deliver evidence-based practices, programs, policies); and 6) how researchers conduct their research, with a focus on racial equity. This is an important time for the field of implementation science to prioritize a foundational focus on justice, equity, and real-world impact through the application of an anti-racism lens in their work.

Impact of the COVID-19 Pandemic on Child and Adolescent Mental Health Policy and Practice Implementation

Background: The impact of the 2019 coronavirus pandemic on the mental health of millions worldwide has been well documented, but its impact on prevention and treatment of mental and behavioral health conditions is less clear. The COVID-19 pandemic also created numerous challenges and opportunities to implement health care policies and programs under conditions that are fundamentally different from what has been considered to be usual care. Methods: We conducted a qualitative study to determine the impact of the COVID-19 pandemic on implementation of evidence-based policy and practice by State Mental Health Authorities (SMHA) for prevention and treatment of mental health problems in children and adolescents. Semi-structured interviews were conducted with 29 SMHA representatives of 21 randomly selected states stratified by coronavirus positivity rate and rate of unmet services need. Data analysis with SMHA stakeholders used procedures embedded in the Rapid Assessment Procedure—Informed Community Ethnography methodology. Results: The need for services increased during the pandemic due primarily to family stress and separation from peers. States reporting an increase in demand had high coronavirus positivity and high unmet services need. The greatest impacts were reduced out-of-home services and increased use of telehealth. Barriers to telehealth services included limited access to internet and technology, family preference for face-to-face services, lack of privacy, difficulty using with young children and youth in need of substance use treatment, finding a Health Insurance Portability and Accountability Act (HIPAA)-compliant platform, training providers and clients, and reimbursement challenges. Policy changes to enable reimbursement, internet access, training, and provider licensing resulted in substantially fewer appointment cancellations or no-shows, greater family engagement, reduction in travel time, increased access for people living in remote locations, and increased provider communication and collaboration. States with high rates of coronavirus positivity and high rates of unmet need were most likely to continue use of telehealth post-pandemic. Despite these challenges, states reported successful implementation of policies designed to facilitate virtual services delivery with likely long-term changes in practice. Conclusions: Policy implementation during the pandemic provided important lessons for planning and preparedness for future public health emergencies. Successful policy implementation requires ongoing collaboration among policy makers and with providers.

A Virtual Reality Resident Training Curriculum on Behavioral Health Anticipatory Guidance: Development and Usability Study

BACKGROUND

Behavioral health disorders have steadily increased and been exacerbated by the COVID-19 pandemic. Though behavioral health disorders can be successfully mitigated with early implementation of evidence-based parent management strategies, education for pediatric residents on behavioral health anticipatory guidance has been limited to date, with training challenges compounded by the physical distancing requirements of the COVID-19 pandemic. Virtual reality (VR) simulations provide an opportunity to train residents on this complex competency by allowing deliberate practice of necessary skills while adhering to current social distancing guidelines.

OBJECTIVE

This study explored the usability of a VR-based behavioral health anticipatory guidance curriculum for pediatric residents.

METHODS

This mixed methods study included 14 postgraduate third-year pediatric residents who completed the behavioral health anticipatory guidance VR curriculum. Residents completed the MEC Spatial Presence Questionnaire to assess immersion in the virtual environment. Semistructured interviews were used to elucidate residents' perspectives on the curriculum's content and format. The interviews were analyzed using conventional content analysis.

RESULTS

Quantitatively, residents reported a high degree of immersion, spatial presence, and cognitive involvement. Most residents (11/14, 79%) agreed or strongly agreed that it seemed as though they took part in the action of the simulation. Qualitatively, two themes emerged from the data: (1) the curriculum expands behavioral health anticipatory guidance and motivational interviewing knowledge and skills and (2) VR technology is uniquely positioned to develop competence. These themes revealed that the curriculum expanded their current level of knowledge and skill, addressed training gaps, and was applicable to all residents. Additionally, residents experienced VR as immersive, feasible, realistic to the clinic setting, and a safe space to practice and learn new skills.

CONCLUSIONS

Pilot data indicates that VR may be an effective tool to teach pediatric residents behavioral health anticipatory guidance, meeting a current gap in medical education training. This VR curriculum is particularly relevant in the context of the COVID-19 pandemic given the increased behavioral health concerns of families.

The Analysis of Mathematics Academic Burden for Primary School Students Based on PISA Data Analysis

To explore the impact of academic burden on the physical and mental health of primary school students, combined with the results of the Programme for International Student Assessment (PISA) report in 2018, the relationship among the development of mathematical literacy, mathematics academic burden, and the physical and mental health of primary school students is studied. First, the relationship between mathematical literacy and mathematics anxiety is analyzed, and related influencing factors and measurement methods of mathematics anxiety are introduced. A questionnaire is then designed for primary school students' mathematical stress, and the reliability and validity of the designed questionnaire are tested. Finally, a questionnaire survey is conducted on students, parents, and teachers in the third, fourth, and fifth grades of three standardized public primary schools. The results of the questionnaire survey show that students, teachers, and parents have a general understanding of the mathematics academic burden of primary school students at this stage. A total of 70% of teachers believe that primary school students have a heavy mathematics burden; 50% of parents think that primary school students are under heavy academic stress; 70% of primary school students believe that the heavy mathematics burden leads to reduced sleep time and extracurricular activities, which has a serious impact on the physical and mental health of primary school students. This research provides a reference for improving the current balance between education and students' physical and mental health in China.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.