A state agency-university partnership for translational research and the dissemination of evidence-based prevention and intervention

A landscape assessment of the activities and capacities of evidence-to-policy intermediaries (EPI) in behavioral health

BACKGROUND

A significant gap exists between the production of research evidence and its use in behavioral health policymaking. Organizations providing consulting and support activities for improving policy represent a promising source for strengthening the infrastructure to address this gap. Understanding the characteristics and activities of these evidence-to-policy intermediary (EPI) organizations can inform the development of capacity-building activities, leading to strengthened evidence-to-policy infrastructure and more widespread evidence-based policymaking.

METHODS

Online surveys were sent to 51 organizations from English-speaking countries involved in evidence-to-policy activities in behavioral health. The survey was grounded in a rapid evidence review of the academic literature regarding strategies used to influence research use in policymaking. The review identified 17 strategies, which were classified into four activity categories. We administered the surveys via Qualtrics and calculated the descriptive statistics, scales, and internal consistency statistics using R.

RESULTS

A total of 31 individuals completed the surveys from 27 organizations (53% response rate) in four English-speaking countries. EPIs were evenly split between university (49%) and non-university (51%) settings. Nearly all EPIs conducted direct program support (mean = 4.19/5 [sd = 1.25]) and knowledge-building (4.03 [1.17]) activities. However, engagement with traditionally marginalized and non-traditional partners (2.84 [1.39]) and development of evidence reviews using formal critical appraisal methods (2.81 [1.70]) were uncommon. EPIs tend to be specialized, focusing on a group of highly related strategies rather than incorporating multiple evidence-to-policy strategies in their portfolios. Inter-item consistency was moderate to high, with scale α's ranging from 0.67 to 0.85. Ratings of respondents' willingness to pay for training in one of three evidence dissemination strategies revealed high interest in program and policy design.

CONCLUSIONS

Our results suggest that evidence-to-policy strategies are frequently used by existing EPIs; however, organizations tend to specialize rather than engage in a breadth of strategies. Furthermore, few organizations reported consistently engaging with non-traditional or community partners. Focusing on building capacity for a network of new and existing EPIs could be a promising strategy for growing the infrastructure needed for evidence-informed behavioral health policymaking.

Mechanisms of change in external implementation support: A conceptual model and case examples to guide research and practice

Background

External implementation support (EIS) is a well-recognized feature of implementation science and practice, often under related terms such as technical assistance and implementation facilitation. Existing models of EIS have gaps related to addressing practice outcomes at both individual and organizational levels, connecting practice activities to intended outcomes, or grounding in well-established theories of behavior and organization change. Moreover, there have been calls to clarify the mechanisms of change through which EIS influences related outcomes.

Method

In this article, we theorize about mechanisms of change within EIS. Our theorizing process aligns with the approach advocated by Kislov et al. We aim to consolidate prior EIS literature, combining related constructs from previous empirical and conceptual work while drawing on our extensive EIS experience to develop a higher-order, midrange theory of change.

Results

Our theory of change is empirically and practically informed, conceptually situated within an established grand theory of change, and guided by eight practice principles and social cognitive theory. The theory of change proposes 10 core practice components as mechanisms of change within EIS. When used according to underlying theory and principles, they are believed to contribute to favorable practice outcomes at individual, team, organizational, and system levels. The model offers flexibility by recognizing the need for sequential support processes and the demand to practice in dynamic and responsive ways. Case examples are presented to illustrate major themes and patterns of the model in action.

Conclusions

The proposed model is intended to support prospective EIS studies by conceptualizing discernable practice components with hypothesized relationships to proximal and distal practice outcomes. The model can be behaviorally operationalized to compliment and extend competency-based approaches to implementation support practitioner (ISP) training and coaching. Over time, the model should be refined based on new empirical findings and contributions from ISPs across the field.

Partnership between academics and practitioners - Addressing the challenges in forensic science

This research discusses the development of academic-practitioner partnerships in forensic science and examines the opinions and experience of those involved in the field. An anonymous online survey was completed by 56 participants who work in the field of forensic science. The questions related to their work experience, their experience of research and partnership, and their opinions on the benefits and barriers that exist. The results were analysed using a mixed methods approach, with quantitative analysis of the responses to closed questions using two-way chi-square statistical analysis, and qualitative analysis of the free text responses using reflexive thematic analysis. This work identifies the demand for partnership, the perceived benefits and barriers that exist, and establishes how the role of the participant (academic, pracademic or practitioner) impacts their view of partnership. We include the term pracademic to mean an individual who has worked as a practitioner and an academic, not necessarily simultaneously. Quantitative analysis identified that there was very little statistically significant difference in the responses between groups. Pracademics considered that 'institutional and cultural' and 'lack of the respect of the other role' were more significant barriers than the other groups. Association was also found between those with greater experience of research and the view that partnership 'improved legitimacy in practice' and 'increased legitimacy of research'. There was also statistical significance in those with more than average experience of partnership who identified 'improved legitimacy in practice' as a benefit of partnership. Reflexive thematic analysis of free text comments identified a need and demand for partnership with three key themes developed as being necessary for successful partnership. These are the 'three 'R's' - the need for effective communication and the development of a Relationship; the Relevance of the partnership to the participants role; and the inclusion of personal Reward such as improved practice or better research.

Reporting and Interpreting Effect Sizes in Applied Health-Related Settings: The Case of Spirituality and Substance Abuse

Inferential analysis using null hypothesis significance testing (NHST) allows accepting or rejecting a null hypothesis. Nevertheless, rejecting a null hypothesis and concluding there is a statistical effect does not provide a clue as to its practical relevance or magnitude. This process is key to assessing the effect size (ES) of significant results, be it using context (comparing the magnitude of the effect to similar studies or day-to-day effects) or statistical estimators, which also should be sufficiently interpreted. This is especially true in clinical settings, where decision-making affects patients' lives. We carried out a systematic review for the years 2015 to 2020 utilizing Scopus, PubMed, and various ProQuest databases, searching for empirical research articles with inferential results linking spirituality to substance abuse outcomes. Out of the 19 studies selected, 11 (57.9%) reported no ES index, and 9 (47.4%) reported no interpretation of the magnitude or relevance of their findings. The results of this review, although limited to the area of substance abuse and spiritual interventions, are a cautionary tale for other research topics. Gauging and interpreting effect sizes contributes to a better understanding of the subject under scrutiny in any discipline.

The implementation potential of a method to monitor empirically-supported children's mental health treatment through claims data

Background

The delivery of evidence-supported treatments (EST) in children’s mental health could be a valuable measure for monitoring mental healthcare quality; however, efforts to monitor the use of EST in real world systems are hindered by the lack of pragmatic methods. This mixed methods study examined the implementation and agency response rate of a pragmatic, claims-based measure of EST designed to be applied as a universal quality measure for child psychotherapy encounters in a state Medicaid system.

Methods

Implementation potential of the EST measure was assessed with healthcare leader rankings of the reporting method’s acceptability, appropriateness and feasibility (n = 53), and post-implementation ratings of EST rate accuracy. Ability of the healthcare system to monitor EST through claims was measured by examining the agency responsivity in using the claims-based measure across 98 Medicaid-contracted community mental health (CMH) agencies in Washington State.

Results

The analysis found the reporting method had high implementation potential. The method was able to measure the use of an EST for 83% of children covered by Medicaid with 58% CMH agencies reporting > 0 ESTs in one quarter. Qualitative analyses revealed that the most significant barrier to reporting ESTs was the operability of electronic health record systems and agencies’ mixed views regarding the accuracy and benefits of reporting.

Conclusions

Measurement of child mental health ESTs through Medicaid claims reporting has acceptable implementation potential and promising real world responsiveness from CMH agencies in one state. Variation in reporting by agency site and low to moderate perceived value by agency leaders suggests the need for additional implementation supports for wider uptake.

Developing a practice and evidence-based guideline for occupational health and safety professionals to prevent and handle musculoskeletal pain in workplaces

Practice guidelines can facilitate the translation of evidence-based knowledge into better occupational health and safety (OHS) prevention. This paper describes the development process, findings and content of a practice and evidence-based guideline for musculoskeletal pain (MSP) to OHS professionals in Denmark. We used a participatory process with involvement of more than 100 OHS professionals in the development of the guideline. The guideline contains three sections: 1) Rapid review of risk factors for MSP (Push/pull, Screen work, Lifting, Awkward postures and Psychosocial factors related to MSP) and single- and multi-stranded interventions targeting MSP. 2) Process recommendations for use of the guideline by a three-phase participatory process 3) Practical recommendations that contain advice and methods for the three-phase participatory process. This paper can promote future guideline development, as it provides specific insight into how OHS professionals can be included in the development of practice and evidence-based guideline through a participatory process.

Testing the Nurturing Environments Framework on Youth Violence across Ethnically and Geographically Diverse Urban and Rural Samples of Adolescents

Although research advocates for comprehensive cross sector youth violence prevention efforts, mobilizing across sectors to translate scientific recommendations into practice has proven challenging. A unifying framework may provide a foundational step towards building a shared understanding of the risk and protective factors that impact youth violence. We conducted two empirical tests of the nurturing environment framework on youth violence across ethnic and geographically diverse rural and urban adolescent samples. Results show that overall the characteristics of nurturing environments are associated with lower levels of aggression and violence. In addition, minimizing exposure to socially toxic conditions had the strongest associations with lower aggression and violence. Findings were supported across both samples, suggesting that this framework may apply in urban and rural, economically disadvantaged contexts.

A Systematic Review of Workplace Interventions to Rehabilitate Musculoskeletal Disorders Among Employees with Physical Demanding Work

Purpose This systematic review investigates the effectiveness of workplace interventions to rehabilitate musculoskeletal disorders (MSDs) among employees with physically demanding work. Methods A systematic search was conducted in bibliographic databases including PubMed and Web of Science Core Collection for English articles published from 1998 to 2018. The PICO strategy guided the assessment of study relevance and the bibliographical search for randomized controlled trials (RCTs) and non-RCTs in which (1) participants were adult workers with physically demanding work and MSD (including specific and non-specific MSD and musculoskeletal pain, symptoms, and discomfort), (2) interventions were initiated and/or carried out at the workplace, (3) a comparison group was included, and (4) a measure of MSD was reported (including musculoskeletal pain, symptoms, prevalence or discomfort). The quality assessment and evidence synthesis adhered to the guidelines developed by the Institute for Work & Health (Toronto, Canada) focusing on developing practical recommendations for stakeholders. Relevant stakeholders were engaged in the review process. Results Level of evidence from 54 high and medium quality studies showed moderate evidence of a positive effect of physical exercise. Within this domain, there was strong evidence of a positive effect of workplace strength training. There was limited evidence for ergonomics and strong evidence for no benefit of participatory ergonomics, multifaceted interventions, and stress management. No intervention domains were associated with "negative effects". Conclusions The evidence synthesis recommends that implementing strength training at the workplace can reduce MSD among workers with physically demanding work. In regard to workplace ergonomics, there was not enough evidence from the scientific literature to guide current practices. Based on the scientific literature, participatory ergonomics and multifaceted workplace interventions seem to have no beneficial effect on reducing MSD among this group of workers. As these interventional domains were very heterogeneous, it should also be recognized that general conclusions about their effectiveness should be done with care.Systematic review registration PROSPERO CRD42018116752 ( https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=116752 ).

Strategies for effective dissemination of research to United States policymakers: a systematic review

BACKGROUND

Research has the potential to influence US social policy; however, existing research in this area lacks a coherent message. The Model for Dissemination of Research provides a framework through which to synthesize lessons learned from research to date on the process of translating research to US policymakers.

METHODS

The peer-reviewed and grey literature was systematically reviewed to understand common strategies for disseminating social policy research to policymakers in the United States. We searched Academic Search Premier, PolicyFile, SocINDEX, Social Work Abstracts, and Web of Science from January 1980 through December 2019. Articles were independently reviewed and thematically analyzed by two investigators and organized using the Model for Dissemination of Research.

RESULTS

The search resulted in 5225 titles and abstracts for inclusion consideration. 303 full-text articles were reviewed with 27 meeting inclusion criteria. Common sources of research dissemination included government, academic researchers, the peer reviewed literature, and independent organizations. The most frequently disseminated research topics were health-related, and legislators and executive branch administrators were the most common target audience. Print materials and personal communication were the most common channels for disseminating research to policymakers. There was variation in dissemination channels by level of government (e.g., a more formal legislative process at the federal level compared with other levesl). Findings from this work suggest that dissemination is most effective when it starts early, galvanizes support, uses champions and brokers, considers contextual factors, is timely, relevant, and accessible, and knows the players and process.

CONCLUSIONS

Effective dissemination of research to US policymakers exists; yet, rigorous quantitative evaluation is rare. A number of cross-cutting strategies appear to enhance the translation of research evidence into policy.

REGISTRATION

Not registered.

Engaged scholarship and public policy decision-making: a scoping review

BACKGROUND

Engaged scholarship includes the coproduction and use of research by partnerships that blend research, policy and/or practice perspectives. This way of doing research attempts to bridge-the-gap between knowledge and its application. Recent reviews have described practices that support engagement and involve the community in research and patients in healthcare but there is less known about how to engage individuals working to inform public policy.

AIMS AND OBJECTIVES

The purpose of this research was to articulate the actions and context that support the coproduction and use of research to inform public policy decisions. The study focuses on partnerships between researchers and stakeholders working in public policy across different levels and sectors of government.

METHODS

A scoping review methodology was used. Relevant articles were identified from six electronic bibliographic databases of peer-reviewed literature.

FINDINGS

A total of 9904 articles were screened and 375 full-text articles were assessed for eligibility. The included 11 studies were from research partnerships internationally and described actions and contextual factors contributing to the coproduction and use of research to inform public policy. Key actions included facilitating frequent interactions with public policy stakeholders, joint planning for research, and collaboration to execute data collection and analysis. Contextual factors included clarity in responsibilities, prior relationships, and mutual respect for partner priorities and perspectives.

CONCLUSIONS

Key actions and contextual factors were identified in this review and warrant further study to strengthen research-policy partnerships and their outcomes.

Psychosocial Intervention in European Addictive Behaviour Recovery Programmes: A Qualitative Study

BACKGROUND

The vulnerability of drug users in treatment increases their risk of social exclusion. Psychosocial interventions aim to solve or minimise this risk, increasing social integration for individuals in treatment. In this study, the effectiveness of various European drug recovery programmes was studied by examining psychosocial intervention groups with vulnerable populations.

METHODS

Research was conducted utilising qualitative methodology in the following manner: bibliographical review (reports and papers) and 18 interviews with key informants (from nine European countries).

RESULTS

Treatment programmes were found to be effective for recovery from opiate use, however, social recovery programmes were not found to be effective. For females in treatment and young drug users, the adequacy of programmes received medium-to-low evaluations.

CONCLUSION

It is necessary to increase the effectiveness of psychosocial intervention in recovery-based programmes. In addition, the relevance of these programmes in relation to psychosocial characteristics of the various risk groups (except opioid consumers) should be validated and revisited. Additionally, psychosocial intervention professionals should take a more active role in such programmes, especially in intervention with young people (in health education, prevention, early detection, and intervention) and with women (in regard to the detection and effective intervention for socio-health problems, social and job reintegration, access inequality, and use of services available due to gender).

Promoting Evidence-Based Practice for Improved Occupational Safety and Health at Workplaces in Sweden. Report on a Practice-Based Research Network Approach

Despite the rapid growth in research and R&D expenditures, the translation of research into practice is limited. One approach to increase the translation and utilization of research is practice based research networks. With the aim of strengthening evidence-based practice (EBP) within occupational health services in Sweden (OH-Services), a practice-based research network (PBRN-OSH) was developed. The PBRN-OSH includes researchers and representatives from end-users. This paper reports on the development, outputs and lessons learned in the PBRN-OSH. The PBRN-OSH resulted in several practice-based research projects as well as different measures to ensure EBP in OSH such as the governmentally sanctioned national guidelines for the OH-services. Moreover, results show that the competence in EBP increased among practitioners at the OH-services. Conducting research in a PBRN is more resource demanding; however, this does not imply that it is less cost effective. To succeed in increasing the utility of research findings via PBRN, resources must be invested into an infrastructure that supports collaboration in the PBRN, including costs for a variety of means of dissemination. Further, translation activities need to be included in academic career paths and reward systems if a major improvement in the impact and return of investments from research is to be expected.

Taking a systems approach to explore the impacts and outcomes of a research and evaluation capacity building partnership: a protocol

INTRODUCTION

Partnership models that bring researchers, policymakers and service providers closer together are gaining traction as a strategy to improve public health practice. Yet, there is little evidence of how these models work, or indeed if they do work. The Sexual Health and Blood-borne Virus Applied Research and Evaluation Network (SiREN) is one such model. SiREN is a partnership between researchers, policymakers and service providers that aims to develop the research and evaluation capacity and evidence-informed decision making capability of professionals working to address sexual health and bloodborne virus issues in Western Australia. This study will use a systems approach to identify the mechanisms of action, impacts and outcomes of SiREN and inform the development of evaluation tools.

METHODS AND ANALYSIS

Data will be collected from organisational documents, surveys, in-depth interviews and a workshop. It will be analysed using a complex adaptive systems lens and findings will be used to inform the development of a type of qualitative systems model called a causal loop diagram. The causal loop diagram will illustrate the: contextual factors influencing engagement; mechanisms of action; and impacts and outcomes of SiREN. Evaluation tools will then be developed that can be used to assess the indicators identified in the causal loop diagram.

ETHICS AND DISSEMINATION

Ethics approval was obtained from the Curtin University Human Research Ethics Committee (approval number: HRE2017-0090). Participants will be free to withdraw from the study at any point and confidentiality will be maintained by de-identifying participant responses in any published or shared data. The findings from this study will be shared in conference presentations, reports, peer-reviewed journals and online through websites and social media.

Differences in implementation of family focused practice in hospitals: a cross-sectional study

BACKGROUND

Changes in Norwegian law and health policy require all health professionals to help safeguard the provision of information and follow-up for the children of parents with mental or physical illness, or substance abuse problems, to decrease their risk of psychosocial problems. There is a lack of knowledge on how the national changes have been received by hospital-based health professionals, and if they have led to an increase in family focused practice.

METHODS

This cross-sectional study examined the adherence of health professionals' (N = 280) in five hospitals to new guidelines for family focused practice, using a translated and generic version of Family Focused Mental Health Practice Questionnaire.

RESULTS

Overall, health professionals scored high on knowledge and skills, and were confident in working with families and children, but reported moderate levels of family support and referrals. Comparison of the five hospitals showed significant differences in terms of workplace support, knowledge and skills and family support. The smallest hospital had less workplace support and less knowledge and skills but scored medium on family support. The two largest hospitals scored highest on family support, but with significant differences on parents refusing to have conversations with children.

CONCLUSIONS

Differences in implementation of family focused practice highlight the need to tailor improvement strategies to specific barriers at the different hospitals. The use of implementation theories and improvement strategies could promote full implementation, where all families and children in need were identified and had access to family support.Trial registration The study is approved by the Regional Committee on Medical and Health Research Ethics South-East Q5 37 (reg. no. 2012/1176) and by the Privacy Ombudsman.

Hospitals implementing changes in law to protect children of ill parents: a cross-sectional study

BACKGROUND

Norway is one of the first countries to require all health professionals to play a part in prevention for children of parents with all kinds of illnesses (mental illness, drug addiction, or severe physical illness or injury) in order to mitigate their increased risk of psychosocial problems. Hospitals are required to have child responsible personnel (CRP) to promote and coordinate support given by health professionals to patients who are parents and to their children.

METHODS

This study examined the extent to which the new law had been implemented as intended in Norwegian hospitals, using Fixsen's Active Implementation Framework. A stratified random sample of managers and child responsible personnel (n = 167) from five Hospitals filled in an adapted version of the Implementation Components Questionnaire (ICQ) about the implementation of policy changes. Additional information was collected from 21 hospital coordinators (H-CRP) from 16 other hospitals.

RESULTS

Significant differences were found between the five hospitals, with lowest score from the smallest hopitals. Additional analysis, comparing the 21 hospitals, as reported by the H-CRP, suggests a clear pattern of smaller hospitals having less innovative resources to implement the policy changes. Leadership, resources and system intervention (strategies to work with other systems) were key predictors of a more successful implementation process.

CONCLUSIONS

Legal changes are helpful, but quality improvements are needed to secure equal chances of protection and support for children of ill parents.

TRIAL REGISTRATION

The study is approved by the Regional Committee on Medical and Health Research Etics South-East (reg.no. 2012/1176 ) and by the Privacy Ombudsmann.

Economic Impact of the Statewide Implementation of an Evidence-Based Treatment: Multisystemic Therapy in New Mexico

Several states have made considerable investments into large-scale implementation of evidence-based treatments (EBTs), yet little is known about key success indicators for these implementation efforts such as cost and sustainability. To that end, the present study examined the economic impact of statewide implementation of multisystemic therapy (MST; Henggeler, Schoenwald, Borduin, Rowland, & Cuningham, 2009), a family- and community-based behavioral EBT for serious juvenile offenders in New Mexico. Participants were 1,869 youth who received MST across 23 teams during the study period. We conducted a cost-benefit analysis using metrics from state data sources that compared the cost of MST to its benefits (i.e., avoided expenses from pre- to posttreatment) in two domains: (a) behavioral health services (i.e., Medicaid claims) and (b) juvenile crime (i.e., taxpayer expenses, tangible and intangible expenses to crime victims). MST costs were based on Medicaid claims, which were reimbursed at an enhanced billing rate that was intended to cover expenses for both clinical and implementation (e.g., training, quality assurance) activities. Results suggest that implementation of MST in New Mexico over the 7-year study period may have produced net benefits, through 2 years posttreatment, of more than $4,643 per youth in avoided behavioral health claims and $15,019 per youth through reductions in juvenile crime. Stated differently, every dollar that New Mexico spent on MST appeared to have returned $3.34 for a total benefit of $64.2 million over the course of the study. We discuss implications of these findings for policymakers, administrators, and researchers who are interested in increasing the sustainability of complex EBTs in community settings.

A research and evaluation capacity building model in Western Australia

Evaluation of public health programs, services and policies is increasingly required to demonstrate effectiveness. Funding constraints necessitate that existing programs, services and policies be evaluated and their findings disseminated. Evidence-informed practice and policy is also desirable to maximise investments in public health. Partnerships between public health researchers, service providers and policymakers can help address evaluation knowledge and skills gaps. The Western Australian Sexual Health and Blood-borne Virus Applied Research and Evaluation Network (SiREN) aims to build research and evaluation capacity in the sexual health and blood-borne virus sector in Western Australia (WA). Partners' perspectives of the SiREN model after 2 years were explored. Qualitative written responses from service providers, policymakers and researchers about the SiREN model were analysed thematically. Service providers reported that participation in SiREN prompted them to consider evaluation earlier in the planning process and increased their appreciation of the value of evaluation. Policymakers noted benefits of the model in generating local evidence and highlighting local issues of importance for consideration at a national level. Researchers identified challenges communicating the services available through SiREN and the time investment needed to develop effective collaborative partnerships. Stronger engagement between public health researchers, service providers and policymakers through collaborative partnerships has the potential to improve evidence generation and evidence translation. These outcomes require long-term funding and commitment from all partners to develop and maintain partnerships. Ongoing monitoring and evaluation can ensure the partnership remains responsive to the needs of key stakeholders. The findings are applicable to many sectors.

The Promise of Prevention Science for Addressing Intergenerational Poverty

This paper reviews research suggesting that the prevention of intergenerational poverty will be enhanced if we add evidence-based family and school prevention programs to existing efforts to reduce poverty in order to address the adverse social environments that often accompany poverty. Government policies such as the Earned Income Tax Credit can reduce family poverty, but simply improving the economic stability of the family will not necessarily prevent the development of child and adolescent problems such as academic failure, antisocial behavior, drug abuse, and depression, all of which can undermine future economic wellbeing. We briefly review the evidence linking family poverty to adverse social environments, which can have deleterious effects on children's psychological, behavioral, neurological, and physical development. We then document the value of evidence-based family- and school-based prevention programs in effectively addressing these behavioral, emotional, cognitive, and neurophysiological factors that can put children at risk for continued poverty in adulthood. We also describe three family-based prevention programs that have been found to have a direct effect on families' future economic wellbeing. The evidence indicates that widely disseminating effective and efficient family- and school-based prevention programs can help to address both poverty itself and the effects of adverse social environments, making future poverty less likely. We conclude with specific recommendations for federal and state policymakers, researchers, and practitioners.

What can we learn from interventions that aim to increase policy-makers' capacity to use research? A realist scoping review

Background

Health policy-making can benefit from more effective use of research. In many policy settings there is scope to increase capacity for using research individually and organisationally, but little is known about what strategies work best in which circumstances. This review addresses the question: What causal mechanisms can best explain the observed outcomes of interventions that aim to increase policy-makers’ capacity to use research in their work?

Methods

Articles were identified from three available reviews and two databases (PAIS and WoS; 1999–2016). Using a realist approach, articles were reviewed for information about contexts, outcomes (including process effects) and possible causal mechanisms. Strategy + Context + Mechanism = Outcomes (SCMO) configurations were developed, drawing on theory and findings from other studies to develop tentative hypotheses that might be applicable across a range of intervention sites.

Results

We found 22 studies that spanned 18 countries. There were two dominant design strategies (needs-based tailoring and multi-component design) and 18 intervention strategies targeting four domains of capacity, namely access to research, skills improvement, systems improvement and interaction. Many potential mechanisms were identified as well as some enduring contextual characteristics that all interventions should consider. The evidence was variable, but the SCMO analysis suggested that tailored interactive workshops supported by goal-focused mentoring, and genuine collaboration, seem particularly promising. Systems supports and platforms for cross-sector collaboration are likely to play crucial roles. Gaps in the literature are discussed.

Conclusion

This exploratory review tentatively posits causal mechanisms that might explain how intervention strategies work in different contexts to build capacity for using research in policy-making.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0277-1) contains supplementary material, which is available to authorized users.

Community-Sourced Intervention Programs: Review of Submissions in Response to a Statewide Call for "Promising Practices"

This study was initiated to add to the nascent literature on locally-grown intervention programs in the youth mental health, child welfare, and juvenile justice service sectors, many of which demonstrate practice-based or community-defined evidence, but may not have been subjected to empirical evaluation. Characteristics of applications submitted in response to three public calls for additions to an inventory of research-supported intervention programs were reviewed on evidence for effectiveness, the use of key quality assurance (QA) elements (e.g., clearly specified training or integrity monitoring procedures), and cultural specificity. Findings indicate that four QA processes were identified in approximately half of all submissions: a specific initial training process, the existence of intervention integrity measures, routine outcome monitoring, and ongoing support post-training. An initial training process and integrity measurement were more commonly described among programs determined to have greater research evidence for their effectiveness. Overall, cultural elements were described relatively infrequently and most often reflected surface-level program delivery characteristics (e.g., offering services in languages other than English). Discussion is focused on the alignment of submitted programs with the larger literatures focused on implementation science and cultural competence.

The Consistencies and Vagaries of the Washington State Inventory of Evidence-Based Practice: The Definition of "Evidence-Based" in a Policy Context

As states increasingly establish the importance of evidence-based practice through policy and funding mandates, the definition of evidence-based practice can have a significant impact on investment decisions. Not meeting established criteria can mean a loss of funding for established programs and the implementation disruption of programs without a strong research base. Whether the definition of "evidence-based" is influenced by these high stakes contexts is an interesting question that can inform the larger field about the value and utility of evidence-based practice lists/inventories for disseminating knowledge. In this paper we review the development of the Washington State Inventory of Evidence-Based, Research-Based and Promising Practices as a case study for the process of defining evidence-based practice in a policy context. As part of this study we also present a comparison of other well-known evidence-based practice inventories and examine consistencies and differences in the process of identifying and developing program ratings.

Developing a Quality Assurance System for Multiple Evidence Based Practices in a Statewide Service Improvement Initiative

Efforts to implement evidence based practices (EBP) are increasingly common in child-serving systems. However, public systems undertaking comprehensive improvement efforts that aim to increase availability of multiple practices at the same time may struggle to build comprehensive and user-friendly strategies to develop the workforce and encourage adoption, faithful implementation, and sustainability of selected EBPs. Given that research shows model adherence predicts positive outcomes, one critical EBP implementation support is systematic quality, fidelity, and compliance monitoring. This paper describes the development and initial implementation of a quality assurance framework for a statewide EBP initiative within child welfare. This initiative aimed to improve provider practice and monitor provider competence and compliance across four different EBPs, and to inform funding and policy decisions. The paper presents preliminary data as an illustration of lessons learned during the quality monitoring process and concludes with a discussion of the promise and challenges of developing and applying a multi-EBP quality assurance framework for use in public systems.

Identifying a practice-based implementation framework for sustainable interventions for improving the evolving working environment: Hitting the Moving Target Framework

Our aim was to identify implementation components for sustainable working environment interventions in the nursing assistant sector to generate a framework to optimize the implementation of workplace improvement initiatives. The implementation framework was informed by: 1) an industry advisory group, 2) interviews with key stakeholder, 3) concept mapping workshops, and 4) an e-mail survey. Thirty five stakeholders were interviewed and contributed in the concept mapping workshops. Eleven implementation components were derived across four domains: 1) A supportive organizational platform, 2) An engaged workplace with mutual goals, 3) The intervention is sustainably fitted to the workplace, and 4) the intervention is an attractive choice. The highest rated component was "Engaged and Active Management" (mean 4.1) and the lowest rated was "Delivered in an Attractive Form" (mean 2.8). The framework provides new insights into implementation in an evolving working environment and is aiming to assist with addressing gaps in effectiveness of workplace interventions and implementation success.

Toward Dynamic Adaptation of Psychological Interventions for Child and Adolescent Development and Mental Health

Children's and adolescents' mental health needs emphasize the necessity of a new era of translational research to enhance development and yield better lives for children, families, and communities. Developmental, clinical, and translational research serves as a powerful tool for managing the inevitable complexities in pursuit of these goals. This article proposes key ideas that will strengthen current evidence-based intervention practices by creating stronger links between research, practice, and complex systems contexts, with the potential of extending applicability, replicability, and impact. As exemplified in some of the articles throughout this special issue, new research and innovative implementation models will likely contribute to better ways of assessing and dynamically adapting structure and intervention practice within mental health systems. We contend that future models for effective interventions with children and adolescents will involve increased attention to (a) the connection of research on the developmental needs of children and adolescents to practice models; (b) consideration of informed contextual and cultural adaptation in implementation; and (c) a rational model of evidence-based planning, using a dynamic, inclusive approach with high support for adaptation, flexibility, and implementation fidelity. We discuss future directions for translational research for researchers, practitioners, and administrators in the field to continue and transform these ideas and their illustrations.

CREATE-ing capacity to take developmental crime prevention to scale: A community-based approach within a national framework

Developmental crime prevention is founded on the long-term outcomes and economic efficiency of about 50 promising or model programs for fostering healthy child and youth development and for preventing crime. However, few if any of these programs have been successfully implemented on a large scale, a problem that is the focus of Type 2 (T2) Translation Research within prevention science. This paper describes one approach to building capacity for population-level community-based developmental prevention using the CREATE model that we developed as an outcome of the Pathways to Prevention Project that operated in a disadvantaged region of Brisbane between 2002 and 2011. CREATE is an acronym: Collaborative; Relationships-driven; Early in the pathway; Accountable; Training-focused; Evidence-driven. CREATE is being used to develop, in T2 Translation terms, a prevention support system (PSS) for the Communities for Children (CfC) program, a prevention delivery system that is operated by the Department of Social Services in 52 communities across Australia. The aim is to build the capacity for schools and community agencies to transcend system silos; foster ethical practices and respectful relationships; and deliver goal-directed, quantitatively evaluated, evidence-based resources that address the needs of families with complex needs, promote child wellbeing and prevent antisocial and criminal behaviours. The PSS combines web-based interactive electronic resources for schools and community agencies serving children in CfC communities with systems and processes established by project personnel called Collective Impact Facilitators who build the skills and knowledge of community coalitions to use the electronic resources and implement CREATE effectively. This capacity building exercise is being evaluated through a comprehensive array of pre- and post-measures of coalition functioning. The PSS integrates with national prevention infrastructure developed by DSS, including a Data Exchange System, an Expert Panel, and an Information Exchange.

The Critical Role of State Agencies in the Age of Evidence-Based Approaches: The Challenge of New Expectations

Evidence-based approaches only benefit individuals when fully and effectively implemented. Since funding and monitoring alone will not ensure the full and effective implementation of effective strategies, state agencies have the opportunity to assess and modify current roles, functions, and policies to align with the requirements of evidence-based strategies. Based on a growing body of knowledge to guide effective implementation processes, state agencies, or designated partner organizations, can develop the capacity, mechanisms, and infrastructure to effectively implement evidence-based strategies. This article describes a framework that can guide this process. Informed by the literature and shaped by "real-world experience," the Active Implementation Frameworks provide a stage-matched approach to purposeful, active, and effective implementation.

Free, brief, and validated: Standardized instruments for low-resource mental health settings

Evidence-based assessment has received little attention despite its critical importance to the evidence-based practice movement. Given the limited resources in the public sector, it is necessary for evidence-based assessment to utilize tools with established reliability and validity metrics that are free, easily accessible, and brief. We review tools that meet these criteria for youth and adult mental health for the most prevalent mental health disorders to provide a clinical guide and reference for the selection of assessment tools for public sector settings. We also discuss recommendations for how to move forward the evidence-based assessment agenda.

Implementing and sustaining evidence-based practice in juvenile justice: a case study of a rural state

US juvenile justice is at the forefront of experimentation with the evidence-based paradigm, whereby the best available research is utilized to help inform more rational and effective practice. Increasingly, state governments are playing a major role in this endeavor. Maine is one of these states and is the focus of this article. Using a case-study design, we set out to develop a fuller understanding of the events and processes that have contributed to the development, implementation, and sustainment of evidence-based practice in juvenile justice in the state. Four major themes emerged. First, Maine has benefited from strong and lasting leadership within its corrections department. These leaders paved the way for the implementation and sustainment of programs, including finding innovative ways to use existing resources. Second, the adoption of the Risk-Need-Responsivity model was important in laying the groundwork for the use of evidence-based programming. Third, collaborations within and among state agencies and public and private groups were essential. Finally, buy-in and support from multiple stakeholders was and continues to be essential to Maine's work. Ongoing problems remain with respect to ensuring agencies prioritize fidelity to the model and locating increasingly scarce funding. Implications for other states are discussed.

Addressing core challenges for the next generation of type 2 translation research and systems: the translation science to population impact (TSci Impact) framework

Evidence-based preventive interventions developed over the past two decades represent great potential for enhancing public health and well-being. Research confirming the limited extent to which these interventions have been broadly and effectively implemented, however, indicates much progress is needed to achieve population-level impact. In part, progress requires Type 2 translation research that investigates the complex processes and systems through which evidence-based interventions are adopted, implemented, and sustained on a large scale, with a strong orientation toward devising empirically-driven strategies for increasing their population impact. In this article, we address two core challenges to the advancement of T2 translation research: (1) building infrastructure and capacity to support systems-oriented scaling up of evidence-based interventions, with well-integrated practice-oriented T2 research, and (2) developing an agenda and improving research methods for advancing T2 translation science. We also summarize a heuristic "Translation Science to Population Impact (TSci Impact) Framework." It articulates key considerations in addressing the core challenges, with three components that represent: (1) four phases of translation functions to be investigated (pre-adoption, adoption, implementation, and sustainability); (2) the multiple contexts in which translation occurs, ranging from community to national levels; and (3) necessary practice and research infrastructure supports. Discussion of the framework addresses the critical roles of practitioner-scientist partnerships and networks, governmental agencies and policies at all levels, plus financing partnerships and structures, all required for both infrastructure development and advances in the science. The article concludes with two sets of recommended action steps that could provide impetus for advancing the next generation of T2 translation science and, in turn, potentially enhance the health and well-being of subsequent generations of youth and families.

Building a science of partnership-focused research: forging and sustaining partnerships to support child mental health prevention and services research

Building on growing interest in translational research, this paper provides an overview of a special issue of Administration and Policy in Mental Health and Mental Health Service Research, which is focused on the process of forging and sustaining partnerships to support child mental health prevention and services research. We propose that partnership-focused research is a subdiscipline of translational research which requires additional research to better refine the theoretical framework and the core principles that will guide future research and training efforts. We summarize some of the major themes across the eight original articles and three commentaries included in the special issue. By advancing the science of partnership-focused research we will be able to bridge the gap between child mental health prevention and services research and practice.