"Developing good taste in evidence": facilitators of and hindrances to evidence-informed health policymaking in state government

Investigating the trustworthiness of research evidence used to inform public health policy: a qualitative interview study on the use of predatory journal citations in policy documents

Evidence-based policymaking has increased policymakers' capacity to make scientifically informed health policy decisions. However, reaping the benefits of this approach requires avoiding untrustworthy research - potential sources of which are predatory journals. In this study, we sought to understand how research cited in policy documents is sourced and evaluated, and identify factors that may be contributing to the citation of predatory journals or other less trustworthy evidence. To this end, we conducted semi-structured interviews with individuals who have prepared public health policy documents. These interviews were thematically analysed, and five key overarching themes were generated regarding the process of deciding how to develop policy documents (e.g. which individuals to involve) and how this may impact which information is included; obstacles such as limited evidence that may hinder policy document development; and concerns around transparency throughout the development process. Our findings highlight that in many cases, information cited in policy documents is sourced and evaluated with variable rigour. This may contribute to the citation of untrustworthy research in policy documents. Certain steps can be taken to help minimize any potential negative impact of relying on such sources (e.g. improving transparency), but a better understanding of policymakers' perspectives regarding how taking these steps would impact their decision-making process may be required to ensure successful implementation.

Longitudinal policy surveillance of state obesity legislation in California, 1999-2020

BACKGROUND

Obesity rates among children and adults continue to accelerate in the U.S., particularly among marginalized and low-income populations. Obesity prevention and reduction policies can significantly impact population health by improving environmental conditions and increasing access to health-promoting resources. Limited research has been conducted to examine state obesity policy change over time. The primary aim of this study is to examine legislative approaches used to prevent and reduce obesity in the state of California (U.S.).

METHODS

We used quantitative policy surveillance methods to develop a state database of obesity-related legislation (bills, resolutions) introduced in California's legislature between 1999 and 2020. Descriptive statistics were used to examine trends of introduced and enacted policy by legislative and policy characteristics. Chi-square tests were used to determine differences in characteristics between enacted and non-enacted legislation. Legislative session and policy characteristics found to be associated with enactment were used to predict adoption in a logistic regression.

RESULTS

A total of 284 obesity-related bills and resolutions were introduced in California's legislature between 1999 and 2020 with a peak of 43 in 2005-2006. On average, 25.8 bills and resolutions were introduced each 2-year legislative cycle. Findings indicate that (a) children and schools were the most frequently specified population and setting; (b) the most common policy topics were nutrition (45%) and physical activity (33%); and (c) only 15% of legislation mentioned race/ethnicity. Overall, 24.9% of bills were enacted compared to 82.1% of resolutions adopted. Legislation to raise awareness about obesity had 5.4 times the odds of being passed compared to other topics. Yet this difference was not statistically significant in a sensitivity analysis when we excluded resolutions.

CONCLUSIONS

This database can be leveraged to advance our knowledge of effective and equitable policy instruments to prevent and reduce obesity. Results reveal important policy elements that may impact legislative success, including policy topic, and contribute to a nascent evidence base for public health law research, legal epidemiology, and practice. Future work should investigate the role of policy effectiveness research and evidence on legislative policymaking.

Perspectives of California Legislators on Institutional Barriers and Facilitators to Non-Partisan Research Evidence Use in State Health Policymaking

BACKGROUND

Bridging the translational gap between research evidence and health policy in state legislatures requires understanding the institutional barriers and facilitators to non-partisan research evidence use. Previous studies have identified individual-level barriers and facilitators to research evidence use, but limited perspectives exist on institutional factors within legislatures that influence non-partisan research evidence use in health policymaking.

OBJECTIVE

We describe the perspectives of California state legislators and legislative staff on institutional barriers and facilitators of non-partisan research evidence use in health policymaking and explore potential solutions for enhancing use.

DESIGN

Case study design involving qualitative interviews.

PARTICIPANTS

We interviewed 24 California state legislators, legislative office staff, and legislative research staff.

APPROACH

Semi-structured recorded interviews were conducted in person or by phone to identify opportunities for enhancing non-partisan research evidence use within state legislatures. We conducted thematic analyses of interview transcripts to identify (1) when research evidence is used during the policymaking process, (2) barriers and facilitators operating at the institutional level, and (3) potential solutions for enhancing evidence use.

RESULTS

Institutional barriers to non-partisan research evidence use in health policymaking were grouped into three themes: institutional policies, practices, and priorities. Interviews also revealed institutional-level facilitators of research evidence use, including (1) access and capacity to engage with research evidence, and (2) perceived credibility of research evidence. The most widely supported institutional-level solution for enhancing evidence-based health policymaking in state legislatures involved establishing independent, impartial research entities to provide legislators with trusted evidence to inform decision-making.

CONCLUSIONS

Potential institutional-level changes within state legislatures may enhance evidence use in health policymaking, leading to improved health outcomes and lower healthcare costs for states.

Evaluating vonoprazan and tegoprazan for gastroesophageal reflux disease treatment in Chinese Healthcare: an EVIDEM framework analysis

BACKGROUND

In Chinese healthcare settings, drug selection decisions are predominantly influenced by the Pharmacy & Therapeutics Committee (PTC). This study evaluates two recently introduced potassium-competitive acid blockers, vonoprazan (VPZ) and tegoprazan (TPZ), utilizing the Evidence and Value: Impact on DEcisionMaking (EVIDEM) framework.

METHODS

The study employed the 10th edition of EVIDEM, which includes a core model with five domains and 13 criteria. Two independent expert panels were involved: the PTC expert panel, tasked with assigning weights using a 5-point scale, defining scoring indicators, examining the evidence matrix, scoring, and decision-making; and the evidence matrix expert panel, responsible for conducting a systematic literature review, creating the evidence matrix, and evaluating the value contributions of VPZ and TPZ.

RESULTS

The analysis estimated the value contributions of VPZ and TPZ to be 0.59 and 0.54, respectively. The domain of 'economic consequences of intervention' showed the most significant variation in value contribution between the two drugs, followed by 'comparative outcomes of intervention' and 'type of benefit of intervention'.

CONCLUSION

Employing the EVIDEM framework, VPZ's value contribution was found to be marginally superior to that of TPZ. The EVIDEM framework demonstrates potential for broader application in Chinese medical institutions.

Using Policy Briefs to Communicate Dental Research Findings to Policymakers

OBJECTIVES

New scientific knowledge is not always available to decision makers. Policy briefs are a way that dental researchers can communicate research findings to policymakers. This study compares usefulness of 2 types of policy briefs about sugar-sweetened beverage (SSB) intake and tooth decay.

METHODS

We developed 2 policy brief types (data focused and narrative focused) and emailed a randomly assigned policy brief to 825 policymakers and staff from 3 levels of government (city, county, and state) in Washington State. Participants completed a 22-item online questionnaire. There were 4 study outcomes: whether the brief was understandable, whether the brief was credible, likelihood of use, and likelihood to be shared (each measured on a 5-point Likert-like scale). The t test was used to evaluate whether outcomes differed by policy brief type and government level (α = 0.05).

RESULTS

There were 108 respondents (adjusted response rate 14.6%). About 41.6% of participants were in city government, 26.9% were in county government, and 29.6% were in state government. Participants reported that both data- and narrative-focused briefs were understandable (mean rating [MR] and standard deviation [SD]: 4.15 ± 0.68 and 4.09 ± 0.81, respectively; P = 0.65) and credible (MR and SD: 4.13 ± 0.70 and 4.09 ± 0.70, respectively; P = 0.74), but they were not likely to use (MR and SD: 2.71 ± 1.15 and 2.55 ± 1.28, respectively; P = 0.51) or share it (MR and SD: 2.62 ± 1.04 and 2.66 ± 1.30, respectively; P = 0.87). The likelihood of sharing briefs differed significantly by level of government (P = 0.017). Participants at the state level were more likely to share information from the briefs (mean rating and SD: 3.10 ± 0.80) than city- and county-level participants (MR and SD: 2.62 ± 1.27, and 2.24 ± 1.21, respectively).

CONCLUSION

Both data- and narrative-focused policy briefs may be a useful way to communicate dental research findings to policymakers, but additional steps are needed to ensure that briefs are used and shared.

KNOWLEDGE TRANSFER STATEMENT

Researchers should disseminate their research findings to maximize scientific impact. Our study findings indicate that policy briefs may be a useful way to communicate dental research findings to policymakers, but additional research is needed on the best ways to disseminate findings.

Challenges for research uptake for health policymaking and practice in low- and middle-income countries: a scoping review

BACKGROUND

An estimated 85% of research resources are wasted worldwide, while there is growing demand for context-based evidence-informed health policymaking. In low- and middle-income countries (LMICs), research uptake for health policymaking and practice is even lower, while little is known about the barriers to the translation of health evidence to policy and local implementation. We aimed to compile the current evidence on barriers to uptake of research in health policy and practice in LMICs using scoping review.

METHODS

The scoping review followed the Preferred Reporting Items for Systematic Review and Meta-Analyses-extension for Scoping Reviews (PRISMA-ScR) and the Arksey and O'Malley framework. Both published evidence and grey literature on research uptake were systematically searched from major databases (PubMed, Cochrane Library, CINAHL (EBSCO), Global Health (Ovid)) and direct Google Scholar. Literature exploring barriers to uptake of research evidence in health policy and practice in LMICs were included and their key findings were synthesized using thematic areas to address the review question.

RESULTS

A total of 4291 publications were retrieved in the initial search, of which 142 were included meeting the eligibility criteria. Overall, research uptake for policymaking and practice in LMICs was very low. The challenges to research uptake were related to lack of understanding of the local contexts, low political priority, poor stakeholder engagement and partnership, resource and capacity constraints, low system response for accountability and lack of communication and dissemination platforms.

CONCLUSION

Important barriers to research uptake, mainly limited contextual understanding and low participation of key stakeholders and ownership, have been identified. Understanding the local research and policy context and participatory evidence production and dissemination may promote research uptake for policy and practice. Institutions that bridge the chasm between knowledge formation, evidence synthesis and translation may play critical role in the translation process.

Examining and Contextualizing Approaches to Establish Policy Support Organizations - A Mixed Method Study

BACKGROUND

There has been an increase in the number of policy support organizations (PSOs) that have been created to foster the systematic use of evidence in health system policymaking. Our aim was to identify approaches for establishing a PSO or similar entities by soliciting insights from those with practical experience with developing and operationalizing PSOs in real-world contexts.

METHODS

We used a sequential mixed method approached. We first conducted a survey to identify the views and experiences of those who were directly involved in the establishment of PSOs that have been developed and implemented across a variety of political-, health- and research-system contexts. The survey findings were then used to develop a purposive sample of PSO leaders and refine an interview guide for interviews with them.

RESULTS

We received 19 completed surveys from leaders of PSOs in countries across the WHO regions and that operate in different settings (eg, as independent organization or within a university or government department) and conducted interviews with 15 senior managers from nine PSOs. Our findings provide in-depth insights about approaches and strategies across four stages for establishing a PSO, which include: (i) building awareness for the PSO; (ii) developing the PSO; (iii) assessing the PSO to identify potential areas for enhancement; and (iv) supporting maturation to build sustainability in the long-term. Our findings provide rich insights about the process of establishing a PSO from leaders who have undertaken the process.

CONCLUSION

While all PSOs share the same objective in supporting evidence-informed policy-making (EIPM), there is no single approach that can be considered to be the most successful in establishing a PSO, and each country should identify the approach based on its context.

A knowledge implementation model in health system management based on the PARIHS model

BACKGROUND

The gap between knowledge and practice, along with postponing or not implementing research findings in practice and policy-making, is one of the reasons for low-quality services. Hence, this study aimed at presenting a model of knowledge implementation in health system management in Iran.

METHODS

The present two-phase study was first performed qualitatively using a directive content analysis approach based on the Promoting Action on Research Implementation in Health Services (PARIHS) model. The researchers extracted the barriers and facilitators by conducting semi-structured individual interviews. Then, in a three-stage Delphi study, 25 health experts determined the barrier removal strategies. Data were analysed using MAXQDA10 software.

RESULTS

 The content analysis of the interviews led to the emergence of 1212 codes under three categories of evidence, context and facilitation. The findings indicate that health managers make fewer decisions based on research findings. Instead, they make decisions regarding the experiences of service providers and organization data. In addition to the subcategories in the PARIHS model, the researchers extracted political, social and administrative factors under the context category. The relationships between the features of evidence, context, facilitation, barriers and strategies were presented in the final model.

CONCLUSION

 The presented model comprehensively emphasizes the evidence resources, context preparation, and facilitation of the knowledge implementation process.

Expert participation in 25 years of Wisconsin abortion policymaking

OBJECTIVE

State-level abortion restrictions grew considerably in number over the last two decades. This study examines the scope of expert testimony given in legislative committee hearings at which these laws are first debated.

STUDY DESIGN

We gathered 265 testimonies given by experts at Wisconsin legislative committee hearings on 34 abortion bills from 1995-2019. We coded testimonies to identify testifiers' ideological leaning and source of expertise. We conducted descriptive analyses of testifiers' participation.

RESULTS

Experts with anti-abortion rights views testified more often than experts with pro-abortion rights views (2.1 vs 1.4 testimonies per expert). Experts with an activism background testified more often than experts in medicine (2.5 vs 1.3 testimonies per expert). Anti-abortion activist experts represented the largest proportion of testimonies (32%) but the smallest proportion of testifiers (16%). Pro-abortion rights medical experts gave the fewest testimonies (24%) relative to their proportion of testifiers (31%). The number of testimonies given by pro-abortion rights activist experts remained stable over the study period. Testimonies given by all other kinds of experts were more numerous in recent years.

CONCLUSIONS

The experts who testify most frequently tend to espouse anti-abortion views and have backgrounds in activism rather than healthcare. These repeat testifiers may have more opportunities to build relationships with legislators and thus influence policy. Anti-abortion rights activist experts' outsized role in legislative hearings, especially in recent years, should concern advocates of evidence-based reproductive health policy. Medical experts may be deterred from giving testimony by logistical or other structural barriers in the legislative process.

IMPLICATIONS

The family planning field should conduct more research on the role of experts in abortion policymaking. Future studies should examine testifiers in other states and identify barriers pro-abortion medical experts may face to testifying, as these experts are key for creating evidence-based abortion policy.

[Promotion of physical activity among the elderly as an example of knowledge translation: How do scientific findings enter community practice?]

BACKGROUND

Scientific findings can be an important source of knowledge for public health stakeholders involved in promoting physical activity, but several barriers hinder their use. Knowledge translation can simplify this process, but it requires the understanding of the stakeholder's needs.

OBJECTIVES

This qualitative study aims to describe how public health stakeholders access information and scientific findings, identify possible barriers, and highlight the needs of stakeholders in terms of presentation and processing.

MATERIALS AND METHODS

Semi-structured interviews were conducted with twelve local- and state-level stakeholders from North Rhine-Westphalia, Saxony-Anhalt, and Thuringia working in the area of physical activity promotion. The interviewees were selected through purposive sampling. The interviews were evaluated using qualitative content analysis.

RESULTS

The benefits of scientific findings are emphasized by the interviewees, but a lack of resources in combination with a flood of information, high complexity, and technical jargon complicate their application. There is a need for tailored preparation in the form of summaries, filter functions, elaboration of practice-relevant elements, and ways of provision.

CONCLUSIONS

To achieve successful knowledge translation, collaboration and interactive exchange between researchers, policymakers, and practice as well as a demand-oriented processing of scientific findings are central. Networking and bundling of knowledge on a platform are important tasks for the future.

Linking Data on Constituent Health with Elected Officials' Opinions: Associations Between Urban Health Disparities and Mayoral Officials' Beliefs About Health Disparities in Their Cities

Policy Points Mayoral officials' opinions about the existence and fairness of health disparities in their city are positively associated with the magnitude of income-based life expectancy disparity in their city. Associations between mayoral officials' opinions about health disparities in their city and the magnitude of life expectancy disparity in their city are not moderated by the social or fiscal ideology of mayoral officials or the ideology of their constituents. Highly visible and publicized information about mortality disparities, such as that related to COVID-19 disparities, has potential to elevate elected officials' perceptions of the severity of health disparities and influence their opinions about the issue.

CONTEXT

A substantive body of research has explored what factors influence elected officials' opinions about health issues. However, no studies have assessed the potential influence of the health of an elected official's constituents. We assessed whether the magnitude of income-based life expectancy disparity within a city was associated with the opinions of that city's mayoral official (i.e., mayor or deputy mayor) about health disparities in their city.

METHODS

The independent variable was the magnitude of income-based life expectancy disparity in US cities. The magnitude was determined by linking 2010-2015 estimates of life expectancy and median household income for 8,434 census tracts in 224 cities. The dependent variables were mayoral officials' opinions from a 2016 survey about the existence and fairness of health disparities in their city (n = 224, response rate 30.3%). Multivariable logistic regression was used to adjust for characteristics of mayoral officials (e.g., ideology) and city characteristics.

FINDINGS

In cities in the highest income-based life expectancy disparity quartile, 50.0% of mayoral officials "strongly agreed" that health disparities existed and 52.7% believed health disparities were "very unfair." In comparison, among mayoral officials in cities in the lowest disparity quartile 33.9% "strongly agreed" that health disparities existed and 22.2% believed the disparities were "very unfair." A 1-year-larger income-based life expectancy disparity in a city was associated with 25% higher odds that the city's mayoral official would "strongly agree" that health disparities existed (odds ratio [OR] = 1.25; P = .04) and twice the odds that the city's mayoral official would believe that such disparities were "very unfair" (OR = 2.24; P <.001).

CONCLUSIONS

Mayoral officials' opinions about health disparities in their jurisdictions are generally aligned with, and potentially influenced by, information about the magnitude of income-based life expectancy disparities among their constituents.

Accelerating the conceptual use of behavioral health research in juvenile court decision-making: study protocol

BACKGROUND

The youth criminal-legal system is under heavy political scrutiny with multiple calls for significant transformation. Leaders within the system are faced with rethinking traditional models and are likely to benefit from behavioral health research evidence as they redesign systems. Little is known about how juvenile court systems access and use behavioral health research evidence; further, the field lacks a validated survey measure of behavioral health research use that can be used to evaluate the effectiveness of evidence dissemination interventions for policy and system leaders. Conceptual research use is a particularly salient construct for system reform as it describes the process of shifting awareness and the consideration of new frameworks for action. A tool designed to measure the conceptual use of behavioral health research would advance the field's ability to develop effective models of research evidence dissemination, including collaborative planning models to support the use of behavioral health research in reforms of the criminal-legal system.

METHODS

The ARC Study is a longitudinal, cohort and measurement validation study. It will proceed in two phases. The first phase will focus on measure development using established methods of construct validity (theoretical review, Delphi methods for expert review, cognitive interviewing). The second phase will involve gathering responses from the developed survey to examine scale psychometrics using Rasch analyses, change sensitivity analyses, and associations between research use exposure and conceptual research use among juvenile court leaders. We will recruit juvenile court leaders (judges, administrators, managers, supervisors) from 80 juvenile court jurisdictions with an anticipated sample size of n = 520 respondents.

DISCUSSION

The study will introduce a new measurement tool for the field that will advance implementation science methods for the study of behavioral health research evidence use in complex policy and decision-making interventions. To date, there are few validated survey measures of conceptual research use and no measures that are validated for measuring change in conceptual frameworks over time among agency leaders. While the study is most directly related to leaders in the youth criminal-legal system, the findings are expected to be informative for research focused on leadership and decision-making in diverse fields.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Public opinion about evidence-informed health policy development in U.S. Congress

Promoting evidence-informed health policymaking is a priority of the Society of Behavioral Medicine and other professional societies. However, politics often impede the translation of research into policy. Public opinion is an important feature of political context that influences policymakers' behaviors, but prior research has not examined public opinion about evidence-informed health policy development. This exploratory study sought to characterize public opinion about the influence that evidence should, and does, have on health policy development in U.S. Congress relative to other factors and examine differences by political party affiliation. A public opinion survey was conducted in 2018 using the SSRS Probability Panel (N = 532). Respondents separately rated the extent to which six factors (e.g., evidence, budget impact, industry interests) "should have" and "currently have" influence on U.S. congresspersons' health policy decisions. Evidence (59%) was the most frequently identified factor that should have "a lot of influence" on health policy development, but only 11% of respondents thought that evidence currently has "a lot of influence" (p < .001). Opinions about evidence did not vary significantly by political party. The interests of insurance and pharmaceutical companies were identified as factors that should have the least influence on policy development, but were perceived as having the most influence (p < .001). There is strong bipartisan public support for evidence to have much more influence on health policy development in U.S. Congress. Efforts that aim to improve evidence-informed health policymaking should consider harnessing the power of public opinion to change elected policymakers' behaviors.

Powerful numbers: Exemplary quantitative studies of science that had policy impact

Much scientometric research aims to be relevant to policy, but such research only rarely has a notable policy impact. In this paper, we examine four exemplary cases of policy impact from quantitative studies of science. The cases are analyzed in light of lessons learned about the use of evidence in policy making in health services, which provides very thorough explorations of the problems inherent in policy use of academic research. The analysis highlights key dimensions of the examples, which offer lessons for those aspiring to influence policy with quantitative studies of science.

Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches

Background

Embedded approaches to implementation research (IR), whereby health system decision-makers participate actively in the research process, are gaining traction as effective approaches to optimise the delivery of health programmes and policies. However, the evidence base on the processes and effectiveness of such collaborative research remains inchoate. Standardised approaches to evaluate these initiatives are needed to identify core elements of ‘embeddedness’, unveil the underlying pathways of change, and assess contribution to evidence uptake in decision-making and overall outcomes of effect. The framework presented in this paper responds to this need, designed to guide the systematic evaluation of embedded IR.

Methods

This evaluation framework for embedded IR approaches is based on the experience of a joint initiative by the Pan American Health Organization/Alliance for Health Policy and Systems Research, which has supported 19 IR grants in 10 Latin American and Caribbean countries from 2014 to 2017. The conceptualisation of this framework drew on various sources of information, including empirical evidence and conceptual insights from the literature, interviews with content experts, and a prospective evaluation of the 2016 cohort that included semi-structured key informant interviews, document analysis, and a research team survey to examine key aspects of embedded research.

Results

We developed a widely applicable conceptual framework to guide the evaluation of embedded IR in various contexts. Focused on uncovering how this collaborative research approach influences programme improvement, it outlines expected processes and intermediate outcomes. It also highlights constructs with which to assess ‘embeddedness’ as well as critical contextual factors. The framework is intended to provide a structure by which to systematically examine such embedded research initiatives, proposing three key stages of evidence-informed decision-making – co-production of evidence, engagement with research, and enactment of programme changes.

Conclusion

Rigorous evaluation of embedded IR is needed to build the evidence on its processes and effectiveness in influencing decision-making. The evaluation framework presented here addresses this gap with consideration of the complexity of such efforts. Its applicability to similar initiatives is bolstered by virtue of being founded on real-world experience; its potential to contribute to a nuanced understanding of embedded IR is significant.

Policy Levers to Promote Access to and Utilization of Children's Mental Health Services: A Systematic Review

Policies have potential to help families obtain behavioral healthcare for their children, but little is known about evidence for specific policy approaches. We reviewed evaluations of select policy levers to promote accessibility, affordability, acceptability, availability, or utilization of children's mental and behavioral health services. Twenty articles met inclusion criteria. Location-based policy levers (school-based services and integrated care models) were associated with higher utilization and acceptability, with mixed evidence on accessibility. Studies of insurance-based levers (mental health parity and public insurance) provided some evidence for affordability outcomes. We found no eligible studies of workforce development or telehealth policy levers, or of availability outcomes.

Supporting the use of research evidence in decision-making in crisis zones in low- and middle-income countries: a critical interpretive synthesis

BACKGROUND

Decision-makers in crisis zones are faced with the challenge of having to make health-related decisions under limited time and resource constraints and in light of the many factors that can influence their decisions, of which research evidence is just one. To address a key gap in the research literature about how best to support the use of research evidence in such situations, we conducted a critical interpretive synthesis approach to develop a conceptual framework that outlines the strategies that leverage the facilitators and address the barriers to evidence use in crisis zones.

METHODS

We systematically reviewed both empirical and non-empirical literature and used an interpretive analytic approach to synthesise the results and develop the conceptual framework. We used a 'compass' question to create a detailed search strategy and conducted electronic searches in CINAHL, EMBASE, MEDLINE, SSCI and Web of Science. A second reviewer was assigned to a representative sample of articles. We purposively sampled additional papers to fill in conceptual gaps.

RESULTS

We identified 21 eligible papers to be analysed and purposively sampled an additional 6 to fill conceptual gaps. The synthesis resulted in a conceptual framework that focuses on evidence use in crisis zones examined through the lens of four systems - political, health, international humanitarian aid and health research. Within each of the four systems, the framework identifies the most actionable strategies that leverage the facilitators and address the barriers to evidence use.

CONCLUSIONS

This study presents a new conceptual framework that outlines strategies that leverage the facilitators and address the barriers to evidence use in crisis zones within different systems. This study expands on the literature pertaining to evidence-informed decision-making.

Organisational factors that facilitate research use in public health policy-making: a scoping review

BACKGROUND

Although important syntheses and theoretical works exist in relation to understanding the organisational factors that facilitate research use, these contributions differ in their scope and object of study as well as their theoretical underpinnings. Therefore, from an exploratory angle, it may be useful to map out the current literature on organisational factors of research use in public health policy-making when revisiting existing theories and frameworks to gain further theoretical insights.

METHODS

Herein, a scoping review technique and thematic content analysis were used to bring together findings from both synthesised and empirical studies of different types to map out the organisational factors that facilitate research use in public health policy-making.

RESULTS

A total of 14 reviews and 40 empirical studies were included in the analysis. These were thematically coded and the intra-organisational factors reported as enabling research use were examined. Five main categories of organisational factors that advance research use in policy organisations - (1) individual factors, (2) the management of research integration, (3) organisational systems and infrastructures of research use, (4) institutional structures and rules for policy-making, and (5) organisational characteristics - were derived as well as 18 subcategories and a total of 64 specific factors, where 27 factors were well supported by research.

CONCLUSIONS

Using a scoping review methodology, the intra-organisational factors influencing research use in policy-making (including individual factors) were systematically mapped and the theories applied in this area of research were assessed. The review findings confirm the importance of an intra-organisational perspective when exploring research use, showing that many organisational factors are critical facilitators of research use but also that many factors and mechanisms are understudied. The synthesis shows a lack of studies on politicians and the need for more theoretically founded research. Despite increased efforts to update the existing evidential and theoretical basis of research use, we still need frameworks that combine different approaches and theories to help us grasp the complex organisational mechanisms that facilitate research use in policy settings.

Use of research evidence in state health policymaking: Menu labeling policy in California

Addressing the translational gap between research evidence and state health policy requires an understanding of the current use of research evidence in the state policymaking process. In this study, we explore the use of research evidence to inform the legislative debate about restaurant nutrition labeling policy in California. In 2008, California was the first state to enact a mandatory menu calorie labeling policy in the U.S. Using a qualitative approach, we examine data sources and types of evidence used in legislative documents (n = 87) related to six menu labeling bills introduced in California's state legislature between 2003 and 2008. Federal- and state-level government agency reports were the most frequently cited sources of technical knowledge. Advocacy coalition members who were active participants involved in the policy debate were also cited as experts. Five of the six bills included evidence in related legislative documents. While documents included considerable evidence on the magnitude and severity of the obesity problem to justify policy enactment, there were a limited number of statements referring to policy effectiveness and only one statement identified attesting to implementation context and acceptability. Reference to evidence on related policy suggests policy precedence may also play an important role in policy decision making. There is a need to improve the dissemination of obesity policy effectiveness and implementation studies in a politically time sensitive manner to influence state health policy debates. Strategies are discussed to effectively integrate the use of research evidence in the state health policymaking process.

The governmental health policy-development process for Syrian refugees: an embedded qualitative case studies in Lebanon and Ontario

BACKGROUND

The unprecedented amount of resources dedicated to humanitarian aid has led many stakeholders to demand the use of reliable evidence in humanitarian aid decisions to ensure that desired impacts are achieved at acceptable costs. However, little is known about the factors that influence the use of research evidence in the policy development in humanitarian crises. We examined how research evidence was used to inform two humanitarian policies made in response to the Syrian refugee crisis.

METHODS

We identified two policies as rich potential case studies to examine the use of evidence in humanitarian aid policy decision-making: Lebanon's 2016 Health Response Strategy and Ontario's 2016 Phase 2: Health System Action Plan, Syrian Refugees. To study each, we used an embedded qualitative case study methodology and recruited senior decision-makers, policy advisors, and healthcare providers who were involved with the development of each policy. We reviewed publicly available documents and media articles that spoke to the factors that influence the process. We used the analytic technique of explanation building to understand the factors that influence the use of research evidence in the policy-development process in crisis zones.

RESULTS

We interviewed eight informants working in government and six in international agencies in Lebanon, and two informants working in healthcare provider organizations and two in non-governmental organizations in Ontario, for a total of 18 key informants. Based on our interviews and documentary analysis, we identified that there was limited use of research evidence and that four broad categories of factors helped to explain the policy-development process for Syrian refugees - development of health policies without significant chance for derailment from other government bodies (Lebanon) or opposition parties (Ontario) (i.e., facing no veto points), government's engagement with key societal actors to inform the policy-development process, the values underpinning the process, and external factors significantly influencing the policy-development process.

CONCLUSIONS

This study suggests that use of research evidence in the policy-development process for Syrian refugees was subordinate to key political factors, resulting in limited influence of research evidence in the development of both the Lebanese and Ontarian policy.

An inductive exploration of the implementation knowledge of research funders

Background

Healthcare research funders may undertake various roles to facilitate implementation of research findings. Their ability to enact such roles depends on several factors, knowledge of implementation being one essential requirement. However, previous studies do not assess the type or level of knowledge about implementation that research funders possess. This paper therefore presents findings from a qualitative, inductive study of the implementation knowledge of research funders. Three aspects of this knowledge are explored, namely how research funders define implementation, their level of self-assessed implementation knowledge and the factors influencing their self-assessment of implementation knowledge.

Methods

Research funders (n = 18) were purposefully selected from a sample of research funding organisations in Sweden (n = 10). In-depth semi-structured interviews were conducted, recorded and transcribed verbatim. An inductive method using a systematic coding procedure was employed to derive the findings.

Results

The research funders defined implementation as either an outcome or a process, with the majority believing that implementation of healthcare research results demands a process, although its complexity varied in the research funders’ view. They perceived their own level of implementation knowledge as either limited or substantial, with a majority regarding it as limited. Clinical research experience, clinical experience and task relevance were singled out as the clearest factors affecting the self-assessment of their own implementation knowledge.

Conclusions

This study, the first to focus on implementation knowledge of research funders, demonstrates that they are a category of policy-makers who may possess knowledge, based on their previous professional experience, that is comparable to some important findings from implementation research. Consequently, the findings not only pinpoint the relevance of professional experience, but also reveal a lack of awareness and knowledge of the results of implementation research among research funders in charge of healthcare research.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0472-8) contains supplementary material, which is available to authorized users.

Considering health and health disparities during state policy formulation: examining Washington state Health Impact Reviews

BACKGROUND

As part of efforts to expand Health in All Policies (HiAP) in Washington State in the U.S., the Washington State Board of Health (BOH) received statutory authority in 2006 to conduct Health Impact Reviews (HIRs). HIRs analyze the potential impacts of proposed legislation and budget decisions on health and health disparities. Public health professionals who are aware of HIRs are interested in adopting a similar process in their states; however, there is limited information about HIRs, how they are perceived, and how they could advance HiAP.

METHODS

This research involved a descriptive analysis of a sample of HIRs and semi-structured interviews with a purposive sample of 17 key informants. For the descriptive analysis, all HIRs requested or completed between January 1, 2007 and April 1, 2016 that had a request form submitted by a legislator or the governor that was available in the BOH's online database were reviewed. Information was collected on several variables including the bill number and title, sponsor and political affiliation, and the sector to which the bill or budgetary proposal pertained. A purposeful sample of legislators, staff, advocates, and lobbyists who were involved with HIRs during the study period were invited to participate in semi-structured interviews. Topic coding was used to identify key themes from the qualitative data.

RESULTS

During the study period, 20 legislators requested 36 HIRs; 32 HIRs were completed. HIRs were requested for several bill topics, including education (11/36) and labor and employment (9/36). Legislators who requested HIRs felt they provided valuable data on health and health disparities for proposed bills. Individuals who were less supportive of HIRs perceived them as an advocacy or political tool. The main barrier to widespread use of HIRs in Washington was a lack of awareness among legislators.

CONCLUSIONS

HIRs are one strategy to advance HiAP for state policy decisions. HIRs are a potentially effective tool for highlighting how legislative proposals and budgets positively and negatively impact health and health disparities. Future efforts should promote awareness and highlight shared benefits of HIRs among legislators and their staff, as well as their scientific integrity, methodological rigor, and objectivity.

A description of a tailored knowledge translation intervention delivered by knowledge brokers within public health departments in Canada

Background

While there is an expectation to demonstrate evidence-informed public health there is an ongoing need for capacity development. The purpose of this paper is to provide a description of a tailored knowledge translation intervention implemented by knowledge brokers (KBs), and reflections on the factors that facilitated or hindered its implementation.

Methods

The 22-month knowledge translation intervention, implemented by two KBs, sought to facilitate evidence-informed public health decision-making. Data on outcomes were collected using a knowledge, skills and behavioural assessment survey. In addition, the KBs maintained reflective journals noting which activities appeared successful or not, as well as factors related to the individual or the organisation that facilitated or hindered evidence-informed decision-making.

Results

Tailoring of the knowledge translation intervention to address the needs, preferences and structure of each organisation resulted in three unique interventions being implemented. A consistent finding across organisations was that each site needed to determine where evidence-informed decision-making ‘fit’ within pre-existing organisational processes. Components of the intervention consistent across the three organisations included one-to-one mentoring of teams through rapid evidence reviews, large group workshops and regular meetings with senior management. Components that varied included the frequency of the KB being physically onsite, the amount of time staff spent with the KB and proportion of time spent one-to-one with a KB versus in workshops. Key facilitating factors for implementation included strong leadership, influential power of champions, supportive infrastructure, committed resources and staff enthusiasm.

Conclusions

The results of this study illustrate the importance of working collaboratively with organisations to tailor knowledge translation interventions to best meet unique needs, preferences, organisational structures and contexts. Organisational factors such as leadership, champions and supportive infrastructure play a key role in determining the impact of the knowledge translation interventions. Future studies should explore how these factors can be fostered and/or developed within organisations. While KBs implemented the knowledge translation intervention in this study, more research is needed to understand the impact of all change agent roles including KBs, as well as how these roles can be maintained in the long-term if proven effective.

A roadmap for strengthening evidence-informed health policy-making in Iran: protocol for a research programme

BACKGROUND

Many initiatives have been taken in the Islamic Republic of Iran to promote evidence-informed health policy-making (EIHP). However, these initiatives are not systematic. Since the implementation of EIHP is not consistent and the interventions in this regard are complex, a comprehensive plan could be a useful tool for employing initiatives to achieve and promote EIHP. Hence, this study aims to develop a roadmap for strengthening EIHP over a 3-year period in Iran.

METHODS

Nine projects will be conducted to define the roadmap for strengthening EIHP. These projects include two reviews and a stakeholder analysis to identify the factors that facilitate or hinder achieving EIHP. The next study will be a qualitative study to prioritise the challenges and outline the main causes. The following steps will be a review of reviews to extract global experiences on interventions used for strengthening EIHP and two qualitative studies to examine the adoption of these interventions and develop an operational plan for strengthening EIHP in Iran. The research will be completed through conducting two qualitative-quantitative studies to design a tool for measuring EIHP and assessing EIHP in Iran at baseline.

DISCUSSION

This national EIHP roadmap will surely be able to identify the gaps and bumps that might exist in the implementation plan for establishing EIHP and eliminate them as needed in the future. This roadmap can be a step in moving towards transparency and accountability in the health system and as thus towards good governance and improvement of the health system's performance. Although the plan can be a good model for developing countries and may promote the use of evidence in health policy-making, we should assume that there are some critical contextual factors that could potentially hinder the complete and successful implementation of EIHP. Thus, to enhance EIHP in these countries with a policy-making context that does not fully support the use of evidence, it is crucial to think about not only those interventions that directly address the EIHP barriers, but also some long-term strategies to make required changes in the context, both beyond and within the health system.

How was research engaged with and used in the development of 131 policy documents? Findings and measurement implications from a mixed methods study

Background

Much has been written about the use of evidence in policy; however, there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators. This paper aims to (1) describe the characteristics of 131 policy documents, (2) describe the ways in which research was engaged with (e.g. was searched for, appraised or generated) and used (e.g. to clarify understanding, persuade others or inform a policy) in the development of these policy documents, and (3) identify the most commonly reported barriers and facilitators and describe their association with research engagement and use.

Methods

Six health policy and program development agencies based in Sydney, Australia, contributed four recently finalised policy documents for consideration over six measurement periods. Structured, qualitative interviews were conducted with the policymakers most heavily involved in developing each of the 131 policy documents. Interviews covered whether and how research was engaged with and used in the development of the policy product and any barriers or facilitators related to this. Interviews were scored using the empirically validated SAGE tool and thematically analysed. Descriptive statistics were calculated for all key variables and comparisons made between agencies. Multiple regression analyses were used to estimate the impact of specific barriers and facilitators on research engagement and use.

Results

Our data shows large variations between policy agencies in the types of policy documents produced and the characteristics of these documents. Nevertheless, research engagement and use was generally moderate across agencies. A number of barriers and facilitators to research use were identified. No barriers were significantly associated with any aspects of research engagement or use. Access to consultants and relationships with researchers were both associated with increased research engagement but not use. Thus, access to consultants and relationships with researchers may increase the extent and quality of the evidence considered in policy development.

Conclusions

Our findings suggest that those wishing to develop interventions and programs designed to improve the use of evidence in policy agencies might usefully target increasing access to consultants and relationships with researchers in order to increase the extent and quality of the research considered, but that a greater consideration of context might be required to develop strategies to increase evidence use.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0886-2) contains supplementary material, which is available to authorized users.

Improving the use of evidence in public health policy development, enactment and implementation: a multiple-case study

Many of the policies shown to benefit the public's health in research studies are never widely implemented. We conducted a qualitative and exploratory multiple-case study of three U.S. academic research-policy translation initiatives that resulted in the development, enactment and/or implementation of evidence-based policies to address public health issues: gun policy (Case 1); opioid policy (Case 2) and drug control policy (Case 3). We conducted semi-structured qualitative interviews with 25 key stakeholders involved in the three cases and analysed transcripts using a hybrid a priori and data-driven approach. Across cases, participants identified four key processes that should be included in research-policy translation initiatives: stakeholder engagement, consensus-building, long-term coalition engagement and use of effective knowledge brokers. Participants perceived differences in how the structure of and activities within each initiative facilitated these processes. For example, participants perceived the gun policy initiative, which included large-scale policy dissemination strategies such as state forums where research experts interacted with state policymakers and advocates, as more effective at long-term coalition engagement than the other two initiatives. Study results suggest concrete and actionable academic research-policy strategies that have the potential for widespread adoption by universities or other organizations.

Using narratives to impact health policy-making: a systematic review

BACKGROUND

There is increased interest in using narratives or storytelling to influence health policies. We aimed to systematically review the evidence on the use of narratives to impact the health policy-making process.

METHODS

Eligible study designs included randomised studies, non-randomised studies, process evaluation studies, economic studies, qualitative studies, stakeholder analyses, policy analyses, and case studies. The MEDLINE, PsycINFO, Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), WHO Global Health Library, Communication and Mass Media Complete, and Google Scholar databases were searched. We followed standard systematic review methodology for study selection, data abstraction and risk of bias assessment. We synthesised the findings narratively and presented the results stratified according to the following stages of the policy cycle: (1) agenda-setting, (2) policy formulation, (3) policy adoption, (4) policy implementation and (5) policy evaluation. Additionally, we presented the knowledge gaps relevant to using narrative to impact health policy-making.

RESULTS

Eighteen studies met the eligibility criteria, and included case studies (n = 15), participatory action research (n = 1), documentary analysis (n = 1) and biographical method (n = 1). The majority were of very low methodological quality. In addition, none of the studies formally evaluated the effectiveness of the narrative-based interventions. Findings suggest that narratives may have a positive influence when used as inspiration and empowerment tools to stimulate policy inquiries, as educational and awareness tools to initiate policy discussions and gain public support, and as advocacy and lobbying tools to formulate, adopt or implement policy. There is also evidence of undesirable effects of using narratives. In one case study, narrative use led to widespread insurance reimbursement of a therapy for breast cancer that was later proven to be ineffective. Another case study described how the use of narrative inappropriately exaggerated the perceived risk of a procedure, which led to limiting its use and preventing a large number of patients from its benefits. A third case study described how optimistic 'cure' or 'hope' stories of children with cancer were selectively used to raise money for cancer research that ignored the negative realities. The majority of included studies did not provide information on the definition or content of narratives, the theoretical framework underlying the narrative intervention or the possible predictors of the success of narrative interventions.

CONCLUSION

The existing evidence base precludes any robust inferences about the impact of narrative interventions on health policy-making. We discuss the implications of the findings for research and policy.

TRIAL REGISTRATION

The review protocol is registered in PROSPERO International prospective register of systematic reviews (ID =  CRD42018085011 ).

Increasing the capacity of policy agencies to use research findings: a stepped-wedge trial

BACKGROUND

This paper describes the trial of a novel intervention, Supporting Policy In health with evidence from Research: an Intervention Trial (SPIRIT). It examines (1) the feasibility of delivering this kind of programme in practice; (2) its acceptability to participants; (3) the impact of the programme on the capacity of policy agencies to engage with research; and (4) the engagement with and use of research by policy agencies.

METHODS

SPIRIT was a multifaceted, highly tailored, stepped-wedge, cluster-randomised, trial involving six health policy agencies in Sydney, Australia. Agencies were randomly allocated to one of three start dates to receive the 1-year intervention programme. SPIRIT included audit, feedback and goal setting; a leadership programme; staff training; the opportunity to test systems to facilitate research use in policies; and exchange with researchers. Outcome measures were collected at each agency every 6 months for 30 months.

RESULTS

Participation in SPIRIT was associated with significant increases in research use capacity at staff and agency levels. Staff reported increased confidence in research use skills, and agency leaders reported more extensive systems and structures in place to support research use. Self-report data suggested there was also an increase in tactical research use among agency staff. Given the relatively small numbers of participating agencies and the complexity of their contexts, findings suggest it is possible to effect change in the way policy agencies approach the use of research. This is supported by the responses on the other trial measures; while these were not statistically significant, on 18 of the 20 different measures used, the changes observed were consistent with the hypothesised intervention effect (that is, positive impacts).

CONCLUSIONS

As an early test of an innovative approach, SPIRIT has demonstrated that it is possible to increase research engagement and use in policy agencies. While more work is needed to establish the replicability and generalisability of these findings, this trial suggests that building staff skills and organisational structures may be effective in increasing evidence use.

Impact of an organization-wide knowledge translation strategy to support evidence-informed public health decision making

Background

The public health sector is moving toward adopting evidence-informed decision making into practice, but effort is still required to effectively develop capacity and promote contextual factors that advance and sustain it. This paper describes the impact of an organization-wide knowledge translation intervention delivered by knowledge brokers on evidence-informed decision making knowledge, skills and behaviour.

Methods

A case study design was implemented with the intervention and data collection tailored to the unique needs of each case (health department). A knowledge broker provided training workshops and mentored small groups through a seven step process of evidence-informed decision making. The intervention was delivered over 22 months; data related to evidence-informed decision making knowledge, skills and behaviour were collected at baseline and follow-up. Mixed effects regression models were developed to assess the impact of involvement in the intervention on the evidence-informed decision making outcomes.

Results

Data from a total of 606 health department staff were collected during baseline: 207 (33%) staff from Case A, 304 (28%) from Case B, and 95 (47%) from Case C. There were a total of 804 participants at follow-up: 258 (42%) from Case A, 391 from Case B (37%), and 155 (50%) from Case C. Statistically significant increases in knowledge and skills were observed overall, and in all three health departments. An increase in evidence-informed decision making behaviour was observed among those intensively involved in the intervention from all cases (statistically significant in Case A). The organizational characteristics of strategic priority, leadership, readiness, and choice of staff emerged as important factors in the change process.

Conclusions

Knowledge brokering is a promising organizational knowledge translation intervention to support evidence-informed decision making. The intervention appeared to have the greatest impact on those who became actively engaged with the knowledge broker in the intervention. Active participation in face-to-face training activities with a knowledge broker, focused specifically on evidence-informed decision making skill development, led to the greatest impact on associated behaviours, knowledge, and skills. Several organizational factors emerged as integral to success of the knowledge translation intervention.

Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries

BACKGROUND

Systematic reviews are an important source of evidence for public health decision-making, but length and technical jargon tend to hinder their use. In non-English speaking countries, inaccessibility of information in the native language often represents an additional barrier. In line with our vision to strengthen evidence-based public health in the German-speaking world, we developed a German language summary format for systematic reviews of public health interventions and undertook user-testing with public health decision-makers in Germany, Austria and Switzerland.

METHODS

We used several guiding principles and core elements identified from the literature to produce a prototype summary format and applied it to a Cochrane review on the impacts of changing portion and package sizes on selection and consumption of food, alcohol and tobacco. Following a pre-test in each of the three countries, we carried out 18 user tests with public health decision-makers in Germany, Austria and Switzerland using the 'think-aloud' method. We analysed participants' comments according to the facets credibility, usability, understandability, usefulness, desirability, findability, identification and accessibility. We also identified elements that hindered the facile and satisfying use of the summary format, and revised it based on participants' feedback.

RESULTS

The summary format was well-received; participants particularly appreciated receiving information in their own language. They generally found the summary format useful and a credible source of information, but also signalled several barriers to a positive user experience such as an information-dense structure and difficulties with understanding statistical terms. Many of the identified challenges were addressed through modifications of the summary format, in particular by allowing for flexible length, placing more emphasis on key messages and relevance for public health practice, expanding the interpretation aid for statistical findings, providing a glossary of technical terms, and only including graphical GRADE ratings. Some barriers to uptake, notably the participants' wish for actionable recommendations and contextual information, could not be addressed.

CONCLUSIONS

Participants welcomed the initiative, but user tests also revealed their problems with understanding and interpreting the findings summarised in our prototype format. The revised summary format will be used to communicate the results of Cochrane reviews of public health interventions.

Learning in the zone: toward workforce development of evidence-based public policy communication

Background

Evidence-based policy communication (EBPC) is an important, emerging focus in public health research. However, we have yet to understand public health workforce ability to develop and/or use it. The study objective was to characterize capacity to develop and use EBPC and identify cooperative learning and development opportunities using the case of Human papillomavirus (HPV).

Methods

Vygotsky’s Zone of Proximal Development (ZPD) informed guided interviews with 27 advocates in Indiana from government, industry, research, state associations and individuals. Participants focused on HPV, cancer, women’s health, school health and minority health.

Results

Capacity to develop and use EBPC was reported to develop through cooperative learning opportunities on the job or in advocacy focused coalitions. Coalition learning appeared to translate across health topics. Notably, policy experience did not assure understanding or use of EBPC.

Conclusions

The ZPD framework can inform workforce EBPC interventions by focusing on actual development, potential development and factors for learning and development in the ZPD. Future studies should further clarify and evaluate emerging indicators in additional public health policy areas with a larger sample.

What can we learn from interventions that aim to increase policy-makers' capacity to use research? A realist scoping review

BACKGROUND

Health policy-making can benefit from more effective use of research. In many policy settings there is scope to increase capacity for using research individually and organisationally, but little is known about what strategies work best in which circumstances. This review addresses the question: What causal mechanisms can best explain the observed outcomes of interventions that aim to increase policy-makers' capacity to use research in their work?

METHODS

Articles were identified from three available reviews and two databases (PAIS and WoS; 1999-2016). Using a realist approach, articles were reviewed for information about contexts, outcomes (including process effects) and possible causal mechanisms. Strategy + Context + Mechanism = Outcomes (SCMO) configurations were developed, drawing on theory and findings from other studies to develop tentative hypotheses that might be applicable across a range of intervention sites.

RESULTS

We found 22 studies that spanned 18 countries. There were two dominant design strategies (needs-based tailoring and multi-component design) and 18 intervention strategies targeting four domains of capacity, namely access to research, skills improvement, systems improvement and interaction. Many potential mechanisms were identified as well as some enduring contextual characteristics that all interventions should consider. The evidence was variable, but the SCMO analysis suggested that tailored interactive workshops supported by goal-focused mentoring, and genuine collaboration, seem particularly promising. Systems supports and platforms for cross-sector collaboration are likely to play crucial roles. Gaps in the literature are discussed.

CONCLUSION

This exploratory review tentatively posits causal mechanisms that might explain how intervention strategies work in different contexts to build capacity for using research in policy-making.

Do policy-makers find commissioned rapid reviews useful?

Background

Rapid reviews are increasingly used by policy agencies to access relevant research in short timeframes. Despite the growing number of programmes, little is known about how rapid reviews are used by health policy agencies. This study examined whether and how rapid reviews commissioned using a knowledge brokering programme were used by Australian policy-makers.

Methods

This study used interview data to examine the use of 139 rapid reviews by health policy agencies that were commissioned between 2006 and 2015. Transcripts were coded to identify how rapid reviews were used, the type of policy processes in which they were used, what evidence of use was provided and what reasons were given when rapid reviews were not used. Fisher’s exact test was used to assess variation between types of agencies.

Results

Overall, 89% of commissioned rapid reviews were used by the commissioning agencies and 338 separate instances of use were identified, namely, on average, three uses per review. Policy-makers used reviews primarily to determine the details of a policy or programme, identify priorities for future action or investment, negotiate interjurisdictional decisions, evaluate alternative solutions for a policy problem, and communicate information to stakeholders. Some variation in use was observed across agencies. Reasons for non-use were related to changes in organisational structures, resources or key personnel in the commissioning agencies, or changes in the broader political environment.

Conclusions

This study found that almost all rapid reviews had been used by the agencies who commissioned them, primarily in policy and programme development, agenda-setting, and to communicate information to stakeholders. Reviews were used mostly in instrumental and conceptual ways and there was little evidence of symbolic use. Variations in use were identified across agencies. The findings suggest that commissioned rapid reviews are an effective means of providing timely relevant research for use in policy processes and that review findings may be applied in a variety of ways.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0293-1) contains supplementary material, which is available to authorized users.

Institutional capacity to generate and use evidence in LMICs: current state and opportunities for HPSR

BACKGROUND

Evidence-informed decision-making for health is far from the norm, particularly in many low- and middle-income countries (LMICs). Health policy and systems research (HPSR) has an important role in providing the context-sensitive and -relevant evidence that is needed. However, there remain significant challenges both on the supply side, in terms of capacity for generation of policy-relevant knowledge such as HPSR, and on the demand side in terms of the demand for and use of evidence for policy decisions. This paper brings together elements from both sides to analyse institutional capacity for the generation of HPSR and the use of evidence (including HPSR) more broadly in LMICs.

METHODS

The paper uses literature review methods and two survey instruments (directed at research institutions and Ministries of Health, respectively) to explore the types of institutional support required to enhance the generation and use of evidence.

RESULTS

Findings from the survey of research institutions identified the absence of core funding, the lack of definitional clarity and academic incentive structures for HPSR as significant constraints. On the other hand, the survey of Ministries of Health identified a lack of locally relevant evidence, poor presentation of research findings and low institutional prioritisation of evidence use as significant constraints to evidence uptake. In contrast, improved communication between researchers and decision-makers and increased availability of relevant evidence were identified as facilitators of evidence uptake.

CONCLUSION

The findings make a case for institutional arrangements in research that provide support for career development, collaboration and cross-learning for researchers, as well as the setting up of institutional arrangements and processes to incentivise the use of evidence among Ministries of Health and other decision-making institutions. The paper ends with a series of recommendations to build institutional capacity in HPSR through engaging multiple stakeholders in identifying and maintaining incentive structures, improving research (including HPSR) training, and developing stronger tools for synthesising non-traditional forms of local, policy-relevant evidence such as grey literature. Addressing challenges on both the supply and demand side can build institutional capacity in the research and policy worlds and support the enhanced uptake of high quality evidence in policy decisions.

Uses of Population Health Rankings in Local Policy Contexts: A Multisite Case Study

Population health rankings are a common strategy to spur evidence-informed health policy making, but little is known about their uses or impacts. The study aims were to (1) understand how and why the County Health Rankings (CH-Rankings) are used in local policy contexts, (2) identify factors that influence CH-Rankings utilization, and (3) explore potentially negative impacts of the CH-Rankings. Forty-four interviews were conducted with health organization officials and public policy makers in 15 purposively selected counties. The CH-Rankings were used instrumentally to inform internal planning decisions, conceptually to educate the public and policy makers about determinants of population health, and politically to advance organizational agendas. Factors related to organizational capacity, county political ideology, and county rank influenced if, how, and why the CH-Rankings were used. The CH-Rankings sometimes had the negative impacts of promoting potentially ineffective interventions in politically conservative counties and prompting negative media coverage in some counties with poor rank.

Mutual Distrust: Perspectives From Researchers and Policy Makers on the Research to Policy Gap in 2013 and Recommendations for the Future

The production of health policy-relevant research is necessary, but not sufficient, to promote its utilization in policy. Our objective was to understand the perspectives of United States’ state-level policy makers and health researchers on the barriers and facilitators to the translation of health evidence into the policy process, with a particular focus on issues related to relationship building. We conducted interviews with 215 US health services and health policy researchers and 40 state-level staffers and legislators. Researchers and policy makers faced the same major barrier to research translation: lack of dedicated time to do so. Some policy makers questioned the credibility of research, and researchers questioned policy makers’ authentic desire to use evidence in decision making. For some study participants, a mutual mistrust of the other group challenges stronger relationship formation. Interventions are needed to help both groups understand a broader role that research plays in policy making and to increase personal contact, and ultimately trusted relationships, across various actors in the policy process.

Improving maternal and child health policymaking processes in Nigeria: an assessment of policymakers' needs, barriers and facilitators of evidence-informed policymaking

BACKGROUND

In Nigeria, interest in the evidence-to-policy process is gaining momentum among policymakers involved in maternal, newborn and child health (MNCH). However, numerous gaps exist among policymakers on use of research evidence in policymaking. The objective of this study was to assess the perception of MNCH policymakers regarding their needs and the barriers and facilitators to use of research evidence in policymaking in Nigeria.

METHODS

The study design was a cross-sectional assessment of perceptions undertaken during a national MNCH stakeholders' engagement event convened in Abuja, Nigeria. A questionnaire designed to assess participants' perceptions was administered in person. Group consultations were also held, which centred on policymakers' evidence-to-policy needs to enhance the use of evidence in policymaking.

RESULTS

A total of 40 participants completed the questionnaire and participated in the group consultations. According to the respondents, the main barriers to evidence use in MNCH policymaking include inadequate capacity of organisations to conduct policy-relevant research; inadequate budgetary allocation for policy-relevant research; policymakers' indifference to research evidence; poor dissemination of research evidence to policymakers; and lack of interaction fora between researchers and policymakers. The main facilitators of use of research evidence for policymaking in MNCH, as perceived by the respondents, include capacity building for policymakers on use of research evidence in policy formulation; appropriate dissemination of research findings to relevant stakeholders; involving policymakers in research design and execution; and allowing policymakers' needs to drive research. The main ways identified to promote policymakers' use of evidence for policymaking included improving policymakers' skills in information and communication technology, data use, analysis, communication and advocacy.

CONCLUSION

To improve the use of research evidence in policymaking in Nigeria, there is a need to establish mechanisms that will facilitate the movement from evidence to policy and address the needs identified by policymakers. It is also imperative to improve organisational initiatives that facilitate use of research evidence for policymaking.

Challenges to generating evidence-informed policy and the role of systematic reviews and (perceived) conflicts of interest

BACKGROUND

Multiple efforts to generate evidence-informed policy have attempted to teach policymakers how to understand and apply scientific research findings in their decision-making. These efforts have had limited success, because policymakers generally do not understand scientific methods.

OBJECTIVE

We piloted efforts to teach policy intermediaries - specifically consumer advocacy groups - how to understand and apply health research, anticipating that they might offer such evidence to policymakers in more accessible forms.

DESIGN

Four workshops focusing on research design and methods were conducted with consumer advocacy groups in 2010. We coded and analyzed participant responses regarding their confidence in interpreting research findings and assessments of research credibility, and the extent to which their knowledge about research findings changed after completing the workshops.

RESULTS

Our findings suggest that although participants expressed strong interest in understanding scientific research, their ability to develop confidence about scientific research methods was limited. However, like policymakers, consumer advocacy group members intuited that financial conflicts of interest could compromise scientific findings, although they initially underestimated their effects on research results. After training, consumer advocates also saw the value of using systematic reviews rather than individual studies.

DISCUSSION AND CONCLUSIONS

Our findings suggest that although advocates may not feel confident in their understanding of scientific research, they found it easier to understand the value of systematic reviews and the risks of conflicts of interest than other statistical concepts and terminology. Focusing on making these types of information available may offer a useful way for policymakers and consumer advocates to interpret the validity of policy-relevant scientific research.

Realist review of policy intervention studies aimed at reducing exposures to environmental hazards in the United States

Background

Exposure to pollution is a significant risk to human health. However few studies have attempted to identify the types of policy interventions that can reduce the health risks of pollution exposure in the United States. The study objective was to conduct a realist review of policy interventions conducted or aimed at reducing chemical exposures in humans or the environment where exposure was measured.

Methods

A systematic literature search identified published articles that assessed policy interventions using exposure data. Two coders independently extracted data from the studies, assessing methods, context, details of interventions, outcomes, and risks of bias. Data were analyzed iteratively and manually to identify the most effective and transferrable types of interventions. The reasons for variability in the success of different interventions were explored.

Results

The review found that regulatory interventions that eliminate point sources of pollution appeared to reduce exposure to environmental hazards. Regular monitoring to provide environmental and human exposure data helped assess compliance with the regulatory standards. Educational and economic interventions were less successful.

Conclusions

Although some types of regulatory interventions appear to reduce exposures, our findings are limited by the nature of existing interventions, the weaknesses of the study designs used in the literature, and the lack of details on implementation. Information on contextual factors that influence implementation would assist with future reviews and could help identify effective interventions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-016-3461-7) contains supplementary material, which is available to authorized users.

Framing research for state policymakers who place a priority on cancer

PURPOSE

Despite the potential for reducing the cancer burden via state policy change, few data exist on how best to disseminate research information to influence state legislators' policy choices. We explored: (1) the relative importance of core framing issues (source, presentation, timeliness) among policymakers who prioritize cancer and those who do not prioritize cancer and (2) the predictors of use of research in policymaking.

METHODS

Cross-sectional data were collected from US state policymakers (i.e., legislators elected to state houses or senates) from January through October 2012 (n = 862). One-way analysis of variance was performed to investigate the association of the priority of cancer variable with outcome variables. Multivariate logistic regression models examined predictors of the influence of research information.

RESULTS

Legislators who prioritized cancer tended to rate characteristics that make research information useful higher than those who did not prioritize cancer. Among differences that were statistically significant were three items in the "source" domain (relevance, delivered by someone respected, supports one's own position), one item in the "presentation" domain (telling a story related to constituents) and two items in the "timeliness" domain (high current state priority, feasible when information is received). Participants who prioritized cancer risk factors were 80 % more likely to rate research information as one of their top reasons for choosing an issue on which to work.

CONCLUSIONS

Our results suggest the importance of narrative forms of communication and that research information needs to be relevant to the policymakers' constituents in a brief, concise format.

Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review

BACKGROUND

We completed a scoping review on the barriers and facilitators to use of systematic reviews by health care managers and policy makers, including consideration of format and content, to develop recommendations for systematic review authors and to inform research efforts to develop and test formats for systematic reviews that may optimise their uptake.

METHODS

We used the Arksey and O'Malley approach for our scoping review. Electronic databases (e.g., MEDLINE, EMBASE, PsycInfo) were searched from inception until September 2014. Any study that identified barriers or facilitators (including format and content features) to uptake of systematic reviews by health care managers and policy makers/analysts was eligible for inclusion. Two reviewers independently screened the literature results and abstracted data from the relevant studies. The identified barriers and facilitators were charted using a barriers and facilitators taxonomy for implementing clinical practice guidelines by clinicians.

RESULTS

We identified useful information for authors of systematic reviews to inform their preparation of reviews including providing one-page summaries with key messages, tailored to the relevant audience. Moreover, partnerships between researchers and policy makers/managers to facilitate the conduct and use of systematic reviews should be considered to enhance relevance of reviews and thereby influence uptake.

CONCLUSIONS

Systematic review authors can consider our results when publishing their systematic reviews. These strategies should be rigorously evaluated to determine impact on use of reviews in decision-making.

A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007-2014

BACKGROUND

Policy has a tremendous potential to improve population health when informed by research evidence. Such evidence, however, typically plays a suboptimal role in policymaking processes. The field of policy dissemination and implementation research (policy D&I) exists to address this challenge. The purpose of this study was to: (1) determine the extent to which policy D&I was funded by the National Institutes of Health (NIH), (2) identify trends in NIH-funded policy D&I, and (3) describe characteristics of NIH-funded policy D&I projects.

METHODS

The NIH Research Portfolio Online Reporting Tool was used to identify all projects funded through D&I-focused funding announcements. We screened for policy D&I projects by searching project title, abstract, and term fields for mentions of "policy," "policies," "law," "legal," "legislation," "ordinance," "statute," "regulation," "regulatory," "code," or "rule." A project was classified as policy D&I if it explicitly proposed to conduct research about the content of a policy, the process through which it was developed, or outcomes it produced. A coding guide was iteratively developed, and all projects were independently coded by two researchers. ClinicalTrials.gov and PubMed were used to obtain additional project information and validate coding decisions. Descriptive statistics--stratified by funding mechanism, Institute, and project characteristics--were produced.

RESULTS

Between 2007 and 2014, 146 projects were funded through the D&I funding announcements, 12 (8.2 %) of which were policy D&I. Policy D&I funding totaled $16,177,250, equivalent to 10.5 % of all funding through the D&I funding announcements. The proportion of funding for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2012. Policy D&I projects were primarily focused on policy outcomes (66.7 %), implementation (41.7 %), state-level policies (41.7 %), and policies within the USA (83.3 %). Tobacco (33.3 %) and cancer (25.0 %) control were the primary topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources.

CONCLUSIONS

NIH has made an initial investment in policy D&I research, but the level of support has varied between Institutes. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies.

State legislators' sources and use of information: bridging the gap between research and policy

Research can inform policymakers of public health issues and shape policy decisions, hopefully benefiting public health; thus, improving dissemination of research to policymakers is important for developing effective public health policies that improve health and health equity. However, the utilization of research among policymakers is often not fully realized. This study builds upon current knowledge about what types of information legislators seek when working on health issues and where they go for information. Further, it explores what kinds of information legislators find most helpful and if there are ways researchers could better provide this evidence. Key-informant interviews were conducted with 25 U.S. state legislators holding health committee leadership positions between July and November, 2010. Regarding types of information sought, most legislators discussed their desire for data and statistics when working on health-related issues. When asked about their most trusted sources of information, participants mentioned government sources as well as advocacy, lobby and industry groups. A few mentioned universities and healthcare professionals. Results from this study offer public health researchers and practitioners' insights into the types of information that may be most helpful to policymakers. Insights gathered may improve the dissemination of research and bridge the gap between knowledge users and knowledge producers.

Incorporating research evidence into decision-making processes: researcher and decision-maker perceptions from five low- and middle-income countries

BACKGROUND

The 'Sponsoring National Processes for Evidence-Informed Policy Making in the Health Sector of Developing Countries' program was launched by the Alliance for Health Policy and Systems Research, WHO, in July 2008. The program aimed to catalyse the use of evidence generated through health policy and systems research in policymaking processes through (1) promoting researchers and policy advocates to present their evidence in a manner that is easy for policymakers to understand and use, (2) creating mechanisms to spur the demand for and application of research evidence in policymaking, and (3) increased interaction between researchers, policy advocates, and policymakers. Grants ran for three years and five projects were supported in Argentina, Bangladesh, Cameroon, Nigeria and Zambia. This paper seeks to understand why projects in some settings were perceived by the key stakeholders involved to have made progress towards their goals, whereas others were perceived to have not done so well. Additionally, by comparing experiences across five countries, we seek to illustrate general learnings to inform future evidence-to-policy efforts in low- and middle-income countries.

METHODS

We adopted the theory of knowledge translation developed by Jacobson et al. (J Health Serv Res Policy 8(2):94-9, 2003) as a framing device to reflect on project experiences across the five cases. Using data from the projects' external evaluation reports, which included information from semi-structured interviews and quantitative evaluation surveys of those involved in projects, and supplemented by information from the projects' individual technical reports, we applied the theoretical framework with a partially grounded approach to analyse each of the cases and make comparisons.

RESULTS AND CONCLUSION

There was wide variation across projects in the type of activities carried out as well as their intensity. Based on our findings, we can conclude that projects perceived as having made progress towards their goals were characterized by the coming together of a number of domains identified by the theory. The domains of Jacobson's theoretical framework, initially developed for high-income settings, are of relevance to the low- and middle-income country context, but may need modification to be fully applicable to these settings. Specifically, the relative fragility of institutions and the concomitantly more significant role of individual leaders point to the need to look at leadership as an additional domain influencing the evidence-to-policy process.