New directions in evidence-based policy research: a critical analysis of the literature

Faculty members as academic knowledge brokers in Iran's health sector: a social network analysis study

BACKGROUND

Interaction between researchers and policymakers is an essential factor to facilitate the evidence-informed policymaking. One of the effective ways to establish this relationship and promote evidence-informed policymaking is to employ people or organizations that can play the role of knowledge brokers. This study aims to analyze the communication network and interactions between researchers and policymakers in Iran's health sector and identify key people serving as academic knowledge brokers.

METHODS

This study was a survey research. Using a census approach, we administered a sociometric survey to faculty members in the health field in top ten Iranian medical universities to construct academic-policymaker network using social network analysis method. Network maps were generated using UCINET and NetDraw software. We used Indegree Centrality, Outdegree Centrality, and Betweenness Centrality indicators to determine knowledge brokers in the network.

RESULTS

The drawn network had a total of 188 nodes consisting of 94 university faculty members and 94 policymakers at three national, provincial, and university levels. The network comprised a total of 177 links, with 125 connecting to policymakers and 52 to peers. Of 56 faculty members, we identified four knowledge brokers. Six policymakers were identified as key policymakers in the network, too.

CONCLUSIONS

It seems that the flow of knowledge produced by research in the health field in Iran is not accomplished well from the producers of research evidence to the users of knowledge. Therefore, it seems necessary to consider incentive and support mechanisms to strengthen the interaction between researchers and policymakers in Iran's health sector.

Evidence-informed stakeholder consultations to promote rights-based approaches for children with disabilities

Purpose

To strengthen the translation of evidence to actionable policy, stakeholder engagement is necessary to synthesize, prioritize and contextualize the academic research content into accessible language. In this manuscript we describe a multi-level evidence-based stakeholder consultation process and related outcomes proposed to promote awareness of and foster cross-sectorial collaborations towards human rights-based approaches for children with disabilities.

Methods

Mixed-methods participatory action research done in three steps: (1) A literature review of peer-reviewed evidence on rights-based approaches in childhood disabilities; (2) Consultation with researchers in diverse fields, grassroot organizations, caregivers, and youth with disabilities; (3) A constructive dialogue with decision makers at federal and provincial levels in Canada to discuss consultations results.

Results

Stakeholders value human rights approaches that can have a direct impact on practical aspects of their daily living. Organizations give high importance to adopting rights-based approaches to measure policy outcomes, while parents value service provision and youth emphasize accessibility.

Conclusion

The implementation of rights-based approaches in childhood disabilities can support policy, services, and daily lives of children with disabilities and the ecosystems around them. It can also guide research priorities, and create a common language to foster collaborations across sectors and interested parties.

The effectiveness of knowledge-sharing techniques and approaches in research funded by the National Institute for Health and Care Research (NIHR): a systematic review

BACKGROUND

The National Institute of Health and Care Research (NIHR), funds, enables and delivers world-leading health and social care research to improve people's health and wellbeing. To achieve this aim, effective knowledge sharing (two-way knowledge sharing between researchers and stakeholders to create new knowledge and enable change in policy and practice) is needed. To date, it is not known which knowledge sharing techniques and approaches are used or how effective these are in creating new knowledge that can lead to changes in policy and practice in NIHR funded studies.

METHODS

In this restricted systematic review, electronic databases [MEDLINE, The Health Management Information Consortium (including the Department of Health's Library and Information Services and King's Fund Information and Library Services)] were searched for published NIHR funded studies that described knowledge sharing between researchers and other stakeholders. One researcher performed title and abstract, full paper screening and quality assessment (Critical Appraisal Skills Programme qualitative checklist) with a 20% sample independently screened by a second reviewer. A narrative synthesis was adopted.

RESULTS

In total 9897 records were identified. After screening, 17 studies were included. Five explicit forms of knowledge sharing studies were identified: embedded models, knowledge brokering, stakeholder engagement and involvement of non-researchers in the research or service design process and organisational collaborative partnerships between universities and healthcare organisations. Collectively, the techniques and approaches included five types of stakeholders and worked with them at all stages of the research cycle, except the stage of formation of the research design and preparation of funding application. Seven studies (using four of the approaches) gave examples of new knowledge creation, but only one study (using an embedded model approach) gave an example of a resulting change in practice. The use of a theory, model or framework to explain the knowledge sharing process was identified in six studies.

CONCLUSIONS

Five knowledge sharing techniques and approaches were reported in the included NIHR funded studies, and seven studies identified the creation of new knowledge. However, there was little investigation of the effectiveness of these approaches in influencing change in practice or policy.

Bridging research-policy gaps: An integrated approach

It is often difficult for policymakers to make informed decisions without evidence-based support, resulting in potentially ineffective policies. The purpose of this article is to advocate for collaboration and communication between researchers and policymakers to enhance evidence-based policymaking. The workshop hosted by the International Society of Environmental Epidemiology-Eastern Mediterranean Chapter further explores the challenges of connecting researchers and policymakers. The article highlights the gap between researchers and policymakers, attributed to different visions and objectives, time constraints, and communication issues. To strengthen the research-policy interface, strategies such as enhanced communication skills and early involvement of policymakers in research are suggested. The article proposes an integrated model combining the Collaborative Knowledge Model and the policy entrepreneurship mindset, emphasizing the co-creation of knowledge and evidence-based policy solutions. The use of this model can lead to the development of evidence-based policies that effectively address societal needs.

Various application roles for Campbell systematic reviews: a citation analysis

OBJECTIVES

Systematic reviews (SRs) are becoming essential evidence in the decision-making process within the field of social sciences. This study aimed to investigate how Campbell SRs were cited and explore their specific application roles.

STUDY DESIGN AND SETTING

We included Campbell SRs published between 2016 and 2020 by searching the Wiley online library, and retrieved the articles and documents citing Campbell SRs from the Web of Science and Google Scholar by December 31, 2021. We described the characteristics of the SRs and citations, and formulated a set of application roles by analyzing the sentences or paragraphs where the SRs were cited.

RESULTS

Sixty nine Campbell SRs were published between 2016 and 2020; they were cited in 641 articles or documents a total of 1,289 times. The primary types of articles that cited Campbell SRs were cross-sectional studies (n = 226, 35.3%), SRs (n = 112, 17.5%), randomized controlled trials (n = 77, 12.0%), and policy reports (n = 57, 8.8%). Articles utilizing Campbell SRs were predominantly led by authors from the United States (n = 184, 28.7%), the United Kingdom (n = 98, 15.3%), and Australia (n = 51, 8.0%). We formulated a set of 10 application roles for the Campbell SRs, of which the most frequent were: describing the current status in the field of interest (n = 691, 53.6%), corroboration of the results (n = 140, 10.9%), identifying research gaps (n = 130, 10.1%), and providing methodological references (n = 126, 9.8%); the role of supporting policy recommendations or decisions accounted for 6.0% (n = 77) of the citations. Approximately 12% of Campbell SRs were used to support policy recommendations or decisions.

CONCLUSION

Campbell SRs are widely applied, particularly in scientific research, to describe the current status in the field of interest. Although the current application of Campbell SRs in supporting policy recommendations and decisions may not be predominant, there is a growing recognition of their value in using Campbell SRs to inform decision-making.

Exploring the typology of decision-makers, institutions, and incentives that shape health decisions in Pakistan and insulate decision makers from citizens feedback

Introduction

In developing countries such as Pakistan, program and policies underperform in providing public good as weak institutions lead to decisions that are unresponsive to citizens and are driven by personal motivations of the incumbents. We describe the decision-making processes in the health sector through the framework of "Public Choice" theory which posits how individual motives shape institutional performance and direction.

Methods

We conducted 84 qualitative interviews with five types of stakeholders: politicians, senior and mid-level bureaucrats, donors, public health experts and media personnel spanning 2 periods over a decade. The initial interviews were conducted during 2013-2015 period and a case study on decision-making during the COVID-19 response was added in 2020-2022 period.

Findings

Most new ideas originate from top political leadership, guided by personal agendas or political expediency. Senior bureaucrats implement politicians' agenda and mid-level officials maintain the status quo and follow established "authority." Since officials' performance, promotions, transfers, and the rare dismissals are based on tenure deviations rather than work performance, individuals and institutions are reluctant to take initiative without "consensus" among their colleagues often leading to inaction or delays that obviate initiative and reform. The public sector lacks institutional memory, formal information gathering, and citizen engagement, impacting public goods, health services, and policies. Media and donor personnel are important influencers. However, media mostly report only "hot issues" in health, with short publication and reader attention cycles. Donor personnel are the most likely to follow evidence for decision making, albeit while following their institutional priorities determined centrally. The COVID-19 response is presented as a contrast from usual practices, where evidence was used to guide decisions, as the pandemic was perceived as a national threat by the highest leadership.

Conclusion

Absence of citizen feedback and formal systems for evidence gathering and processing leads to decisions that neglect the needs of those they serve, prioritizing personal or political gains and perpetuating the status quo. However, the COVID-19 pandemic emphasized the importance of evidence-based decision-making and offers valuable lessons for reforming decision-making processes.

Factors that influence evidence-informed meso-level regional primary health care planning: a qualitative examination and conceptual framework

BACKGROUND

Evidence-informed primary health care (PHC) planning in decentralised, meso-level regional organisations has received little research attention. In this paper we examine the factors that influence planning within this environment, and present a conceptual framework.

METHODS

We employed mixed methods: case studies of five Australian Primary Health Networks (PHNs), involving 29 primary interviews and secondary analysis of 38 prior interviews; and analysis of planning documents from all 31 PHNs. The analysis was informed by a WHO framework of evidence-informed policy-making, and institutional theory.

RESULTS

Influential actors included federal and state/territory governments, Local Health Networks, Aboriginal Community Controlled Health Organisations, local councils, public hospitals, community health services, and providers of allied health, mental health and aged care services. The federal government was most influential, constraining PHNs' planning scope, time and funding. Other external factors included: the health service landscape; local socio-demographic and geographic characteristics; (neoliberal) ideology; interests and politics; national policy settings and reforms; and system reorganisation. Internal factors included: organisational structure; culture, values and ideology; various capacity factors; planning processes; transition history; and experience. The additional regional layer of context adds to the complexity of planning.

CONCLUSIONS

Like national health policy-making, meso-level PHC planning occurs in a complex environment, but with additional regional factors and influences. We have developed a conceptual framework of the meso-level PHC planning environment, which can be employed by similar regional organisations to elucidate influential factors, and develop strategies and tools to promote transparent, evidence-informed PHC planning for better health outcomes.

Using evidence in mental health policy agenda-setting in low- and middle-income countries: a conceptual meta-framework from a scoping umbrella review

The purpose of this article is to close the gap in frameworks for the use of evidence in the mental health policy agenda-setting in low- and middle-income countries (LMICs). Agenda-setting is important because mental health remains a culturally sensitive and neglected issue in LMICs. Moreover, effective evidence-informed agenda-setting can help achieve, and sustain, the status of mental health as a policy priority in these low-resource contexts. A scoping 'review of reviews' of evidence-to-policy frameworks was conducted, which followed preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines. Nineteen reviews met the inclusion criteria. A meta-framework was developed from analysis and narrative synthesis of these 19 reviews, which integrates the key elements identified across studies. It comprises the concepts of evidence, actors, process, context and approach, which are linked via the cross-cutting dimensions of beliefs, values and interests; capacity; power and politics; and trust and relationships. Five accompanying questions act as a guide for applying the meta-framework with relevance to mental health agenda-setting in LMICs. This is a novel and integrative meta-framework for mental health policy agenda-setting in LMICs and, as such, an important contribution to this under-researched area. Two major recommendations are identified from the development of the framework to enhance its implementation. First, given the paucity of formal evidence on mental health in LMICs, informal evidence based on stakeholder experience could be better utilized in these contexts. Second, the use of evidence in mental health agenda-setting in LMICs would be enhanced by involving a broader range of stakeholders in generating, communicating and promoting relevant information.

Research evidence use in local government-led public health interventions: a systematic review

BACKGROUND

Local governments play an important role in improving public health outcomes globally, critical to this work is applying the best-available research evidence. Despite considerable exploration of research use in knowledge translation literature, how research is practically applied by local governments remains poorly understood. This systematic review examined research evidence use in local government-led public health interventions. It focused on how research was used and the type of intervention being actioned.

METHODS

Quantitative and qualitative literature published between 2000 and 2020 was searched for studies that described research evidence use by local governments in public health interventions. Studies reporting interventions developed outside of local government, including knowledge translation interventions, were excluded. Studies were categorised by intervention type and their level of description of research evidence use (where 'level 1' was the highest and 'level 3' was the lowest level of detail).

FINDINGS

The search identified 5922 articles for screening. A final 34 studies across ten countries were included. Experiences of research use varied across different types of interventions. However, common themes emerged including the demand for localised research evidence, the legitimising role of research in framing public health issues, and the need for integration of different evidence sources.

CONCLUSIONS

Differences in how research was used were observed across different local government public health interventions. Knowledge translation interventions aiming to increase research use in local government settings should consider known barriers and facilitators and consider contextual factors associated with different localities and interventions.

Evidence use in the UK's COVID-19 Free School Meals Policy: a thematic content analysis

Free School Meals (FSM) are a well-recognised intervention for tackling food insecurity among school children. National school closures during the COVID-19 pandemic meant that there was a need to rapidly adapt the delivery of FSM. A range of food-assistance policies were implemented, but it is not clear if they were evidence-based. This study aimed to determine the transparency of evidence use and identify other competing influences in the UK's FSM policy decisions. Thematic content analysis was used to review 50 publicly available policy documents and debate transcripts on FSM policy published between March 2020-2021. This period covered the first national school closures (March 2020-July 2020), school holidays and the second national school closures (January 2021- March 2021). The Evidence Transparency Framework was used to assess the transparency of evidence use in policy documents. We found that overall transparency of evidence use was poor but was better for the Holiday Activities and Food (HAF) programme. The Government showed preference for replacing FSM with food parcels, rather than more agentic modes of food assistance such as cash-vouchers. This preference appeared to be closely aligned with ideological views on the welfare state. With an absence of evidence, value-based reasoning took precedent and was polarised by social media. This paper highlights the need for a formal review into FSM, one which includes a comparison of low and high agentic food assistance policies. Such a review would address the evidence gap, improve food assistance policy, and aid policymakers in future periods of uncertainty.

Evidence synthesis to policy: development and implementation of an impact-oriented approach from the Eastern Mediterranean Region

BACKGROUND

Despite the importance of evidence syntheses in informing policymaking, their production and use remain limited in the Eastern Mediterranean region (EMR). There is a lack of empirical research on approaches to promote and use policy-relevant evidence syntheses to inform policymaking processes in the EMR.

OBJECTIVE

This study sought to describe the development of an impact-oriented approach to link evidence synthesis to policy, and its implementation through selected case studies in Lebanon, a middle-income country in the EMR.

METHODS

This study followed a multifaceted and iterative process that included (i) a review of the literature, (ii) input from international experts in evidence synthesis and evidence-informed health policymaking, and (iii) application in a real-world setting (implementation). We describe four selected case studies of implementation. Surveys were used to assess policy briefs, deliberative dialogues, and post-dialogue activities. Additionally, Kingdon's stream theory was adopted to further explain how and why the selected policy issues rose to the decision agenda.

RESULTS

The approach incorporates three interrelated phases: (1) priority setting, (2) evidence synthesis, and (3) uptake. Policy-relevant priorities are generated through formal priority setting exercises, direct requests by policymakers and stakeholders, or a focusing event. Identified priorities are translated into focused questions that can be addressed via evidence synthesis (phase 1). Next, a scoping of the literature is conducted to identify existing evidence syntheses addressing the question of interest. Unless the team identifies relevant, up-to-date and high-quality evidence syntheses, it proceeds to conducting SRs addressing the priority questions of interest (phase 2). Next, the team prepares knowledge translation products (e.g., policy briefs) for undertaking knowledge uptake activities, followed by monitoring and evaluation (phase 3). There are two prerequisites to the application of the approach: enhancing contextual awareness and capacity strengthening. The four case studies illustrate how evidence produced from the suites of activities was used to inform health policies and practices.

CONCLUSIONS

To our knowledge, this is the first study to describe both the development and implementation of an approach to link evidence synthesis to policy in the EMR. We believe the approach will be useful for researchers, knowledge translation platforms, governments, and funders seeking to promote evidence-informed policymaking and practice.

Piloting a classification framework for the types of evidence used in alcohol policymaking

INTRODUCTION

Most studies of alcohol policy have focussed on the role of industry. However, little is known about the evidence base used in alcohol policymaking or policymakers' actions in the field. Here, we mapped the different evidence types used in a case study to construct a classification framework of the evidence types used in alcohol policymaking.

METHODS

Using a case study from the state-level in Australia, we used content analysis to delineate the evidence types cited across six phases of a policymaking process. We then grouped these types into a higher-level classification framework. We used descriptive statistics to study how the different evidence types were used in the policymaking process.

RESULTS

Thirty-one evidence types were identified in the case study, across four classes of knowledge: person knowledge, shared knowledge, studied knowledge and practice knowledge. The participating public preferenced studied knowledge. Policymakers preferenced practice knowledge over all other types of knowledge.

DISCUSSION AND CONCLUSION

The classification framework expands on models of evidence and knowledge used across public health, by mapping new evidence types and proposing an inductive method of classification. The policymakers' preferences found here are in line with theories regarding the alcohol industry's influence on policymaking. The classification framework piloted here can provide a useful tool to examine the evidence base used in decision-making. Further study of evidence types used in policymaking processes can help inform research translation and advocacy efforts to produce healthier alcohol policies.

Finagle's laws of information: lessons learnt evaluating a complex health intervention in Nigeria

Evaluations cannot support evidence-informed decision making if they do not provide the information needed by decision-makers. In this article, we reflect on our own difficulties evaluating the Geo-Referenced Infrastructure and Demographic Data for Development (GRID3) approach, an intervention that provides high-resolution demographic and geographical information to support health service delivery. GRID3 was implemented in Nigeria's northern states to support polio (2012-2019) and measles immunisation campaigns (2017-2018). Generalising from our experience we argue that Finagle's four laws of information capture a particular set of challenges when evaluating complex interventions: the weak causal claims derived from quasi-experimental studies and secondary analyses of existing data (the information we have is not the data we want); the limited external validity of counterfactual impact evaluations (the information we want is not the information we need); the absence of reliable monitoring data on implementation processes (the information we need is not what we can obtain) and the overly broad scope of evaluations attempting to generate both proof of concept and evidence for upscaling (the information we can obtain costs more than we want to pay). Evaluating complex interventions requires a careful selection of methods, thorough analyses and balanced judgements. Funders, evaluators and implementers share a joint responsibility for their success.

Tailoring dissemination strategies to increase evidence-informed policymaking for opioid use disorder treatment: study protocol

BACKGROUND

Policy is a powerful tool for systematically altering healthcare access and quality, but the research to policy gap impedes translating evidence-based practices into public policy and limits widespread improvements in service and population health outcomes. The US opioid epidemic disproportionately impacts Medicaid members who rely on publicly funded benefits to access evidence-based treatment including medications for opioid use disorder (MOUD). A myriad of misaligned policies and evidence-use behaviors by policymakers across federal agencies, state Medicaid agencies, and managed care organizations limit coverage of and access to MOUD for Medicaid members. Dissemination strategies that improve policymakers' use of current evidence are critical to improving MOUD benefits and reducing health disparities. However, no research describes key determinants of Medicaid policymakers' evidence use behaviors or preferences, and few studies have examined data-driven approaches to developing dissemination strategies to enhance evidence-informed policymaking. This study aims to identify determinants and intermediaries that influence policymakers' evidence use behaviors, then develop and test data-driven tailored dissemination strategies that promote MOUD coverage in benefit arrays.

METHODS

Guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, we will conduct a national survey of state Medicaid agency and managed care organization policymakers to identify determinants and intermediaries that influence how they seek, receive, and use research in their decision-making processes. We will use latent class methods to empirically identify subgroups of agencies with distinct evidence use behaviors. A 10-step dissemination strategy development and specification process will be used to tailor strategies to significant predictors identified for each latent class. Tailored dissemination strategies will be deployed to each class of policymakers and assessed for their acceptability, appropriateness, and feasibility for delivering evidence about MOUD benefit design.

DISCUSSION

This study will illuminate key determinants and intermediaries that influence policymakers' evidence use behaviors when designing benefits for MOUD. This study will produce a critically needed set of data-driven, tailored policy dissemination strategies. Study results will inform a subsequent multi-site trial measuring the effectiveness of tailored dissemination strategies on MOUD benefit design and implementation. Lessons from dissemination strategy development will inform future research about policymakers' evidence use preferences and offer a replicable process for tailoring dissemination strategies.

Applying behaviour change models to policy-making: development and validation of the Policymakers' Information Use Questionnaire (POLIQ)

BACKGROUND

The purpose of this study was to develop and validate the Policymakers' Information Use Questionnaire (POLIQ) to capture the intention of individuals in decision-making positions, such as health policy-makers, to act on research-based evidence in order to inform theory and the application of behaviour change models to decision-making spheres.

METHODS

The development and validation comprised three steps: item generation, qualitative face validation with cognitive debriefing and factorial construct validation. Confirmatory factor analysis was applied to estimate item-domain correlations for five predefined constructs relating to content, beliefs, behaviour, control and intent. Cronbach's alpha coefficient was calculated to assess the overall consistency of questionnaire items with the predefined constructs. Participants in the item generation and face validation were health and policy researchers and two former decision-makers (former assistant deputy ministers) from the Canadian provincial level. Participants in the construct validation were 39 Canadian decision-makers at various positions of municipal, provincial and federal jurisdiction who participated in a series of policy dialogues focused on childhood disability.

RESULTS

Cognitive debriefing allowed for small adjustments in language for clarity, including simultaneous validation of the English and French questionnaires. Participants found that the questions were clear and addressed the domains being targeted. Internal consistency of items belonging to the respective questionnaire domains was moderate to high, with estimated Cronbach's alpha values ranging from 0.67 to 0.84. Estimated item-domain correlations indicated moderate to high measurement performance for the domains norm, control and beliefs, whereas weak to moderate correlations resulted for the constructs content and intent. Estimated imprecision of factor loadings (95% confidence interval widths) was considerable for the questionnaire domains content and intent.

CONCLUSION

Measuring decision-makers' behaviour in relation to research evidence use is challenging. We provide initial evidence on face validity and appropriate measurement properties of the POLIQ based on a convenience sample of decision-makers in social and health policy. Larger validation studies and further psychometric property testing will support further utility of the POLIQ.

Research to law: A qualitative study of Massachusetts' 2018 Care Act expanding emergency department initiation of medication for opioid use disorder

BACKGROUND

Initiating medication for opioid use disorder (MOUD) during emergency department (ED) visits is an important innovation to engage individuals in addiction treatment. In 2018, Massachusetts passed the CARE Act, becoming the first state to legislate that hospitals with EDs must be able to offer MOUD. We performed a qualitative study to explore factors influencing policy enactment.

METHODS

Semi-structured interviews were conducted in 2019 with ten key stakeholders involved in the policymaking process representing state government, hospitals, physician professional societies, and recovery/behavioral health organizations. Data were analyzed in 2020-2021 using a hybrid inductive-deductive approach.

RESULTS

The first key theme stakeholders expressed was the importance of research and public health consensus; they described consensus building within existing coalitions regarding the pressing need for action, and supporting expansion of treatment with this evidence-based strategy. Second, stakeholders discussed overcoming financing and feasibility concerns by passing budget-neutral legislation and ensuring flexibility for diverse hospital types. Lastly, stakeholders looked towards implementation, describing the implementation guide development process and ensuring capacity for continuing treatment existed throughout the state.

CONCLUSIONS

This study suggests that research supporting the effectiveness of ED MOUD induction drove the passage of this state legislation. Long-term collaboration between diverse stakeholders towards a common goal of increasing access to evidence-based treatment to address the opioid epidemic was also perceived as facilitating the law's passage. Policymakers and advocates in other states may look towards Massachusetts's legislative process as a model for implementing similar legislation as part of their strategies to address the drug overdose crisis.

Health Policy and Systems Research Capacities in Ethiopia and Ghana: Findings From a Self-Assessment

Government investment in strengthening health policy and systems research capacities is needed to enhance the generation of evidence for effective policy making. Researchers’ engagement in the policy-making process helps shape policy-relevant research and support policy-relevant decisions.

Introduction:

Health systems are complex. Policies targeted at health system development may be informed by health policy and systems research (HPSR). This study assesses HPSR capacity to generate evidence and inform policy in Ethiopia and Ghana.

Methods:

We used a mixed-methods approach including a self-administered survey at selected HPSR institutes and in-depth interviews of policy makers.

Results:

Both countries have limited capacity to generate HPSR evidence, especially in terms of mobilizing adequate funding and retaining a critical number of competent researchers who understand complex policy processes, have the skills to influence policy, and know policy makers’ demands for evidence. Common challenges are limited government research funding, rigidity in executing the research budget, and reliance on donor funding that might not respond to national health priorities. There are no large research programs in either country. The annual number of HPSR projects per research institute in Ethiopia (10 projects) was higher than in Ghana (2.5 projects), Ethiopia has a significantly smaller annual budget for health research. Policy makers in the 2 countries increasingly recognize the importance of evidence-informed policy making, but various challenges remain in building effective interactions with HPSR institutes.

Conclusion:

We propose 3 synergistic recommendations to strengthen HPSR capacity in Ethiopia and Ghana. First, strengthen researchers’ capacity and enhance their opportunities to know policy actors; engage with the policy community; and identify and work with policy entrepreneurs, who have attributes, skills, and strategies to achieve a successful policy. Second, deliver policy-relevant research findings in a timely way and embed research into key health programs to guide effective implementation. Third, mobilize local and international funding to strengthen HPSR capacities as well as address challenges with recruiting and retaining a critical number of talented researchers. These recommendations may be applied to other low- and middle-income countries to strengthen HPSR capacities.

Evidence-Informed Policy-Making: Are We Doing Enough? Comment on "Examining and Contextualizing Approaches to Establish Policy Support Organizations - A Mixed Method Study"

In their study of manifestations of policy support organizations (PSOs), Al Sabahi et al found that PSOs are united in their goal to support evidence-informed policy-making (EIPM), albeit with differing approaches. Their article is an important contribution to the body of research on evidence utilization and implementation. The unprecedented evidence climate presented by coronavirus disease 2019 (COVID-19) provides a unique window to motivate EIPM implementation. Research such as Al Sabahi and colleagues must prompt a dialogue regarding how best to address some of the current shortcomings in the field of EIPM. Monitoring and evaluation of best practices in EIPM is scarce. EIPM uptake is unsatisfactory, and the scientific community needs to ask itself why that is and what can be done. And, we should strive to develop a gradient that discerns between the convenient and the essential so countries can evaluate and pursue the policies to best address their greatest pain points through evidence.

Trends and Opportunities for Bridging Prevention Science and US Federal Policy

Prevention science sheds light on complex social policy problems, yet its social impact cannot reach full potential without the uptake of research evidence by policymakers. This mixed-methods study examined the US federal legislation pertaining to justice-involved youth to reveal opportunities for strengthening the use of prevention science in legislation. The results indicated that research language, particularly references to the type of study (e.g., longitudinal) or methodology (e.g., data mining), within bills predicted bill progression out of committee and enactment. Rigorous scientific methods may either lend credence to a bill during its progression in the legislative process or may be infused in language during mark-up and negotiation of bills that successfully progress in Congress. In-depth bill coding illustrated the ways that research has been used in legislation to define problems, reinforce effective practice, generate knowledge through research and evaluation, and disseminate findings. A prominent implication of these findings is that policies could be used to improve data monitoring and evaluation capacity in ways that enhance the implementation of evidence-based interventions. The comprehensive use of research in legislation increases the likelihood that policies reach their intended outcomes and benefit those they are designed to serve.

Inter-agency collaboration is associated with increased frequency of research use in children's mental health policy making

OBJECTIVE

To determine whether the self-report frequency of inter-agency collaboration about children's mental health issues is associated with the self-report frequency of using research evidence in children's mental health policy and program decision making in mental health agencies (MHAs).

DATA SOURCES

Primary data were collected through web-based surveys of state (N = 221) and county (N = 117) MHA officials.

DESIGN

The primary independent variable was a composite score quantifying the frequency of collaboration about children's mental health issues between officials in MHAs and six other state agencies. The dependent variables were composite scores quantifying the frequency of research use in children's mental health policy and program decision making in general and for specific purposes (i.e., conceptual, instrumental, tactical, imposed). Covariates were composite scores quantifying well-established determinants of research use (e.g., agency leadership, research use skills) in agency policy and program decision making.

DATA METHODS

Separate multiple linear regression models estimated associations between frequency of inter-agency collaboration and research use scores, adjusting for other determinants of research use, respondent state, and other covariates. Data from state and county officials were analyzed separately.

PRINCIPAL FINDINGS

The frequency of inter-agency collaboration was positively and independently associated with the frequency of research use in children's mental health policy making among state (β = 0.22, p = 0.004) and county (β = 0.39, p < 0.0001) MHA officials. Inter-agency collaboration was also the only variable significantly associated with the frequency of research use for all four specific purposes among state MHA officials, and similar findings we observed among county MHA officials. The magnitudes of associations between inter-agency collaboration and frequency of research use were generally stronger than for more well-established determinants of research use in policy making.

CONCLUSIONS

Strategies that promote collaboration between MHA officials and external agencies could increase the use of research evidence in children's mental health policy and program decision making in MHAs.

Complexity and Evidence in Health Sector Decision Making: Lessons from Tuberculosis Infection Prevention in South Africa

To better understand and plan health systems featuring multiple levels and complex causal elements, there have been increasing attempts to incorporate tools arising from complexity science to inform decisions. The utilization of new planning approaches can have important implications for the types of evidence that inform health policymaking and the mechanisms through which they do so. This paper presents an empirical analysis of the application of one such tool—system dynamics modelling (SDM)—within a tuberculosis control programme in South Africa in order to explore how SDM was utilized, and to reflect on the implications for evidence-informed health policymaking. We observed group model building workshops that served to develop the SDM process and undertook 19 qualitative interviews with policymakers and practitioners who partook in these workshops. We analysed the relationship between the SDM process and the use of evidence for policymaking through four conceptual perspectives: (1) a rationalist knowledge-translation view that considers how previously-generated research can be taken up into policy; (2) a programmatic approach that considers existing goals and tasks of decision-makers, and how evidence might address them; (3) a social constructivist lens exploring how the process of using an evidentiary planning tool like SDM can shape the understanding of problems and their solutions; and (4) a normative perspective that recognizes that stakeholders may have different priorities, and thus considers which groups are included and represented in the process. Each perspective can provide useful insights into the SDM process and the political nature of evidence use. In particular, SDM can provide technical information to solve problems, potentially leave out other concerns and influence how problems are conceptualized by formalizing the boundaries of the policy problem and delineating particular solution sets. Undertaking the process further involves choices on stakeholder inclusion affecting whose interests may be served as evidence to inform decisions.

Mapping the Qualitative Evidence Base on the Use of Research Evidence in Health Policy-Making: A Systematic Review

BACKGROUND

The use of research evidence in health policy-making is a popular line of inquiry for scholars of public health and policy studies, with qualitative methods constituting the dominant strategy in this area. Research on this subject has been criticized for, among other things, disproportionately focusing on high-income countries; overemphasizing 'barriers and facilitators' related to evidence use to the neglect of other, less descriptive concerns; relying on descriptive, rather than in-depth explanatory designs; and failing to draw on insights from political/policy studies theories and concepts. We aimed to comprehensively map the global, peer-reviewed qualitative literature on the use of research evidence in health policy-making and to provide a descriptive overview of the geographic, temporal, methodological, and theoretical characteristics of this body of literature.

METHODS

We conducted a systematic review following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. We searched nine electronic databases, hand-searched 11 health- and policy-related journals, and systematically scanned the reference lists of included studies and previous reviews. No language, date or geographic limitations were imposed.

RESULTS

The review identified 319 qualitative studies on a diverse array of topics related to the use of evidence in health policy-making, spanning 72 countries and published over a nearly 40 year period. A majority of these studies were conducted in high-income countries, but a growing proportion of the research output in this area is now coming from low- and middle-income countries, especially from sub-Saharan Africa. While over half of all studies did not use an identifiable theory or framework, and only one fifth of studies used a theory or conceptual framework drawn from policy studies or political science, we found some evidence that theory-driven and explanatory (eg, comparative case study) designs are becoming more common in this literature. Investigations of the barriers and facilitators related to evidence use constitute a large proportion but by no means a majority of the work in this area.

CONCLUSION

This review provides a bird's eye mapping of the peer reviewed qualitative research on evidence-to-policy processes, and has identified key features of - and gaps within - this body of literature that will hopefully inform, and improve, research in this area moving forward.

Open science at the science-policy interface: bringing in the evidence?

Part of the current enthusiasm about open science stems from its promises to reform scientific practice in service of the common good, to ensure that scientific outputs will be found and reused more easily, and to enhance scientific impact on policy and society. With this article, we question this optimism by analysing the potential for open science practices to enhance research uptake at the science–policy interface. Science advice is critical to help policy-makers make informed decisions. Likewise, some interpretations of open science hold that making research processes and outputs more transparent and accessible will also enhance the uptake of results by policy and society at large. However, we argue that this hope is based on an unjustifiably simplistic understanding of the science–policy interface that leaves key terms (“impact”, “uptake”) undefined. We show that this understanding—based upon linear models of research uptake—likewise grounds the influential “evidence–policy gap” diagnosis which holds that to improve research uptake, communication and interaction between researchers and policy-makers need to be improved. The overall normative stance of both discussions has sidelined empirical description of the science–policy interface, ignoring questions about the underlying differences between the policy domain and academia. Importantly, both open science and literature on closing the evidence–policy gap recommend improving communication (in terms of either the content or the means) as a viable strategy. To correct some of these views, we combine insights from policy theory with a narrative review of the literature on the evidence–policy gap in the health domain and find that removing barriers to access by itself will not be enough to foster research uptake.

PROTOCOL: Guidance for stakeholder engagement in guideline development: A systematic review

This is the protocol for a Campbell systematic review. The objectives are as follows: to identify, describe, and summarize existing guidance and methods for multistakeholder engagement throughout the health guideline development process.

Knowledge Gaps and Research Priorities on the Health Effects of Heatwaves: A Systematic Review of Reviews

Although extreme weather events have played a constant role in human history, heatwaves (HWs) have become more frequent and intense in the past decades, causing concern especially in light of the increasing evidence on climate change. Despite the increasing number of reviews suggesting a relationship between heat and health, these reviews focus primarily on mortality, neglecting other important aspects. This systematic review of reviews gathered the available evidence from research syntheses conducted on HWs and health. Following the PRISMA guidelines, 2232 records were retrieved, and 283 reviews were ultimately included. Information was extracted from the papers and categorized by topics. Quantitative data were extracted from meta-analyses and, when not available, evidence was collected from systematic reviews. Overall, 187 reviews were non-systematic, while 96 were systematic, of which 27 performed a meta-analysis. The majority evaluated mortality, morbidity, or vulnerability, while the other topics were scarcely addressed. The following main knowledge gaps were identified: lack of a universally accepted definition of HW; scarce evidence on the HW-mental health relationship; no meta-analyses assessing the risk perception of HWs; scarcity of studies evaluating the efficacy of adaptation strategies and interventions. Future efforts should meet these priorities to provide high-quality evidence to stakeholders.

Nutrition modeling tools: a qualitative study of influence on policy decision making and determining factors

Nutrition modeling tools (NMTs) generate evidence to inform policy and program decision making; however, the literature is generally limited to modeling methods and results, rather than use cases and their impacts. We aimed to document the policy influences of 12 NMTs and identify factors influencing them. We conducted semistructured interviews with 109 informants from 30 low‐ and middle‐income country case studies and used thematic analysis to understand the data. NMTs were mostly applied by international organizations to inform national government decision making. NMT applications contributed to enabling environments for nutrition and influenced program design and policy in most cases; however, this influence could be strengthened. Influence was shaped by processes for applying the NMTs; ownership of the analysis and data inputs, and capacity building in NMT methods, encouraged uptake. Targeting evidence generation at specific policy cycle stages promoted uptake; however, where advocacy capacity allowed, modeling was embedded ad hoc into emerging policy discussions and had broader influence. Meanwhile, external factors, such as political change and resource constraints of local partner organizations, challenged NMT implementation. Importantly, policy uptake was never the result of NMTs exclusively, indicating they should be nested persistently and strategically within the wider evidence and advocacy continuum, rather than being stand‐alone activities.

Domains and processes for institutionalizing evidence-informed health policy-making: a critical interpretive synthesis

BACKGROUND

While calls for institutionalization of evidence-informed policy-making (EIP) have become stronger in recent years, there is a paucity of methods that governments and organizational knowledge brokers can use to sustain and integrate EIP as part of mainstream health policy-making. The objective of this paper was to conduct a knowledge synthesis of the published and grey literatures to develop a theoretical framework with the key features of EIP institutionalization.

METHODS

We applied a critical interpretive synthesis (CIS) that allowed for a systematic, yet iterative and dynamic analysis of heterogeneous bodies of literature to develop an explanatory framework for EIP institutionalization. We used a "compass" question to create a detailed search strategy and conducted electronic searches to identify papers based on their potential relevance to EIP institutionalization. Papers were screened and extracted independently and in duplicate. A constant comparative method was applied to develop a framework on EIP institutionalization. The CIS was triangulated with the findings of stakeholder dialogues that involved civil servants, policy-makers and researchers.

RESULTS

We identified 3001 references, of which 88 papers met our eligibility criteria. This CIS resulted in a definition of EIP institutionalization as the "process and outcome of (re-)creating, maintaining and reinforcing norms, regulations, and standard practices that, based on collective meaning and values, actions as well as endowment of resources, allow evidence to become-over time-a legitimate and taken-for-granted part of health policy-making". The resulting theoretical framework comprised six key domains of EIP institutionalization that capture both structure and agency: (1) governance; (2) standards and routinized processes; (3) partnership, collective action and support; (4) leadership and commitment; (5) resources; and (6) culture. Furthermore, EIP institutionalization is being achieved through five overlapping stages: (i) precipitating events; (ii) de-institutionalization; (iii) semi-institutionalization (comprising theorization and diffusion); (iv) (re)-institutionalization; and (v) renewed de-institutionalization processes.

CONCLUSIONS

This CIS advances the theoretical and conceptual discussions on EIP institutionalization, and provides new insights into an evidence-informed framework for initiating, strengthening and/or assessing efforts to institutionalize EIP.

Using machine learning as a surrogate model for agent-based simulations

In this proof-of-concept work, we evaluate the performance of multiple machine-learning methods as surrogate models for use in the analysis of agent-based models (ABMs). Analysing agent-based modelling outputs can be challenging, as the relationships between input parameters can be non-linear or even chaotic even in relatively simple models, and each model run can require significant CPU time. Surrogate modelling, in which a statistical model of the ABM is constructed to facilitate detailed model analyses, has been proposed as an alternative to computationally costly Monte Carlo methods. Here we compare multiple machine-learning methods for ABM surrogate modelling in order to determine the approaches best suited as a surrogate for modelling the complex behaviour of ABMs. Our results suggest that, in most scenarios, artificial neural networks (ANNs) and gradient-boosted trees outperform Gaussian process surrogates, currently the most commonly used method for the surrogate modelling of complex computational models. ANNs produced the most accurate model replications in scenarios with high numbers of model runs, although training times were longer than the other methods. We propose that agent-based modelling would benefit from using machine-learning methods for surrogate modelling, as this can facilitate more robust sensitivity analyses for the models while also reducing CPU time consumption when calibrating and analysing the simulation.

Dynamics of cumulative advantage and threats to equity in open science: a scoping review

Open Science holds the promise to make scientific endeavours more inclusive, participatory, understandable, accessible and re-usable for large audiences. However, making processes open will not per se drive wide reuse or participation unless also accompanied by the capacity (in terms of knowledge, skills, financial resources, technological readiness and motivation) to do so. These capacities vary considerably across regions, institutions and demographics. Those advantaged by such factors will remain potentially privileged, putting Open Science's agenda of inclusivity at risk of propagating conditions of 'cumulative advantage'. With this paper, we systematically scope existing research addressing the question: 'What evidence and discourse exists in the literature about the ways in which dynamics and structures of inequality could persist or be exacerbated in the transition to Open Science, across disciplines, regions and demographics?' Aiming to synthesize findings, identify gaps in the literature and inform future research and policy, our results identify threats to equity associated with all aspects of Open Science, including Open Access, Open and FAIR Data, Open Methods, Open Evaluation, Citizen Science, as well as its interfaces with society, industry and policy. Key threats include: stratifications of publishing due to the exclusionary nature of the author-pays model of Open Access; potential widening of the digital divide due to the infrastructure-dependent, highly situated nature of open data practices; risks of diminishing qualitative methodologies as 'reproducibility' becomes synonymous with quality; new risks of bias and exclusion in means of transparent evaluation; and crucial asymmetries in the Open Science relationships with industry and the public, which privileges the former and fails to fully include the latter.

The development of clinical guidelines in China: insights from a national survey

Background

Previous research suggests that the quality of clinical guidelines (CGs) in China is suboptimal. However, little is known about the methodology that CGs follow. We conducted a national survey of methods used by Chinese CG developers for CG development, adaptation, and updating.

Methods

We used a previously piloted questionnaire based on methodologies of CG development, adaptation, and updating, which was distributed during September–November 2020 to 114 organizations identified from published Chinese CGs (searched 2017–2020), recommended by Chinese CG developers, and recommended by clinical discipline experts.

Results

We collected 48 completed questionnaires (42.1% response). Most organizations developed CGs based on scientific evidence (89.6%), existing CGs (75%), or expert experience and opinion (64.6%). Only a few organizations had a specific CG development division (6.3%), a CG monitoring plan (on clinicians 33.3%; on patients 18.8%), funding (33.3%), or a conflict-of-interest (COI) management policy (23.4%). Thirty (62.5%) organizations reported using a CG development methodology handbook, from international organizations (14/30, 46.7%), methodology or evaluation resources (3/30, 10.0%), expert experience and opinion (3/30, 10.0%), or in-house handbooks (3/30, 10.0%). One organization followed a published adaptation methodology. Thirty-eight organizations (88.4%) reported de novo CG development: 21 (55.3%) formed a CG working group, and 29 (76.3%) evaluated the quality of evidence (21 [72.4%] using a methodological tool). Nineteen organizations (52.8%) reported CG adaptation: three (31.6%) had an adaptation working group, and 12 (63.2%) evaluated the quality of source CGs (2 (16.7%) using the AGREE II instrument). Thirty-three organizations (68.8%) updated their CGs, seven (17.5%) using a formal updating process.

Conclusions

Our study describes how CGs are developed in a middle-income country like China. To ensure better healthcare, there is still an important need for improvement in the development, adaptation, and updating of CG in China.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00799-7.

Health evidence in United Kingdom House of Commons Select Committee inquiries

OBJECTIVES

The objective of the study was to explore how health evidence is submitted to national policymakers in the United Kingdom outside of the health sector.

STUDY DESIGN

The study design used is document review.

METHODS

The parliament.uk website was searched for House of Commons Select Committee inquiries relevant to health but not undertaken by the Health and Social Care Committee between 2015 and 2019. The written and oral evidence used in these inquiries were searched for submissions by health sources.

RESULTS

Twenty-two inquiries were found, covering a range of determinants of health. A median of 10 pieces of written evidence (interquartile range [IQR] 5-16) and 16% of the oral evidence (IQR 6-32%) was provided by health sources, per inquiry. Health researchers contributed written evidence to 19 and oral evidence to 13 inquires. National public health organisations contributed written evidence to 11 and oral evidence to nine inquiries.

CONCLUSIONS

A significant number of inquiries relating to the determinants of health were not carried out by the Health and Social Care Committee. Health sources variably contributed evidence to these inquiries. There is an opportunity to submit more evidence to national policymakers to advocate for healthy policies across sectors.

Policymakers' Research Capacities, Engagement, and Use of Research in Public Health Policymaking

The use of research in public health policymaking is one of the prerequisites for successfully implemented health policies which have better population health as an outcome. This policy process is influenced by the actors involved under the policy umbrella, with inter-related contextual factors and specific structural and institutional circumstances. Our study investigates how policymakers’ research capacities influence the use of research in the health policy process and identify areas where capacity-building interventions give the most meaning and impact. Furthermore, we investigate policymakers’ research engagement and use this to inform public health policy in the public sector in Denmark. We collect and report data using Seeking, Engaging with, and Evaluation Research (SEER) methodology. Policymakers are reported to have research capacity, but it is questionable how those competences have actually been used in policymaking. Decision-makers were often not aware or did not know about the existing organizational tools and systems for research engagement and use and two third of respondents had not been part of any research activities or had any collaboration with researchers. Overall, research use in public health policymaking and evaluation was limited. As a conclusion, we propose that capacity-building interventions for increasing research use and collaboration in EIPM should be context-oriented, measurable, and sustainable in developing individual and organizational competences.

Closing the Virtual Gap in Health Care: A Series of Case Studies Illustrating the Impact of Embedding Evaluation Alongside System Initiatives

Early decisions relating to the implementation of virtual care relied on necessity and clinical judgement, but there is a growing need for the generation of evidence to inform policy and practice designs. The need for stronger partnerships between researchers and decision-makers is well recognized, but how these partnerships can be structured and how research can be embedded alongside existing virtual care initiatives remain unclear. We present a series of case studies that illustrate how embedded research can inform policy decisions related to the implementation of virtual care, where decisions are either to (1) discontinue (red light), (2) redesign (yellow light), or (3) scale up existing initiatives (green light). Data were collected through document review and informal interviews with key study personnel. Case 1 involved an evaluation of a mobile diabetes platform that demonstrated a mismatch between the setting and the technology (decision outcome: discontinue). Case 2 involved an evaluation of a mental health support platform that suggested evidence-based modifications to the delivery model (decision outcome: redesign). Case 3 involved an evaluation of video visits that generated evidence to inform the ideal model of implementation at scale (decision outcome: scale up). In this paper, we highlight the characteristics of the partnership and the process that enabled success and use the cases to illustrate how these characteristics were operationalized. Structured communication included monthly check-ins and iterative report development. We also outline key characteristics of the partnership (ie, trust and shared purpose) and the process (ie, timeliness, tailored reporting, and adaptability) that drove the uptake of evidence in decision-making. Across each case, the evaluation was designed to address policy questions articulated by our partners. Furthermore, structured communication provided opportunities for knowledge mobilization. Structured communication was operationalized through monthly meetings as well as the delivery of interim and final reports. These case studies demonstrate the importance of partnering with health system decision-makers to generate and mobilize scientific evidence. Embedded research partnerships founded on a shared purpose of system service provided an effective strategy to bridge the oft-cited gap between science and policy. Structured communication provided a mechanism for collaborative problem-solving and real-time feedback, and it helped contextualize emerging insights.

Linking Knowledge With Action When Engagement is Out of Reach: Three Contextual Features of Effective Public Health Communication

Scholars and practitioners often promote direct engagement between policymakers, health workers and researchers as a strategy for overcoming barriers to utilizing scientific knowledge in health policy. However, in many settings public health officials rarely have opportunities to interact with researchers, which is a problem further exacerbated by the coronavirus disease 2019 pandemic. One prominent theory argues that policy actors will trust and utilize research findings when they perceive them to be salient, credible and legitimate. We draw on this theory to examine the conditions facilitating greater uptake of new knowledge among health officials when engagement is out of reach and they are instead exposed to new ideas through written mass communication. Using data from a survey experiment with about 260 health workers and administrators in Honduras, we find that messages from a technocratic sender based on statistical evidence improved perceptions of salience, credibility and legitimacy (SCL). Additionally, perceptions of SCL are three contextual features that operate as joint mediators between knowledge and action, and several individual characteristics also influence whether officials trust research findings enough to apply them when formulating and implementing health policies. This research can help inform the design of context-sensitive knowledge translation and exchange strategies to advance the goals of evidence-based public health, particularly in settings where direct engagement is difficult to achieve.

Lawmakers' use of scientific evidence can be improved

Core to the goal of scientific exploration is the opportunity to guide future decision-making. Yet, elected officials often miss opportunities to use science in their policymaking. This work reports on an experiment with the US Congress-evaluating the effects of a randomized, dual-population (i.e., researchers and congressional offices) outreach model for supporting legislative use of research evidence regarding child and family policy issues. In this experiment, we found that congressional offices randomized to the intervention reported greater value of research for understanding issues than the control group following implementation. More research use was also observed in legislation introduced by the intervention group. Further, we found that researchers randomized to the intervention advanced their own policy knowledge and engagement as well as reported benefits for their research following implementation.

Situating agent-based modelling in population health research

Today's most troublesome population health challenges are often driven by social and environmental determinants, which are difficult to model using traditional epidemiological methods. We agree with those who have argued for the wider adoption of agent-based modelling (ABM) in taking on these challenges. However, while ABM has been used occasionally in population health, we argue that for ABM to be most effective in the field it should be used as a means for answering questions normally inaccessible to the traditional epidemiological toolkit. In an effort to clearly illustrate the utility of ABM for population health research, and to clear up persistent misunderstandings regarding the method's conceptual underpinnings, we offer a detailed presentation of the core concepts of complex systems theory, and summarise why simulations are essential to the study of complex systems. We then examine the current state of the art in ABM for population health, and propose they are well-suited for the study of the 'wicked' problems in population health, and could make significant contributions to theory and intervention development in these areas.

How does embedded implementation research work? Examining core features through qualitative case studies in Latin America and the Caribbean

Innovative strategies are needed to improve the delivery of evidence-informed health interventions. Embedded implementation research (EIR) seeks to enhance the generation and use of evidence for programme improvement through four core features: (1) central involvement of programme/policy decision-makers in the research cycle; (2) collaborative research partnerships; (3) positioning research within programme processes and (4) research focused on implementation. This paper examines how these features influence evidence-to-action processes and explores how they are operationalized, their effects and supporting conditions needed. We used a qualitative, comparative case study approach, drawing on document analysis and semi-structured interviews across multiple informant groups, to examine three EIR projects in Bolivia, Colombia and the Dominican Republic. Our findings are presented according to the four core EIR features. The central involvement of decision-makers in EIR was enhanced by decision-maker authority over the programme studied, professional networks and critical reflection. Strong research-practice partnerships were facilitated by commitment, a clear and shared purpose and representation of diverse perspectives. Evidence around positioning research within programme processes was less conclusive; however, as all three cases made significant advances in research use and programme improvement, this feature of EIR may be less critical than others, depending on specific circumstances. Finally, a research focus on implementation demanded proactive engagement by decision-makers in conceptualizing the research and identifying opportunities for direct action by decision-makers. As the EIR approach is a novel approach in these low-resource settings, key supports are needed to build capacity of health sector stakeholders and create an enabling environment through system-level strategies. Key implications for such supports include: promoting EIR and creating incentives for decision-makers to engage in it, establishing structures or mechanisms to facilitate decision-maker involvement, allocating funds for EIR, and developing guidance for EIR practitioners.

Opportunities and disconnects in the use of primary research on schistosomiasis and soil-transmitted helminths for policy and practice: results from a survey of researchers

Even with efforts to facilitate use of evidence in health policy and practice, limited attention has been paid to researchers' perspectives on use of their research in informing public health policy and practice at local, national, and global levels. We conducted a systematic literature search to identify published primary research related to schistosomiasis or soil-transmitted helminths, or both. We then surveyed corresponding authors. Results indicate differences by locations of authors and in conduct of research, especially for research conducted in low- and middle-income countries. Our findings exemplify disparities in research leadership discussed elsewhere. Researchers' perspectives on the use of their work suggest limited opportunities and 'disconnects' that hinder their engagement with policy and other decision-making processes. These findings highlight a need for additional efforts to address structural barriers and enable engagement between researchers and decision-makers.

Loops and Building Blocks: a Knowledge co-Production Framework for Equitable Urban Health

This paper sets out a structured process for the co-production of knowledge between researchers and societal partners and illustrates its application in an urban health equity project in Accra, Ghana. The main insight of this approach is that research and knowledge co-production is always partial, both in the sense of being incomplete, as well as being circumscribed by the interests of participating researchers and societal partners. A second insight is that project-bound societal engagement takes place in a broader context of public and policy debate. The approach to co-production described here is formed of three recursive processes: co-designing, co-analysing, and co-creating knowledge. These 'co-production loops' are themselves iterative, each representing a stage of knowledge production. Each loop is operationalized through a series of research and engagement practices, which we call building blocks. Building blocks are activities and interaction-based methods aimed at bringing together a range of participants involved in joint knowledge production. In practice, recursive iterations within loops may be limited due of constraints on time, resources, or attention. We suggest that co-production loops and building blocks are deployed flexibly.

Dissemination Science in School Mental Health: A Framework for Future Research

There has been an increase in school mental health research aimed at producing generalizable knowledge to address longstanding science-to-practice gaps to increase children's access to evidence-based mental health services. Successful dissemination and implementation are both important pieces to address science-to-practice gaps, but there is conceptual and semantic imprecision that creates confusion regarding where dissemination ends and implementation begins, as well as an imbalanced focus in research on implementation relative to dissemination. In this paper, we provide an enhanced operational definition of dissemination; offer a conceptual model that outlines elements of effective dissemination that can produce changes in awareness, knowledge, perceptions, and motivation across different stakeholder groups; and delineate guiding principles that can inform dissemination science and practice. The overarching goal of this paper is to stimulate future research that aims to advance dissemination science and practice in school mental health.

Competing interests, clashing ideas and institutionalizing influence: insights into the political economy of malaria control from seven African countries

This article explores how malaria control in sub-Saharan Africa is shaped in important ways by political and economic considerations within the contexts of aid-recipient nations and the global health community. Malaria control is often assumed to be a technically driven exercise: the remit of public health experts and epidemiologists who utilize available data to select the most effective package of activities given available resources. Yet research conducted with national and international stakeholders shows how the realities of malaria control decision-making are often more nuanced. Hegemonic ideas and interests of global actors, as well as the national and global institutional arrangements through which malaria control is funded and implemented, can all influence how national actors respond to malaria. Results from qualitative interviews in seven malaria-endemic countries indicate that malaria decision-making is constrained or directed by multiple competing objectives, including a need to balance overarching global goals with local realities, as well as a need for National Malaria Control Programmes to manage and coordinate a range of non-state stakeholders who may divide up regions and tasks within countries. Finally, beyond the influence that political and economic concerns have over programmatic decisions and action, our analysis further finds that malaria control efforts have institutionalized systems, structures and processes that may have implications for local capacity development.

Assessing organisational capacity for evidence-informed health policy and planning: an adaptation of the ORACLe tool for Australian primary health care organizations

Background

Many nations have established primary health care (PHC) organizations that conduct PHC planning for defined geographical areas. The Australian Government established Primary Health Networks (PHNs) in 2015 to develop and commission PHC strategies to address local needs. There has been little written about the capacity of such organizations for evidence-informed planning, and no tools have been developed to assess this capacity, despite their potential to contribute to a comprehensive effective and efficient PHC sector.

Methods

We adapted the ORACLe tool, originally designed to examine evidence-informed policy-making capacity, to examine organizational capacity for evidence-informed planning in meso-level PHC organizations, using PHNs as an example. Semi-structured interviews were conducted with 14 participants from five PHNs, using the ORACLe tool, and scores assigned to responses, in seven domains of capacity.

Results

There was considerable variation between PHNs and capacity domains. Generally, higher capacity was demonstrated in regard to mechanisms which could inform planning through research, and support relationships with researchers. PHNs showed lower capacity for evaluating initiatives, tools and support for staff, and staff training.

Discussion and conclusions

We critique the importance of weightings and scope of some capacity domains in the ORACLe tool. Despite this, with some minor modifications, we conclude the ORACLe tool can identify capacity strengths and limitations in meso-level PHC organizations. Well-targeted capacity development enables PHC organizations’ strategies to be better informed by evidence, for optimal impact on PHC and population health outcomes.

Co-production of two whole-school sexual health interventions for English secondary schools: positive choices and project respect

BACKGROUND

Whole-school interventions represent promising approaches to promoting adolescent sexual health, but they have not been rigorously trialled in the UK and it is unclear if such interventions are feasible for delivery in English secondary schools. The importance of involving intended beneficiaries, implementers and other key stakeholders in the co-production of such complex interventions prior to costly implementation and evaluation studies is widely recognised. However, practical accounts of such processes remain scarce. We report on co-production with specialist providers, students, school staff, and other practice and policy professionals of two new whole-school sexual heath interventions for implementation in English secondary schools.

METHODS

Formative qualitative inquiry involving 75 students aged 13-15 and 23 school staff. A group of young people trained to advise on public health research were consulted on three occasions. Twenty-three practitioners and policy-makers shared their views at a stakeholder event. Detailed written summaries of workshops and events were prepared and key themes identified to inform the design of each intervention.

RESULTS

Data confirmed acceptability of addressing unintended teenage pregnancy, sexual health and dating and relationships violence via multi-component whole-school interventions and of curriculum delivery by teachers (providing appropriate teacher selection). The need to enable flexibility for the timetabling of lessons and mode of parent communication; ensure content reflected the reality of young people's lives; and develop prescriptive teaching materials and robust school engagement strategies to reflect shrinking capacity for schools to implement public-health interventions were also highlighted and informed intervention refinements. Our research further points to some of the challenges and tensions involved in co-production where stakeholder capacity may be limited or their input may conflict with the logic of interventions or what is practicable within the constraints of a trial.

CONCLUSIONS

Multi-component, whole-school approaches to addressing sexual health that involve teacher delivered curriculum may be feasible for implementation in English secondary schools. They must be adaptable to individual school settings; involve careful teacher selection; limit additional burden on staff; and accurately reflect the realities of young people's lives. Co-production can reduce research waste and may be particularly useful for developing complex interventions, like whole-school sexual health interventions, that must be adaptable to varying institutional contexts and address needs that change rapidly. When co-producing, potential limitations in relation to the representativeness of participants, the 'depth' of engagement necessary as well as the burden on participants and how they will be recompensed must be carefully considered. Having well-defined, transparent procedures for incorporating stakeholder input from the outset are also essential. Formal feasibility testing of both co-produced interventions in English secondary schools via cluster RCT is warranted.

TRIAL REGISTRATION

Project Respect: ISRCTN12524938 . Positive Choices: ISRCTN65324176.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Perspectives and Experiences of Policy Makers, Researchers, Health Information Technology Professionals, and the Public on Evidence-Based Health Policies: Protocol for a Qualitative Study

Background

Evidence-based health policy (EBHP) development is critical to the judicious use of public funds. EBHPs increase transparency, accountability, effectiveness, and efficiency of policies. Encouraging collaboration between researchers or knowledge producers and policy makers is important because both communities have distinct professional cultures, resulting in them working separately without understanding each other. Knowledge sharing is a complex process that requires understanding of cultural aspects that may reduce cultural differences and increase the use of common language. Health information technology (HIT) is a useful tool to increase knowledge translation, which may result in the transparent use of evidence and networking in developing EBHPs. Our vision is to leverage HIT tools for a better health system that includes digitalized, open source, evidence-based, and transparent ways for collaboration and development of robust mechanisms and for sharing of synthesized evidence with knowledge user–friendly forms.

Objective

The aim of this study is to develop a conceptual framework on Knowledge translation and health Information Technology for Transparency (KhITT) in policy making and EBHPs (ie, the KhITT framework). The framework will be informed by the views of four key stakeholder groups (ie, policy makers, knowledge producers, HIT professionals, and the public) toward EBHP. The informants may also describe practices that demonstrate the EBHP development process and suggest technology platforms to enable this process.

Methods

We propose an exploratory, descriptive qualitative study to take place in British Columbia, Canada, using in-depth semistructured interviews. To ensure data saturation and trustworthiness, we will use a nonprobability, purposive snowball sample of up to 15 eligible participants in each of the four stakeholder groups. We will analyze the data using content analysis.

Results

The KhITT framework focuses on various stakeholders’ perspectives to better understand their perceived needs and priorities in identifying issues with EBHP, in order to make informed recommendations. Ethics approval has been obtained by the harmonized Behavioural Research Ethics Board at the University of British Columbia. We anticipate that we will complete data collection and analysis by December 2020. Preliminary results will be published in summer 2021.

Conclusions

Our ultimate goal of this study is to develop a conceptual framework and describe the technology platforms that would enable the EBHP process. We anticipate that our rigorous content analysis will be able to produce insights and themes that are able to address our objectives, contribute to an in-depth understanding of the EBHP process within British Columbia, highlight all influential factors, explicitly disseminate and communicate the study results, identify issues with EBHP and provide informed recommendations to address them, and enhance efforts toward transparent EBHPs.

International Registered Report Identifier (IRRID)

PRR1-10.2196/16268

Applying systems thinking to knowledge mobilisation in public health

CONTEXT

Knowledge mobilisation (KM) is a vital strategy in efforts to improve public health policy and practice. Linear models describing knowledge transfer and translation have moved towards multi-directional and complexity-attuned approaches where knowledge is produced and becomes meaningful through social processes. There are calls for systems approaches to KM but little guidance on how this can be operationalised. This paper describes the contribution that systems thinking can make to KM and provides guidance about how to put it into action.

METHODS

We apply a model of systems thinking (which focuses on leveraging change in complex systems) to eight KM practices empirically identified by others. We describe how these models interact and draw out some key learnings for applying systems thinking practically to KM in public health policy and practice. Examples of empirical studies, tools and targeted strategies are provided.

FINDINGS

Systems thinking can enhance and fundamentally transform KM. It upholds a pluralistic view of knowledge as informed by multiple parts of the system and reconstituted through use. Mobilisation is conceived as a situated, non-prescriptive and potentially destabilising practice, no longer conceptualised as a discrete piece of work within wider efforts to strengthen public health but as integral to and in continual dialogue with those efforts. A systems approach to KM relies on contextual understanding, collaborative practices, addressing power imbalances and adaptive learning that responds to changing interactions between mobilisation activities and context.

CONCLUSION

Systems thinking offers valuable perspectives, tools and strategies to better understand complex problems in their settings and for strengthening KM practice. We make four suggestions for further developing empirical evidence and debate about how systems thinking can enhance our capacity to mobilise knowledge for solving complex problems - (1) be specific about what is meant by 'systems thinking', (2) describe counterfactual KM scenarios so the added value of systems thinking is clearer, (3) widen conceptualisations of impact when evaluating KM, and (4) use methods that can track how and where knowledge is mobilised in complex systems.

“Following the Science”: In Search of Evidence-Based Policy for Indoor Air Pollution from Radon in Ireland

Radon, a naturally occurring radioactive gas that can accumulate inside dwellings, represents the second biggest cause of lung cancer globally. In Ireland, radon is linked to approximately 300 lung cancer cases every year, equating to 12% of all lung cancer deaths. Despite the health risks posed by radon air pollution, Ireland lacks well-defined and universally applicable air pollution-related public health policies. Through purposive literature sampling, we critically examine the case of indoor radon policy development in Ireland. Specifically, we analyse the evidence-based policymaking process relating to indoor radon pollution from three different knowledge dimensions, namely political, scientific, and practical knowledge. In doing so, we identify various challenges inherent to pollution-related public policymaking. We highlight the difficulties of balancing and integrating information from multiple disciplines and perspectives and argue that input from multiple scientific areas is crucial, but can only be achieved through continued, dialogic communication between stakeholders. On the basis of our analysis, we suggest that a transdisciplinary perspective, defined as a holistic approach which subordinates disciplines and looks at the dynamics of whole systems, will allow evidence-based policymaking to be effective. We end with recommendations for evidence-based policymaking when it comes to public health hazards such as radon, which are applicable to sustainable air pollution management beyond Ireland.