Use of research to inform public policymaking

Introduction to the Supplement 'Coming together to fight cancer: a series of policy briefs taking stock of the implementation of Europe's Beating Cancer Plan in Belgium'

Cancer is one of the main public health challenges globally. In Europe, it is also the second leading cause of mortality and incidence is likely to increase from nearly 4 million cases per year in 2020 to over 5 million new cases per year by 2040. Recognizing the urgency of tackling the entire disease pathway and supporting European Union (EU) Member States (MS), the European Commission (EC) launched two major initiatives: Europe's Beating Cancer Plan (EBCP) and the Mission on Cancer (MoC). Belgium is one of the few EU MS that has set up a dedicated structure, known as the Belgian EBCP Mirror Group (MG), to support the implementation of the EU strategy against cancer. The MG is a large national stakeholder platform coordinated by the Cancer Centre at Sciensano (Belgian Institute of Health). This Supplement between includes eight policy briefs developed by the MG on the most pressing needs to address in the Belgian cancer field. The policy briefs take into account relevant interventions at national level as well as Belgium's participation in European projects as of 2021-2023.

Evaluating the effectiveness of large language models in abstract screening: a comparative analysis

OBJECTIVE

This study aimed to evaluate the performance of large language models (LLMs) in the task of abstract screening in systematic review and meta-analysis studies, exploring their effectiveness, efficiency, and potential integration into existing human expert-based workflows.

METHODS

We developed automation scripts in Python to interact with the APIs of several LLM tools, including ChatGPT v4.0, ChatGPT v3.5, Google PaLM 2, and Meta Llama 2, and latest tools including ChatGPT v4.0 turbo, ChatGPT v3.5 turbo, Google Gemini 1.0 pro, Meta Llama 3, and Claude 3. This study focused on three databases of abstracts and used them as benchmarks to evaluate the performance of these LLM tools in terms of sensitivity, specificity, and overall accuracy. The results of the LLM tools were compared to human-curated inclusion decisions, gold standard for systematic review and meta-analysis studies.

RESULTS

Different LLM tools had varying abilities in abstract screening. Chat GPT v4.0 demonstrated remarkable performance, with balanced sensitivity and specificity, and overall accuracy consistently reaching or exceeding 90%, indicating a high potential for LLMs in abstract screening tasks. The study found that LLMs could provide reliable results with minimal human effort and thus serve as a cost-effective and efficient alternative to traditional abstract screening methods.

CONCLUSION

While LLM tools are not yet ready to completely replace human experts in abstract screening, they show great promise in revolutionizing the process. They can serve as autonomous AI reviewers, contribute to collaborative workflows with human experts, and integrate with hybrid approaches to develop custom tools for increased efficiency. As technology continues to advance, LLMs are poised to play an increasingly important role in abstract screening, reshaping the workflow of systematic review and meta-analysis studies.

Towards stronger tobacco control policies to curb the smoking epidemic in Spain

Smoking and exposure to secondhand smoke pose a significant risk to the health of populations. Although this evidence is not new, the commitment of countries to implement laws aimed at controlling consumption and eliminating exposure to secondhand smoke is uneven. Thus, in North America or in Europe, locations like California or Ireland, are pioneers in establishing policies aimed at protecting the population against smoking and secondhand smoke. Identifying measures that have worked would help control this important Public Health problem in other countries that are further behind in tobacco control policies. In Spain, there has been almost 15 years of little political action in legislation oriented to control the tobacco epidemic. If we want to achieve the tobacco endgame, new legislative measures must be implemented. In this paper, we have elucidated tobacco control policies that could be implemented and show how different countries have done so.

Global issues, local action: exploring local governments use of research in "tackling climate change and its impacts on health" in Victoria, Australia

BACKGROUND

Local government plays an important role in addressing complex public health challenges. While the use of research in this work is important, it is often poorly understood. This study aimed to build knowledge about how research is used by investigating its use by local government authorities (LGAs) in Victoria, Australia in responding to a new legislative requirement to prioritise climate and health in public health planning. The role of collaboration was also explored.

METHODS

Informed by Normalization Process Theory (NPT), this study adopted multiple research methods, combining data from an online survey and face-to-face interviews. Quantitative data were analysed using descriptive statistics; thematic analysis was used to analyse qualitative data.

RESULTS

Participants comprised 15 interviewees, and 46 survey respondents from 40 different LGAs. Research was most commonly accessed via evidence synthesis, and largely used to inform understanding about climate and health. When and how research was used was shaped by contextual factors including legislation, community values and practical limitations of how research needed to be communicated to decision-makers. Collaboration was more commonly associated with research access than use.

CONCLUSIONS

Greater investment in the production and dissemination of localised research, that identifies local issues (e.g. climate risk factors) and is tailored to the communication needs of local audiences is needed to foster more impactful research use in local public health policy.

Discovering new pathways toward integration between health and sustainable development goals with natural language processing and network science

BACKGROUND

Research on health and sustainable development is growing at a pace such that conventional literature review methods appear increasingly unable to synthesize all relevant evidence. This paper employs a novel combination of natural language processing (NLP) and network science techniques to address this problem and to answer two questions: (1) how is health thematically interconnected with the Sustainable Development Goals (SDGs) in global science? (2) What specific themes have emerged in research at the intersection between SDG 3 ("Good health and well-being") and other sustainability goals?

METHODS

After a descriptive analysis of the integration between SDGs in twenty years of global science (2001-2020) as indexed by dimensions.ai, we analyze abstracts of articles that are simultaneously relevant to SDG 3 and at least one other SDG (N = 27,928). We use the top2vec algorithm to discover topics in this corpus and measure semantic closeness between these topics. We then use network science methods to describe the network of substantive relationships between the topics and identify 'zipper themes', actionable domains of research and policy to co-advance health and other sustainability goals simultaneously.

RESULTS

We observe a clear increase in scientific research integrating SDG 3 and other SDGs since 2001, both in absolute and relative terms, especially on topics relevant to interconnections between health and SDGs 2 ("Zero hunger"), 4 ("Quality education"), and 11 ("Sustainable cities and communities"). We distill a network of 197 topics from literature on health and sustainable development, with 19 distinct network communities - areas of growing integration with potential to further bridge health and sustainability science and policy. Literature focused explicitly on the SDGs is highly central in this network, while topical overlaps between SDG 3 and the environmental SDGs (12-15) are under-developed.

CONCLUSION

Our analysis demonstrates the feasibility and promise of NLP and network science for synthesizing large amounts of health-related scientific literature and for suggesting novel research and policy domains to co-advance multiple SDGs. Many of the 'zipper themes' identified by our method resonate with the One Health perspective that human, animal, and plant health are closely interdependent. This and similar perspectives will help meet the challenge of 'rewiring' sustainability research to co-advance goals in health and sustainability.

Use of existing systematic reviews for the development of evidence-based vaccination recommendations: Guidance from the SYSVAC expert panel

National immunization technical advisory groups (NITAGs) develop immunization-related recommendations and assist policy-makers in making evidence informed decisions. Systematic reviews (SRs) that summarize the available evidence on a specific topic are a valuable source of evidence in the development of such recommendations. However, conducting SRs requires significant human, time, and financial resources, which many NITAGs lack. Given that SRs already exist for many immunization-related topics, and to prevent duplication and overlap of reviews, a more practical approach may be for NITAGs to use existing SRs. Nevertheless, it can be challenging to identify relevant SRs, to select one SR from among multiple SRs, or to critically assess and effectively use them. To support NITAGs, the London School of Hygiene and Tropical Medicine, Robert Koch Institute and collaborators developed the SYSVAC project, which consists of an online registry of systematic reviews on immunization-related topics and an e-learning course, that supports the use of them (both freely accessible at https://www.nitag-resource.org/sysvac-systematic-reviews). Drawing from the e-learning course and recommendations from an expert panel, this paper outlines methods for using existing systematic reviews when making immunization-related recommendations. With specific examples and reference to the SYSVAC registry and other resources, it offers guidance on locating existing systematic reviews; assessing their relevance to a research question, up-to-dateness, and methodological quality and/or risk of bias; and considering the transferability and applicability of their findings to other populations or settings.

A systematic review to describe patterns of animal and human viral research in Rwanda

Rwanda is located in the Central East African region where several viral pathogens with global importance were originally described, including human immunodeficiency virus (HIV), Ebola, Zika, Rift Valley Fever (RVF), dengue and a long list of other neglected tropical viral pathogens. Due to many factors, this region has the potential to become a global hotspot for viral emergence. In Rwanda, viral diseases are underreported and the question is whether this is due to the absence of these viruses or a lack of investigation. Like many developing countries, capabilities in Rwanda need improvement despite research efforts throughout the years. This review describes the status of human and animal virus research in Rwanda and identifies relevant research and operational gaps. A comprehensive search was conducted in PubMed for virus research in Rwanda: 233 primary studies on viruses/viral diseases are indexed with connection to Rwanda. From 1958 to 2020, yearly publications generally increased and HIV/acquired immunodeficiency syndrome is the most studied virus. Compared with human viruses, few studies focus on animal and/or zoonotic viruses. The occurrence of the current severe acute respiratory syndrome coronavirus 2 pandemic shows strengthening warning and surveillance systems is critical to efficient preparedness and response. We recommend investment in human capacity, laboratory facilities and research to inform policy for viral surveillance in Rwanda.

The Impact of Information Presentation and Cognitive Dissonance on Processing Systematic Review Summaries: A Randomized Controlled Trial on Bicycle Helmet Legislation

Background: Summaries of systematic reviews are a reference method for the dissemination of research evidence on the effectiveness of public health interventions beyond the scientific community. Motivated reasoning and cognitive dissonance may interfere with readers' ability to process the information included in such summaries. Methods: We conducted a web experiment on a panel of university-educated North Americans (N = 259) using a systematic review of the effectiveness of bicycle helmet legislation as a test case. The outcome variables were the perceived tentativeness of review findings and attitude toward bicycle helmet legislation. We manipulated two types of uncertainty: (i) deficient uncertainty (inclusion vs. non-inclusion of information on limitations of the studies included in the review) and (ii) consensus uncertainty (consensual findings showing legislation effectiveness vs. no evidence of effectiveness). We also examined whether reported expertise in helmet legislation and the frequency of wearing a helmet while cycling interact with the experimental factors. Results: None of the experimental manipulations had a main effect on the perceived tentativeness. The presentation of consensual efficacy findings had a positive main effect on the attitude toward the legislation. Self-reported expertise had a significant main effect on the perceived tentativeness, and exposing participants with reported expertise to results showing a lack of evidence of efficacy increased their favorable attitude toward the legislation. Participants' helmet use was positively associated with their attitude toward the legislation (but not with perceived tentativeness). Helmet use did not interact with the experimental manipulations. Conclusions: Motivated reasoning and cognitive dissonance influence a reader's ability to process information contained in a systematic review summary.

How Can Policymakers be Encouraged to Support People With Spinal Cord Injury-Scoping Review

STUDY DESIGN

Scoping review.

OBJECTIVE

Regarding that inappropriate medical care approaches, absence of rehabilitation services, and existing barriers in physical, social, and policy environments lead to poor outcomes in individuals with spinal cord injury (SCI) and provision for appropriate interventions and care must be created by health policymakers, we conducted this scoping review to investigate how policymakers can be persuaded to set new plans for individuals with SCI.

METHODS

This review was performed according to Arksey and O'Malley's framework. PubMed was searched in February2019 without language limitation. We looked for other potential gray literature sources and some professional websites. References sections of selected articles were also scanned for other relevant literature.

RESULTS

We included literature that met inclusion criteria to answer our research question. The literature was divided into 3 categories. The first category included economic impact of SCI. The second category included the role of research and developing research strategy. The third category included effective interaction and communication with policymakers.

CONCLUSION

It is essential to consider multiple factors for influencing policymakers' decisions. These factors include knowing how to communicate with policymakers and presenting constructive ideas, providing a source of valid, reliable, and consistent data, considering the role of patients' advocacy groups and Non-Governmental Organizations (NGOs), and presentation of the importance of early intervention in reducing healthcare system costs. Ultimately, the goal is to have a comprehensive and flexible plan for influencing policymakers.

Case Study With a Participatory Approach: Rethinking Pragmatics of Stakeholder Engagement for Implementation Research

The case study design is particularly useful for implementation analysis of complex health care innovations in primary care that can be influenced by the context of dynamic environments. Case studies may be combined with participatory approaches where academics conduct joint research with nonacademic stakeholders, to foster translation of findings results into practice. The aim of this article is to clarify epistemological and methodological considerations of case studies with a participatory approach. It also aims to propose best practice recommendations when using this case study approach. We distinguish between the participatory case study with full co-construction and co-governance, and the case study with a participatory approach whereby stakeholders are consulted in certain phases of the research. We then compare the epistemological posture of 3 prominent case study methodologists, Yin, Stake, and Merriam, to present the epistemological posture of case studies with a participatory approach. The relevance, applications, and procedures of a case study with a participatory approach methodology are illustrated through a concrete example of a primary care research program (PriCARE). We propose 12 steps for designing and conducting a case study with a participatory approach that may help guide researchers in the implementation analysis of complex health care innovations in primary care.

Policymakers' Research Capacities, Engagement, and Use of Research in Public Health Policymaking

The use of research in public health policymaking is one of the prerequisites for successfully implemented health policies which have better population health as an outcome. This policy process is influenced by the actors involved under the policy umbrella, with inter-related contextual factors and specific structural and institutional circumstances. Our study investigates how policymakers’ research capacities influence the use of research in the health policy process and identify areas where capacity-building interventions give the most meaning and impact. Furthermore, we investigate policymakers’ research engagement and use this to inform public health policy in the public sector in Denmark. We collect and report data using Seeking, Engaging with, and Evaluation Research (SEER) methodology. Policymakers are reported to have research capacity, but it is questionable how those competences have actually been used in policymaking. Decision-makers were often not aware or did not know about the existing organizational tools and systems for research engagement and use and two third of respondents had not been part of any research activities or had any collaboration with researchers. Overall, research use in public health policymaking and evaluation was limited. As a conclusion, we propose that capacity-building interventions for increasing research use and collaboration in EIPM should be context-oriented, measurable, and sustainable in developing individual and organizational competences.

Using existing systematic reviews for developing vaccination recommendations: Results of an international expert workshop

National immunization technical advisory groups (NITAGs) develop immunization-related recommendations. Systematic reviews are recommended to be used in this process, but conducting them requires significant resources, which many NITAGs lack. Using existing systematic reviews could help address this problem. The Robert Koch Institute and collaborators set up the SYSVAC2 project to facilitate the retrieval of existing systematic reviews and offer guidance on using them. This will include an online registry of systematic reviews relevant to immunization policy and an online course on how to use existing reviews. This report describes an international expert workshop held in December 2019 to develop consensus on methods for using existing reviews and other relevant factors for the registry and course. Members from NITAGs representing different regions of the world presented their experiences of using systematic reviews and reflected on challenges inhibiting use. Three methodologists considered different aspects of using systematic reviews. Interactive sessions followed, where implications for SYSVAC2 were discussed. Participants supported having critical appraisal ratings, plain language summaries, keyword search, and data visualization functions in the registry. They suggested tailoring course content to different audiences and including overviews of reviews as a topic and examples of how NITAGs have used or could use existing reviews. Participants agreed that whether a review is out-of-date should be decided by those using the review rather than registry staff. The registry could help by highlighting the date of literature search or included primary studies. Participants recommended a visualization function to highlight overlap across reviews and guidance on handling challenges to using reviews, ideally, involving a practical element. No consensus was reached on which critical appraisal tool to use for reviews in the registry, but a majority of participants wanted registry staff to perform appraisals. Formative research is planned before the registry and online course are launched in 2020.

From premise to practice: Applicability of a consensus statement for supporting adults with IDD who are frail

BACKGROUND

Study of frailty in adults with intellectual and developmental disabilities (IDD) is relatively new. To build the body of literature, an international consensus statement on how to support adults with IDD as they become frail was developed based on fictional cases. This study examined the face validity and applicability of the consensus statement to real-world care planning.

METHOD

Twenty-three care plans for adults with IDD who were classified along the frailty continuum were reviewed. Documented goals, actions and outcomes were coded according to the consensus statement's principles and recommendations.

RESULTS

The recommendations 'Improvement and maintenance are viable goals' and 'Intersectoral collaboration is needed' were documented the most often. Attention to the needs of formal and informal caregivers was mentioned the least often in care plans.

CONCLUSIONS

This study provides some support for the face validity of the consensus statement and its applicability to supporting adults with IDD who are frail.

Reporting of conflicts of interest by authors of primary studies on health policy and systems research: a cross-sectional survey

OBJECTIVES

The objective of this study was to assess the frequency and types of conflict of interest (COI) disclosed by authors of primary studies of health policy and systems research (HPSR).

DESIGN

We conducted a cross-sectional survey using standard systematic review methodology for study selection and data extraction. We conducted descriptive analyses.

SETTING

We collected data from papers published in 2016 in 'health policy and service journals' category in Web of Science database.

PARTICIPANTS

We included primary studies (eg, randomised controlled trials, cohort studies, qualitative studies) of HPSR published in English in 2016 peer-reviewed health policy and services journals.

OUTCOME MEASURES

Reported COI disclosures including whether authors reported COI or not, form in which COI disclosures were provided, number of authors per paper who report any type of COI, number of authors per paper who report specific types and subtypes of COI.

RESULTS

We included 200 eligible primary studies of which 132 (66%) included COI disclosure statements of authors. Of the 132 studies, 19 (14%) had at least one author reporting at least one type of COI and the most frequently reported type was individual financial COI (n=15, 11%). None of the authors reported individual intellectual COIs or personal COIs. Financial and individual COIs were reported more frequently compared with non-financial and institutional COIs.

CONCLUSION

A low percentage of HPSR primary studies included authors reporting COI. Non-financial or institutional COIs were the least reported types of COI.

Policy Debates Regarding Nicotine Vaping Products in Australia: A Qualitative Analysis of Submissions to a Government Inquiry from Health and Medical Organisations

Australia has maintained a highly restrictive regulatory framework for nicotine vaping products (NVPs) and the regulatory approach differs from most other high income countries. This paper employed a thematic analysis to assess policy consultation submissions made to a government inquiry regarding use and marketing of NVPs. We included in the analysis submissions (n = 40) made by Australian institutions that influence or contribute to health policy-making including government agencies, health bodies and charities (n = 23), and public health academics and healthcare professionals (n = 18). Submissions from commercial entities and consumers were excluded. The majority of submissions from representatives of government agencies, health bodies and charities recommended maintaining current restrictions on NVPs. Arguments against widening access to NVPs included the demand for long-term evidence on safety and efficacy of an unusually high standard. There was widespread support for restrictions on sales, advertising and promotion, with most submissions supporting similar controls as for tobacco products. In contrast, the majority of individual submissions from healthcare professionals and public health academics advocated for widening access to NVPs for smokers and emphasized the potential benefits of smokers switching to vaping and the policy incoherence of regulating less harmful nicotine products more strictly than tobacco cigarettes. Progress in resolving the policy debate concerning NVP regulation in Australia will require policy makers, clinicians and the public health community to engage in a meaningful dialogue which gives due consideration to both intended and unintended consequences of proposed policies.

What should governments be doing to prevent diabetes throughout the life course?

Health systems and governments are increasingly required to implement measures that target at-risk populations to prevent noncommunicable diseases. In this review we lay out what governments should be doing to prevent diabetes throughout the life course. The following four target groups were used to structure the specific recommendations: (1) pregnant women and young families, (2) children and adolescents, (3) working age population, and (4) the elderly. The evidence to date supports the effectiveness of some known government policy measures, such as sugar taxes and regulatory measures in the (pre-)school setting for children and adolescents. Many of these appear to be more effective if they are part of a bundle of strategies and if they are supplemented by communication strategies. Although there is a current focus on strategies that target the individual, governments can make use of evidence-based population-level prevention strategies. More research and continuous evaluation of the overall and subgroup-specific effectiveness of policy strategies using high-quality longitudinal studies are needed.

Evaluation of an HIV-specific rapid response service for community-based organisations in Ontario, Canada

Background

To support AIDS service organisations and other community-based organisations’ use of research evidence to inform HIV-related programmes, services and policies, the Ontario HIV Treatment Network (OHTN) developed a Rapid Response Service. The final product of the rapid response process at the OHTN, which is more streamlined than that of traditional systematic reviews, consists of a detailed report answering questions regarding an HIV-specific issue and how the findings apply within the local context. In 2016, the OHTN conducted an evaluation to assess the effectiveness of its Rapid Response Service. This article reports on the development of this service as well as the results of the evaluation.

Methods

All rapid responses published between January 1, 2009, and September 30, 2016, by the OHTN (n = 102) were analysed using univariate analyses. Frequency distributions were determined for the following variables for each rapid response: populations observed, topics covered, requestor affiliations and number of downloads from the OHTN’s website. Requestors of rapid responses were also interviewed regarding perceived helpfulness and utility of the service and final products, and suggestions for changes to the service. Six-month follow-up interviews were conducted to determine how affiliated organisations used the evidence from the rapid response they requested.

Results

The 102 rapid responses published covered 14 different populations of interest. Topics covered included the HIV prevention, engagement and care cascade, determinants of health, syndemics, and comorbidities. Requestor affiliations consisted of AIDS service organisations, government agencies and policy-makers, non-HIV-focused community-based organisations, and hospitals, universities or health centres. Requestors perceived most aspects of the Rapid Response Service as very helpful and most frequently suggested that the rapid responses should provide recommendations. Follow-up interviews regarding the impact of rapid responses show that rapid responses have been used to assist organisations in numerous activities.

Conclusions

Organisations that have used the OHTN’s Rapid Response Service describe it as a valuable service useful for the development of programmes and policies. Improvements in capacity-building efforts may increase its utility. Describing the findings of this evaluation may serve as a reference for similar programmes to increase the use of research evidence among public health decision-makers.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0476-4) contains supplementary material, which is available to authorized users.

Enjeux de la planification publique participative d'un Plan régional de promotion de la santé dans le contexte institutionnel belge

Définir un cadre de prévention et de promotion de la santé est un enjeu majeur par la Région wallonne qui a hérité depuis 2014 de ces compétences. Une première partie d'un Plan Prévention et Promotion de la Santé a été élaborée en 2017, permettant de définir les priorités régionales en santé. L'objectif de cet article est de présenter le processus de construction pour ses composantes opérationnelles et de discuter des enjeux de l'implémentation d'un tel cadre d'action sur un plan politique et de démarche participative. En prenant appui sur la première partie du Plan, l'administration régionale a identifié 11 groupes de travail thématiques qui ont rassemblé plus de 150 partenaires. Un guide méthodologique développe les chaines « objectifs de santé-objectifs spécifiques-actions » désirées, le cadre de priorisation des actions basé sur des critères de pertinence, de cohérence et de faisabilité ainsi que sur la prise en compte d'objectifs plus transversaux. Le Plan opérationnel repose sur des valeurs et des principes qui constituent les fondements des pratiques des acteurs. La proposition de programmation est structurée en 6 axes qui comportent un complément au diagnostic de situation, des principes d'intervention et une programmation opérationnelle. Il a aussi été élaboré un cadre d'implémentation, de gestion et de suivi, d'évaluation et de mise à jour du Plan. Le temps politique n'est pas le même que le temps nécessaire à une planification de qualité. Ce travail de programmation, basé sur des dimensions participatives, a été incomplet de par les limites du processus qui ont influencé les contenus. Bien que les propositions soient en majorité issues d'actions existantes, il faut souligner que les participants ont vu dans le Plan l'opportunité d'innover et de pallier certains manques ressentis actuellement en Wallonie en termes de prévention et de promotion de la santé.

A roadmap for strengthening evidence-informed health policy-making in Iran: protocol for a research programme

BACKGROUND

Many initiatives have been taken in the Islamic Republic of Iran to promote evidence-informed health policy-making (EIHP). However, these initiatives are not systematic. Since the implementation of EIHP is not consistent and the interventions in this regard are complex, a comprehensive plan could be a useful tool for employing initiatives to achieve and promote EIHP. Hence, this study aims to develop a roadmap for strengthening EIHP over a 3-year period in Iran.

METHODS

Nine projects will be conducted to define the roadmap for strengthening EIHP. These projects include two reviews and a stakeholder analysis to identify the factors that facilitate or hinder achieving EIHP. The next study will be a qualitative study to prioritise the challenges and outline the main causes. The following steps will be a review of reviews to extract global experiences on interventions used for strengthening EIHP and two qualitative studies to examine the adoption of these interventions and develop an operational plan for strengthening EIHP in Iran. The research will be completed through conducting two qualitative-quantitative studies to design a tool for measuring EIHP and assessing EIHP in Iran at baseline.

DISCUSSION

This national EIHP roadmap will surely be able to identify the gaps and bumps that might exist in the implementation plan for establishing EIHP and eliminate them as needed in the future. This roadmap can be a step in moving towards transparency and accountability in the health system and as thus towards good governance and improvement of the health system's performance. Although the plan can be a good model for developing countries and may promote the use of evidence in health policy-making, we should assume that there are some critical contextual factors that could potentially hinder the complete and successful implementation of EIHP. Thus, to enhance EIHP in these countries with a policy-making context that does not fully support the use of evidence, it is crucial to think about not only those interventions that directly address the EIHP barriers, but also some long-term strategies to make required changes in the context, both beyond and within the health system.

Fostering access to and use of contextualised knowledge to support health policy-making: lessons from the Policy Information Platform in Nigeria

Background

Contextualising evidence to inform policy-making is increasingly recognised as key to developing and implementing effective health policies. Creating a one-stop shop for evidence is an approach that can facilitate timely access to the best evidence to inform policy decisions. We report outcomes after implementation of the Policy Information Platform (PIP), a pilot one-stop evidence repository in Nigeria designed to alleviate barriers to accessing policy-relevant knowledge.

Methods

This cross-sectional study involved five phases, namely (1) consultation with Nigerian policy-makers to identify priority policy issues, areas of health policy information needs, and challenges and capacity constraints in accessing evidence for policy-making; (2) a stakeholder engagement workshop to formally launch the PIP; (3) extraction of data and other information from scientific articles, policy briefs, evaluation reports, grey literature and health policy documents relevant to policy-making in Nigeria (identified by Google and PubMed searches and by examination of websites of relevant Nigerian government ministries, agencies and parastatals), for use in developing the PIP website; (4) promotion of the PIP in national and state health policy meetings; and (5) evaluation of the PIP using a stakeholder survey questionnaire distributed via email and critical appraisal of the grey literature included in the PIP using the authority, accuracy, coverage, objectivity, date and significance (AACODS) checklist.

Results

Priority policy areas identified by policy-makers were disease control and prevention, population health issues and health administration. Challenges identified by policy-makers were a lack of adequate capacity to access policy-relevant evidence and transform the evidence into policy. Policy-makers suggested using systematic reviews, policy briefs and rapid response mechanisms and involving policy-makers in research as ways of increasing evidence uptake for policy. A total of 126 policy-relevant, peer-reviewed scientific articles, 85 health policy documents and 201 policy-relevant grey literature documents were selected for inclusion in the PIP. Of the 195 individuals contacted via email to evaluate the PIP, 31 (15.9%) provided a response. Respondents noted that the PIP facilitated access to information based on local evidence and context-sensitive data. Barriers identified included lack of knowledge about the PIP and limited capacity of end-users to use the data compiled in the platform.

Conclusion

An easily accessible one-stop shop of policy-relevant evidence can considerably improve policy-makers’ access to evidence for use in policy-making and practice.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0431-4) contains supplementary material, which is available to authorized users.

Lack of transparency in reporting narrative synthesis of quantitative data: a methodological assessment of systematic reviews

OBJECTIVE

To assess the adequacy of reporting and conduct of narrative synthesis of quantitative data (NS) in reviews evaluating the effectiveness of public health interventions.

STUDY DESIGN AND SETTING

A retrospective comparison of a 20% (n = 474/2,372) random sample of public health systematic reviews from the McMaster Health Evidence database (January 2010-October 2015) to establish the proportion of reviews using NS. From those reviews using NS, 30% (n = 75/251) were randomly selected and data were extracted for detailed assessment of: reporting NS methods, management and investigation of heterogeneity, transparency of data presentation, and assessment of robustness of the synthesis.

RESULTS

Most reviews used NS (56%, n = 251/446); meta-analysis was the primary method of synthesis for 44%. In the detailed assessment of NS, 95% (n = 71/75) did not describe NS methods; 43% (n = 32) did not provide transparent links between the synthesis data and the synthesis reported in the text; of 14 reviews that identified heterogeneity in direction of effect, only one investigated the heterogeneity; and 36% (n = 27) did not reflect on limitations of the synthesis.

CONCLUSION

NS methods are rarely reported in systematic reviews of public health interventions and many NS reviews lack transparency in how the data are presented and the conclusions are reached. This threatens the validity of much of the evidence synthesis used to support public health. Improved guidance on reporting and conduct of NS will contribute to improved utility of NS systematic reviews.

Subjective judgements - no more, no less? A response to Malterud, Bjelland and Elvbakken

In 2016, three researchers published a report where they concluded that systematic reviews from my institution, the Norwegian Knowledge Centre for the Health Services, are of little use for health policy decision-making. Based on their reading of the conclusion section in 14 reports, they argue that our systematic reviews are not useful due to their lack of clear and conclusive findings. I have reviewed the same documents and I beg to differ. Unfortunately, the description of their methodological approach is both sparse and difficult to grasp, making it very hard to understand how they arrived at their conclusions.

A content analysis of popular media reporting regarding increases in minimum ages of legal access for tobacco

BACKGROUND

In the late 20th century, US localities began increasing the minimum age of legal access (MLA) for tobacco from 18 to 21 years by enacting "Tobacco 21" ordinances. Although these policies have a strong evidence base and broad popular support, popular media coverage of tobacco control laws has not always been accurate. This study sought to determine if contemporaneous popular media reporting accurately reflected the scientific findings regarding increased tobacco MLAs.

METHODS

We searched LexisNexis for popular media reports that (1) addressed proposed or enacted Tobacco 21 ordinances and were (2) published in English, (3) drawn from a US news source, and (4) written after January 2004. We conducted a content analysis for quality based on a validated measure of accuracy of reporting, the Index of Scientific Quality (ISQ), which allows assessment of articles by assigning scores ranging from 1 (lowest) to 5 (highest).

RESULTS

Searches yielded 378 articles; after screening for relevance and duplicates, 98 were included in the review. All studies identified through the keyword searches addressed Tobacco 21 policies. The average global score identifying the scientific quality of the articles was 2.98 of 5. Over three-quarters of the popular media articles addressing Tobacco 21 laws were written after a systematic review of these policies was released by the Institute of Medicine and approximately 4 in 10 cited findings from that review.

CONCLUSIONS

Popular media reports on Tobacco 21 laws demonstrated average overall quality and relied on both anecdotal and scientific evidence, in contrast to previous studies found that popular media reports on tobacco issues demonstrated low overall quality and relied primarily on anecdotal evidence. The systematic review of increased MLAs for tobacco written by the Institute of Medicine diffused quickly into popular reporting, suggesting that this type of evidence might improve research translation.

Assessing the influence of knowledge translation platforms on health system policy processes to achieve the health millennium development goals in Cameroon and Uganda: a comparative case study

There is a scarcity of empirical data on the influence of initiatives supporting evidence-informed health system policy-making (EIHSP), such as the knowledge translation platforms (KTPs) operating in Africa. To assess whether and how two KTPs housed in government-affiliated institutions in Cameroon and Uganda have influenced: (1) health system policy-making processes and decisions aiming at supporting achievement of the health millennium development goals (MDGs); and (2) the general climate for EIHSP. We conducted an embedded comparative case study of four policy processes in which Evidence Informed Policy Network (EVIPNet) Cameroon and Regional East African Community Health Policy Initiative (REACH-PI) Uganda were involved between 2009 and 2011. We combined a documentary review and semi structured interviews of 54 stakeholders. A framework-guided thematic analysis, inspired by scholarship in health policy analysis and knowledge utilization was used. EVIPNet Cameroon and REACH-PI Uganda have had direct influence on health system policy decisions. The coproduction of evidence briefs combined with tacit knowledge gathered during inclusive evidence-informed stakeholder dialogues helped to reframe health system problems, unveil sources of conflicts, open grounds for consensus and align viable and affordable options for achieving the health MDGs thus leading to decisions. New policy issue networks have emerged. The KTPs indirectly influenced health policy processes by changing how interests interact with one another and by introducing safe-harbour deliberations and intersected with contextual ideational factors by improving access to policy-relevant evidence. KTPs were perceived as change agents with positive impact on the understanding, acceptance and adoption of EIHSP because of their complementary work in relation to capacity building, rapid evidence syntheses and clearinghouse of policy-relevant evidence. This embedded case study illustrates how two KTPs influenced policy decisions through pathways involving policy issue networks, interest groups interaction and evidence-supported ideas and how they influenced the general climate for EIHSP.

Development and pretesting of "Becoming Breastfeeding Friendly": Empowering governments for global scaling up of breastfeeding programmes

Scaling up breastfeeding programmes has not been highly prioritized despite overwhelming evidence that breastfeeding benefits the health of mothers and children. Lack of evidence-based tools for scaling up may deter countries from prioritizing breastfeeding. To fill this gap, Becoming Breastfeeding Friendly (BBF) was developed to guide countries in effectively scaling up programmes to protect, promote, and support breastfeeding. BBF includes an evidence-based toolbox that consists of a BBF Index, case studies, and a 5-meeting process. These three interrelated components enable countries to assess their breastfeeding scaling up environment, identify gaps, propose policy recommendations, develop a scaling up plan, and track progress. The toolbox was developed based on current evidence and expert guidance from a Technical Advisory Group, which was composed of global breastfeeding and metric experts with experience in the scaling up of health and nutrition programmes in low-, middle-, and high-income countries. The BBF toolbox required a step-by-step iterative approach to describe and systematize each component, thus an operational manual was developed. The BBF toolbox and BBF operational manual underwent intensive pretesting in two countries, Ghana and Mexico, resulting in the modification of each component plus the operational manual. Pretesting continues in six additional countries demonstrating that BBF is a robust and dynamic multi-sectoral process that, with relatively minor adaptations, can be successfully implemented in countries across world regions.

Do evidence summaries increase health policy-makers' use of evidence from systematic reviews? A systematic review

This review summarizes the evidence from six randomized controlled trials that judged the effectiveness of systematic review summaries on policymakers' decision making, or the most effective ways to present evidence summaries to increase policymakers' use of the evidence. This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one. The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary. None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?" Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.

Plain language summary

Policy briefs make systematic reviews easier to understand but little evidence of impact on use of study findings: It is likely that evidence summaries are easier to understand than complete systematic reviews. Whether these summaries increase the use of evidence from systematic reviews in policymaking is not clear.What is this review about?: Systematic reviews are long and technical documents that may be hard for policymakers to use when making decisions. Evidence summaries are short documents that describe research findings in systematic reviews. These summaries may simplify the use of systematic reviews.Other names for evidence reviews are policy briefs, evidence briefs, summaries of findings, or plain language summaries. The goal of this review was to learn whether evidence summaries help policymakers use evidence from systematic reviews. This review also aimed to identify the best ways to present the evidence summary to increase the use of evidence.What are the main findings of this review?: This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one.The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary.None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?"Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read.. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.What do the findings of this review mean?: Our review suggests that evidence summaries help policymakers to better understand the findings presented in systematic reviews. In short, evidence summaries should be developed to make it easier for policymakers to understand the evidence presented in systematic reviews. However, right now there is very little evidence on the best way to present systematic review evidence to policymakers.How up to date is this review?: The authors of this review searched for studies through June 2016.

Executive summary/Abstract

Background: Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. Strategies to promote the use of evidence to decision makers are required, and evidence summaries have been suggested as a facilitator. Evidence summaries include policy briefs, briefing papers, briefing notes, evidence briefs, abstracts, summary of findings tables, and plain language summaries. There are many organizations developing and disseminating systematic review evidence summaries for different populations or subsets of decision makers. However, evidence on the usefulness and effectiveness of systematic review summaries is lacking. We present an overview of the available evidence on systematic review evidence summaries.Objectives: This systematic review aimed to 1) assess the effectiveness of evidence summaries on policy-makers' use of the evidence and 2) identify the most effective summary components for increasing policy-makers' use of the evidence.Search methods: We searched several online databases (Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Global Health Library, Popline, Africa-wide, Public Affairs Information Services, Worldwide Political Science Abstracts, Web of Science, and DfiD), websites of research groups and organizations which produce evidence summaries, and reference lists of included summaries and related systematic reviews. These databases were searched in March-April, 2016.Selection criteria: Eligible studies included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) studies, and interrupted time series (ITS) studies. We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of "evidence summary", "policy brief", or other product derived from systematic reviews that presented evidence in a summarized form. These interventions could be compared to active comparators (e.g. other summary formats) or no intervention.The primary outcomes were: 1) use of systematic review summaries decision-making (e.g. self-reported use of the evidence in policy-making, decision-making) and 2) policymaker understanding, knowledge, and/or beliefs (e.g. changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g. format) of the summaries.Results: Our database search combined with our grey literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text and we included 6 studies (reported in 7 papers, 1661 participants) as well as protocols from 2 ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (4 studies) and perceived usefulness or usability (3 studies). Summary of Findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for Summary of Findings tables, certain elements, such as reporting study event rates and absolute differences were preferred as well as avoiding the use of footnotes. No studies assessed adverse effects. The risks of bias in these studies were mainly assessed as unclear or low however, two studies were assessed as high risk of bias for incomplete outcome data due to very high rates of attrition.Authors' conclusions: Evidence summaries may be easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Assessing the applicability of public health intervention evaluations from one setting to another: a methodological study of the usability and usefulness of assessment tools and frameworks

BACKGROUND

Public health interventions can be complicated, complex and context dependent, making the assessment of applicability challenging. Nevertheless, for them to be of use beyond the original study setting, they need to be generalisable to other settings and, crucially, research users need to be able to identify to which contexts it may be applicable. There are many tools with set criteria for assessing generalisability/applicability, yet few seem to be widely used and there is no consensus on which should be used, or when. This methodological study aimed to test these tools to assess how easy they were to use and how useful they appeared to be.

METHODS

We identified tools from an existing review and an update of its search. References were screened on pre-specified criteria. Included tools were tested by using them to assess the applicability of a Swedish weight management intervention to the English context. Researcher assessments and reflections on the usability and utility of the tools were gathered using a standard pro-forma.

RESULTS

Eleven tools were included. Their length, content, style and time required to complete varied. No tool was considered ideal for assessing applicability. Their limitations included unrealistic criteria (requiring unavailable information), a focus on implementation to the neglect of transferability (i.e. little focus on potential effectiveness in the new setting), overly broad criteria (associated with low reliability), and a lack of an explicit focus on how interventions worked (i.e. their mechanisms of action).

CONCLUSION

Tools presenting criteria ready to be used may not be the best method for applicability assessments. They are likely to be either too long or incomplete, too focused on differences and fail to address elements that matter for the specific topic of interest. It is time to progress from developing lists of set criteria that are not widely used in the literature, to creating a new approach to applicability assessment. Focusing on mechanisms of action, rather than solely on characteristics, could be a useful approach, and one that remains underutilised in current tools. New approaches to assessing generalisability that evolve away from checklist style assessments need to be developed, tested, reported and discussed.

Improving Conduct and Reporting of Narrative Synthesis of Quantitative Data (ICONS-Quant): protocol for a mixed methods study to develop a reporting guideline

Introduction

Reliable evidence syntheses, based on rigorous systematic reviews, provide essential support for evidence-informed clinical practice and health policy. Systematic reviews should use reproducible and transparent methods to draw conclusions from the available body of evidence. Narrative synthesis of quantitative data (NS) is a method commonly used in systematic reviews where it may not be appropriate, or possible, to meta-analyse estimates of intervention effects. A common criticism of NS is that it is opaque and subject to author interpretation, casting doubt on the trustworthiness of a review’s conclusions. Despite published guidance funded by the UK’s Economic and Social Research Council on the conduct of NS, recent work suggests that this guidance is rarely used and many review authors appear to be unclear about best practice. To improve the way that NS is conducted and reported, we are developing a reporting guideline for NS of quantitative data.

Methods

We will assess how NS is implemented and reported in Cochrane systematic reviews and the findings will inform the creation of a Delphi consensus exercise by an expert panel. We will use this Delphi survey to develop a checklist for reporting standards for NS. This will be accompanied by supplementary guidance on the conduct and reporting of NS, as well as an online training resource.

Ethics and dissemination

Ethical approval for the Delphi survey was obtained from the University of Glasgow in December 2017 (reference 400170060). Dissemination of the results of this study will be through peer-reviewed publications, and national and international conferences.

Patterns and Gaps Identified in a Systematic Review of the Hepatitis C Virus Care Continuum in Studies among People Who Use Drugs

Introduction

Systematic reviews are useful for synthesizing data on various health conditions and for identifying gaps in available data. In the US, the main risk group for hepatitis C virus (HCV) infection is people who use drugs (PWUD); as a group, PWUD have the highest prevalence of chronic HCV. While the care continuum construct has been increasingly applied to studies of HCV care among PWUD, what constitutes the steps in an HCV care continuum is not standardized. We sought to examine the range of HCV care continuum outcomes that studies reported on, to identify gaps in the literature, and to develop strategies that allowed for valuable syntheses of care continuum data.

Methods

We conducted searches of electronic databases for published literature. Reports were eligible if they provided original data from 1990 to 2016 from the US, presented data on one or more HCV care continuum outcomes, and provided outcome data on PWUD as a distinct group.

Results

A total of 313 full-text reports were assessed for eligibility. Of 212 potentially eligible reports, 32 (15.1%) did not present outcomes for PWUD separately from those who were non-PWUD. Among 101 eligible reports, a total of 166 care continuum outcomes were extracted; outcomes could be grouped into three categories that represent the HCV care continuum: testing (39.8%, n = 66/166); linkage to care (16.9%, n = 28/166); and treatment (43.4%, n = 72/166). Seventy-four reports (73.3%, n = 74/101) presented data on only one step. Linkage to care occurred variably after only antibody, or after antibody and viral load (VL) testing. Six (5.9%, n = 6/101) reports presented data on all three steps.

Conclusion

Reports examined a variety of HCV care continuum outcomes that could be grouped into the three steps of testing, linkage to care, and treatment. The application of this care continuum model would facilitate subsequent data synthesis for program comparison and public health evaluation. Given the two-step nature of HCV testing, analyses also need to account for variation in whether linkage to care occurred after antibody testing or after sequential antibody and VL testing. Additional data are needed on the progression of PWUD through the entire care continuum.

The many meanings of evidence: a comparative analysis of the forms and roles of evidence within three health policy processes in Cambodia

BACKGROUND

Discussions within the health community routinely emphasise the importance of evidence in informing policy formulation and implementation. Much of the support for the evidence-based policy movement draws from concern that policy decisions are often based on inadequate engagement with high-quality evidence. In many such discussions, evidence is treated as differing only in quality, and assumed to improve decisions if it can only be used more. In contrast, political science scholars have described this as an overly simplistic view of the policy-making process, noting that research 'use' can mean a variety of things and relies on nuanced aspects of political systems. An approach more in recognition of how policy-making systems operate in practice can be to consider how institutions and ideas influence which pieces of evidence appear to be relevant for, and are used within, different policy processes.

METHODS

Drawing on in-depth interviews undertaken in 2015-2016 with key health sector stakeholders in Cambodia, we investigate the evidence perceived to be relevant to policy decisions for three contrasting health policy examples, namely tobacco control, HIV/AIDS and performance-based salary incentives. These cases allow us to examine the ways that policy-relevant evidence may differ given the framing of the issue and the broader institutional context in which evidence is considered.

RESULTS

The three health issues show few similarities in how pieces of evidence were used in various aspects of policy-making, despite all being discussed within a broad policy environment in which evidence-based policy-making is rhetorically championed. Instead, we find that evidence use can be better understood by mapping how these health policy issues differ in terms of the issue characteristics, and also in terms of the stakeholders structurally established as having a dominant influence for each issue. Both of these have important implications for evidence use. Contrasting concerns of key stakeholders meant that evidence related to differing issues could be understood in terms of how it was relevant to policy. The stakeholders involved, however, could further be seen to possess differing logics about how to go about achieving their various outcomes - logics that could further help explain the differences seen in evidence utilisation.

CONCLUSION

A comparative approach reiterates that evidence is not a uniform concept for which more is obviously better, but rather illustrates how different constructions and pieces of evidence become relevant in relation to the features of specific health policy decisions. An institutional approach that considers the structural position of stakeholders with differing core goals or objectives, as well as their logics related to evidence utilisation, can further help to understand some of the complexities of evidence use in health policy-making.

Knowledge mobilisation for policy development: implementing systems approaches through participatory dynamic simulation modelling

BACKGROUND

Evidence-based decision-making is an important foundation for health policy and service planning decisions, yet there remain challenges in ensuring that the many forms of available evidence are considered when decisions are being made. Mobilising knowledge for policy and practice is an emergent process, and one that is highly relational, often messy and profoundly context dependent. Systems approaches, such as dynamic simulation modelling can be used to examine both complex health issues and the context in which they are embedded, and to develop decision support tools.

OBJECTIVE

This paper reports on the novel use of participatory simulation modelling as a knowledge mobilisation tool in Australian real-world policy settings. We describe how this approach combined systems science methodology and some of the core elements of knowledge mobilisation best practice. We describe the strategies adopted in three case studies to address both technical and socio-political issues, and compile the experiential lessons derived. Finally, we consider the implications of these knowledge mobilisation case studies and provide evidence for the feasibility of this approach in policy development settings.

CONCLUSION

Participatory dynamic simulation modelling builds on contemporary knowledge mobilisation approaches for health stakeholders to collaborate and explore policy and health service scenarios for priority public health topics. The participatory methods place the decision-maker at the centre of the process and embed deliberative methods and co-production of knowledge. The simulation models function as health policy and programme dynamic decision support tools that integrate diverse forms of evidence, including research evidence, expert knowledge and localised contextual information. Further research is underway to determine the impact of these methods on health service decision-making.

Factors Associated With Unhealthy Snacks Consumption Among Adolescents in Iran's Schools

Background: Well-informed interventions are needed if school-based health promotion is to be effective. Among other aims, the Iranian Health Promoting School (IHPS) program that was launched in 2011, has an important aim of promoting dietary behaviors of adolescents. The present study, therefore, aimed to investigate the factors affecting unhealthy snacking of adolescents and provide evidence for a more effective IHPS program.

Methods: In a cross-sectional study design, 1320 students from 40 schools in Kerman city were selected using a proportional stratified random sampling method. A modified qualitative Food Frequency Questionnaire (FFQ) was used to gather data about unhealthy snacking behavior. Data about intrapersonal and environmental factors were obtained using a validated and reliable questionnaire. A mixed-effects negative-binomial regression model was used to analyze the data.

Results: Taste and sensory perception (prevalence rate ratio [PRR]=1.18; 95% CI: 1.09-1.27), being a male (PRR=1.20; 95% CI: 1.05-1.38) and lower nutritional knowledge (PRR=0.96; 95% CI: 0.91-0.99) were associated with higher weekly unhealthy snaking. Perceived self-efficacy (PRR=0.95; 95% CI: 0.91-1.00) negatively influenced the frequency of unhealthy snaking, with this approaching significance (P<.06). In case of environmental factors, high socio-economic status (SES) level (PRR=1.45; 95% CI: 1.26-1.67), single-parent family (PRR=1.14; 95% CI: 1.01-1.30), more social norms pressure (PRR=1.08; 95% CI: 1.01-1.17), pocket money allowance (PRR=1.21; 95% CI: 1.09-1.34), easy accessibility (PRR=1.06; 95% CI:1.01-1.11), and less perceived parental control (PRR=0.96; 95% CI: 0.92-0.99) all had a role in higher consumption of unhealthy snacks. Interestingly, larger school size was associated with less unhealthy snacking (PRR=0.79; 95% CI: 0.68-0.92).

Conclusion: Unhealthy snacking behavior is influenced by individual, socio-cultural and physical-environmental influences, namely by factors relating to poor parenting practices, high SES level, family characteristics, improper social norms pressure, and less knowledge and self-efficacy of students. This evidence can be used to inform a more evidencebased IHPS program through focusing on supportive strategies at the home, school, and local community levels.

A Qualitative Study of Challenges and Opportunities in Mobilizing Research Knowledge on Violence Against Women

Background Effective delivery of interventions by health and social services requires research-based knowledge which identifies the causes and consequences of violence against women. Methods to effectively share new knowledge with violence against women decision-makers remain under studied. Purpose This paper examines how new research-based knowledge-namely, the lack of efficacy of health-care screening for exposure to intimate partner violence against women-is received by stakeholders in the violence against women field. Methods Data from 10 stakeholder group discussions ( N = 86) conducted during a knowledge-sharing forum were analyzed to assess how stakeholders responded to the new knowledge. Results Participant reactions ranged from full acceptance to significant resistance to the research findings. We suggest themes that help explain these reactions, including the context and content of our findings and their epistemological match to participants' experiences and beliefs, and the perceived value of research evidence, compared to other forms of knowledge. Conclusions Violence against women is a complex psycho-social phenomenon, and people with an interest in this field bring diverse and even conflicting perspectives regarding its causes, consequences, and potential solutions.

A Qualitative Assessment of the Evidence Utilization for Health Policy-Making on the Basis of SUPPORT Tools in a Developing Country

Background: SUPPORT tools consist of 18 articles addressing the health policy-makers so that they can learn how to make evidence-informed health policies. These tools have been particularly recommended for developing countries. The present study tries to explain the process of evidence utilization for developing policy documents in the Iranian Ministry of Health and Medical Education (MoHME) and to compare the findings with those of SUPPORT tools.

Methods: A qualitative research was conducted, using the framework analysis approach. Participants consisted of senior managers and technicians in MoHME. Purposeful sampling was done, with a maximum variety, for the selection of research participants: individuals having at least 5 years of experience in preparing evidence-based policy documents. Face-to-face interviews were conducted for data collection. As a guideline for the interviews, ‘the Utilization of Evidence in Policy-Making Organizations’ procedure was used. The data were analyzed through the analysis of the framework method using MAXQDA 10 software.

Results: The participants acquired the research evidence in a topic-based form, and they were less likely to search on the basis of the evidence pyramid. To assess the quality of evidence, they did not use standard critical tools; to adapt the evidence and interventions with the local setting, they did not use the ideas and experiences of all stakeholders, and in preparing the evidence-based policy documents, they did not take into consideration the window of opportunity, did not refrain from using highly technical terms, did not write user-friendly summaries, and did not present alternative policy options. In order to develop health policies, however, they used the following innovations: attention to the financial burden of policy issues on the agenda, sensitivity analysis of the preferred policy option on the basis of technical, sociopolitical, and economic feasibility, advocacy from other scholars, using the multi-criteria decision-making models for the prioritization of policy options, implementation of policy based on the degree of readiness of policy-implementing units, and the classification of policy documents on the basis of different conditions of policy-making (urgent, short-term, and long-term).

Conclusion: Findings showed that the process of evidence utilization in IR-MoH enjoys some innovations for the support of health policy development. The present study provides IR-MoH with considerable opportunities for the improvement of evidence-informed health policy-making. Moreover, the SUPPORT process and tools are recommended to be used in developing countries.

An Assessment of National Maternal and Child Health Policy-Makers' Knowledge and Capacity for Evidence- Informed Policy-Making in Nigeria

BACKGROUND

There is increasing interest globally in the use of more rigorous processes to ensure that maternal, newborn, and child health (MNCH) care recommendations are informed by the best available research evidence use. The purpose of this study was to engage Nigerian MNCH policy-makers and other stakeholders to consider issues around research to policy and practice interface and to assess their existing knowledge and capacity on the use of research evidence for policy-making and practice.

METHODS

The study design is a cross-sectional evaluation of MNCH stakeholders' knowledge as it pertains different dimensions of research to practice. This was undertaken during a national MNCH stakeholders' engagement event convened under the auspices of the West African Health Organization (WAHO) and the Federal Ministry of Health (FMoH) in Abuja, Nigeria. A questionnaire was administered to participants, which was designed to assess participants' knowledge, capacity and organizational process of generation, synthesis and utilization of research evidence in policy-making regarding MNCH.

RESULTS

A total of 40 participants signed the informed consent form and completed the questionnaire. The mean ratings (MNRs) of participants' knowledge of electronic databases and capacity to identify and obtain relevant research evidence from electronic databases ranged from 3.62-3.68 on the scale of 5. The MNRs of participants' level of understanding of a policy brief, a policy dialogue and the role of researchers in policy-making ranged from 3.50-3.86. The MNRs of participants' level of understanding of evidence in policy-making context, types and sources of evidence, capacity to identify, select, adapt, and transform relevant evidence into policy ranged from 3.63-4.08. The MNRs of the participants' organization's capacity to cover their geographical areas of operation were generally low ranging from 3.32-3.38 in terms of manpower, logistics, facilities, and external support. The lowest MNR of 2.66 was recorded in funding.

CONCLUSION

The outcomes of this study suggest that a stakeholders' engagement event can serve as an important platform to assess policy-makers' knowledge and capacity for evidence-informed policy-making and for the promotion of evidence use in the policy process.

Interpretation and use of evidence in state policymaking: a qualitative analysis

INTRODUCTION

Researchers advocating for evidence-informed policy have attempted to encourage policymakers to develop a greater understanding of research and researchers to develop a better understanding of the policymaking process. Our aim was to apply findings drawn from studies of the policymaking process, specifically the theory of policy windows, to identify strategies used to integrate evidence into policymaking and points in the policymaking process where evidence was more or less relevant.

METHODS

Our observational study relied on interviews conducted with 24 policymakers from the USA who had been trained to interpret scientific research in multiple iterations of an evidence-based workshop. Participants were asked to describe cases where they had been involved in making health policy and to provide examples in which research was used, either successfully or unsuccessfully. Interviews were transcribed, independently coded by multiple members of the study team and analysed for content using key words, concepts identified by participants and concepts arising from review of the texts.

RESULTS

Our results suggest that policymakers who focused on health issues used multiple strategies to encourage evidence-informed policymaking. The respondents used a strict definition of what constituted evidence, and relied on their experience with research to discourage the use of less rigorous research. Their experience suggested that evidence was less useful in identifying problems, encouraging political action or ensuring feasibility and more useful in developing policy alternatives.

CONCLUSIONS

Past research has suggested multiple strategies to increase the use of evidence in policymaking, including the development of rapid-response research and policy-oriented summaries of data. Our findings suggest that these strategies may be most relevant to the policymaking stream, which develops policy alternatives. In addition, we identify several strategies that policymakers and researchers can apply to encourage evidence-informed policymaking.

The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review

Background

Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which make them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision makers. This systematic review aimed to (1) assess the effectiveness of evidence summaries on policymakers’ use of the evidence and (2) identify the most effective summary components for increasing policymakers’ use of the evidence. We present an overview of the available evidence on systematic review derivative products.

Methods

We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of “evidence summary,” “policy brief,” or other products derived from systematic reviews that presented evidence in a summarized form. The primary outcomes were the (1) use of systematic review summaries in decision-making (e.g., self-reported use of the evidence in policymaking and decision-making) and (2) policymakers’ understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g., format) of the summaries.

Results

Our database search combined with our gray literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text, and we included six studies (reported in seven papers) as well as protocols from two ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (four studies), and perceived usefulness or usability (three studies). Summary of findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for summary of findings tables, certain elements, such as reporting study event rates and absolute differences, were preferred as well as avoiding the use of footnotes.

Conclusions

Evidence summaries are likely easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Trial registration

The protocol was published in the journal Systematic Reviews (2015;4:122)

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0530-3) contains supplementary material, which is available to authorized users.

The international generalisability of evidence for health policy: A cross country comparison of medication adherence following policy change

Copayments for prescriptions may increase morbidity and mortality via reductions in adherence to medications. Relevant data can inform policy to minimise such unintended effects. We explored the generalisability of evidence for copayments by comparing two international copayment polices, one in Massachusetts and one in Ireland, to assess whether effects on medication adherence were comparable. We used national prescription data for public health insurance programmes in Ireland and Medicaid data in the U.S. New users of oral anti-hypertensive, anti-hyperlipidaemic and diabetic drugs were included (total n=14,259 in U.S. and n=43,843 in Ireland). We examined changes in adherence in intervention and comparator groups in each setting using segmented linear regression with generalised estimating equations. In Massachusetts, a gradual decrease in adherence to anti-hypertensive medications of -1% per month following the policy occurred. In contrast, the response in Ireland was confined to a -2.9% decrease in adherence immediately following the policy, with no further decrease over the 8 month follow-up. Reductions in adherence to oral diabetes drugs were larger in the U.S. group in comparison to the Irish group. No difference in adherence changes between the two settings for anti-hyperlipidaemic drugs occurred. Evidence on cost-sharing for prescription medicines is not 'one size fits all'. Time since policy implementation and structural differences between health systems may influence the differential impact of copayment policies in international settings.

Examining the use of health systems and policy research in the health policymaking process in Israel: views of researchers

BACKGROUND

All too often, health policy and management decisions are made without making use of or consulting with the best available research evidence, which can lead to ineffective and inefficient health systems. One of the main actors that can ensure the use of evidence to inform policymaking is researchers. The objective of this study is to explore Israeli health systems and policy researchers' views and perceptions regarding the role of health systems and policy research (HSPR) in health policymaking and the barriers and facilitators to the use of evidence in the policymaking process.

METHODS

A survey of researchers who have conducted HSPR in Israel was developed. The survey consisted of a demographics section and closed questions, which focused on support both within the researchers' organisations and the broader environment for KTE activities, perceptions on the policymaking process, and the potential influencing factors on the process. The survey was sent to all health systems and policy researchers in Israel from academic institutions, hospital settings, government agencies, the four health insurance funds, and research institutes (n = 107). All responses were analyzed using descriptive statistics. For close-ended questions about level of agreement we combined together the two highest categories (agree or strongly agree) for analysis.

RESULTS

Thirty-seven respondents participated in the survey. While many respondents felt that the use of HSPR may help raise awareness on policy issues, the majority of respondents felt that the actual use of HSPR was hindered for many reasons. While facilitators do exist to support the use of research evidence in policymaking, numerous barriers hinder the process such as challenges in government/provider relations, policymakers lacking the expertise for acquiring, assessing, and applying HSPR and priorities in the health system drawing attention away from HSPR. Furthermore, it is perceived by a majority of respondents that the health insurance funds and the physician organisations exert a strong influence in the policymaking process.

CONCLUSIONS

Health system and policy researchers in Israel need to be introduced to the benefits and potential advantages of evidence-informed policy in an organised and systematic way. Future research should examine the perceptions of policymakers in Israel and thus we can gain a broader perspective on where the actual issues lie.

Development of training for medicines-oriented policymakers to apply evidence

Background

Health systems globally promote appropriate prescribing by healthcare providers and safe and effective medicine use by consumers. Rx for Change, a publicly available database, provides access to systematic reviews regarding best practices for prescribing and using medicines. Despite the value of the database for improving prescribing and medicine use, its use remains suboptimal. This study aimed to develop a training program for five medicine-focused organisations in Canada and Australia to facilitate the use and understanding of the Rx for Change database.

Methods

Four steps were undertaken: 1) key informant interviews were completed across all organisations to understand the knowledge user perspective; 2) a directed content analysis was completed of the interview transcripts and proposed training was developed; 3) a second round of feedback on the proposed training by knowledge users was gathered; and 4) feedback was integrated to develop the final training.

Results

Sixteen key informant interviews with knowledge users were conducted. Themes for training content included the scope of, navigation and strategies for using Rx for Change (generic content) and practical examples on incorporating evidence within their workplace context (tailored content). The final training consisted of an informational video, a 60-minute face-to-face workshop and two post-training reminders.

Conclusions

A method of engaging knowledge users in the development of a training program to improve the use of an on-line database of systematic reviews was established and used to design training. Next steps include the delivery and evaluation of the training.

Views of health system policymakers on the role of research in health policymaking in Israel

Background

The use of research evidence in health policymaking is an international challenge. Health systems, including that of Israel, are usually characterized by scarce resources and the necessity to make rapid policy decisions. Knowledge transfer and exchange (KTE) has emerged as a paradigm to start bridging the “know-do” gap. The purpose of this study was to explore the views of health system policymakers and senior executives involved in the policy development process in Israel regarding the role of health systems and policy research (HSPR) in health policymaking, the barriers and facilitators to the use of evidence in the policymaking process, and suggestions for improving the use of HSPR in the policymaking process.

Methods

A survey and an interview were verbally administered in a single face-to-face meeting with health system policymakers and senior executives involved in the policy development process in Israel. The data collection period was from July to October 2014. The potential participants included members of Knesset, officials from Israel’s Ministry of Health, Ministry of Finance, health services organizations, and other stakeholder organizations (i.e., National Insurance Institute). The close-ended questions were based on previous surveys that had been conducted in this field. Interviews were tape recorded and transcribed. Descriptive statistics were conducted for close ended survey-questions and thematic analysis was conducted for open-ended interview questions.

Results

There were 32 participants in this study. Participants felt that the use of HSPR helps raise awareness on policy issues, yet the actual use of HSPR was hindered for many reasons. Facilitators do exist to support the use of HSPR in the policymaking process, such as a strong foundation of relationships between researchers and policymakers. However, many barriers exist such as the lack of relevance and timeliness of much of the currently available research to support decision-making and the paucity of funding to support research use. Suggestions to improve the use of HSPR focused on improving dissemination of research findings and ensuring that the research was more relevant and timely.

Conclusions

This research demonstrated that health systems policymakers in Israel perceive having strong relationships and collaborations with researchers however there is room for improvement, e.g. partnering in research projects to ensure relevance and use. Furthermore, health system policymakers seem to be interested in receiving relevant research in a more useable format and are open to using research in decision making.

Why do policies change? Institutions, interests, ideas and networks in three cases of policy reform

Policy researchers have used various categories of variables to explain why policies change, including those related to institutions, interests and ideas. Recent research has paid growing attention to the role of policy networks-the actors involved in policy-making, their relationships with each other, and the structure formed by those relationships-in policy reform across settings and issues; however, this literature has largely ignored the theoretical integration of networks with other policy theories, including the '3Is' of institutions, interests and ideas. This article proposes a conceptual framework integrating these variables and tests it on three cases of policy change in Burkina Faso, addressing the need for theoretical integration with networks as well as the broader aim of theory-driven health policy analysis research in low- and middle-income countries. We use historical process tracing, a type of comparative case study, to interpret and compare documents and in-depth interview data within and between cases. We found that while network changes were indeed associated with policy reform, this relationship was mediated by one or more of institutions, interests and ideas. In a context of high donor dependency, new donor rules affected the composition and structure of actors in the networks, which enabled the entry and dissemination of new ideas and shifts in the overall balance of interest power ultimately leading to policy change. The case of strategic networking occurred in only one case, by civil society actors, suggesting that network change is rarely the spark that initiates the process towards policy change. This analysis highlights the important role of changes in institutions and ideas to drive policymaking, but hints that network change is a necessary intermediate step in these processes.

Use of Balanced Scorecard Methodology for Performance Measurement of the Health Extension Program in Ethiopia

In 2004, Ethiopia introduced a community-based Health Extension Program to deliver basic and essential health services. We developed a comprehensive performance scoring methodology to assess the performance of the program. A balanced scorecard with six domains and 32 indicators was developed. Data collected from 1,014 service providers, 433 health facilities, and 10,068 community members sampled from 298 villages were used to generate weighted national, regional, and agroecological zone scores for each indicator. The national median indicator scores ranged from 37% to 98% with poor performance in commodity availability, workforce motivation, referral linkage, infection prevention, and quality of care. Indicator scores showed significant difference by region (P < 0.001). Regional performance varied across indicators suggesting that each region had specific areas of strength and deficiency, with Tigray and the Southern Nations, Nationalities and Peoples Region being the best performers while the mainly pastoral regions of Gambela, Afar, and Benishangul-Gumuz were the worst. The findings of this study suggest the need for strategies aimed at improving specific elements of the program and its performance in specific regions to achieve quality and equitable health services.

Finding harmony so the music plays on: pragmatic trial design considerations to promote organizational sustainment of an empirically-supported behavior therapy

Background

Pragmatic trials of empirically-supported behavior therapies may inform clinical and policy decisions concerning therapy sustainment. This retrospective trial design paper describes and discusses pragmatic features of a hybrid type III implementation/effectiveness trial of a contingency management (CM) intervention at an opioid treatment program. Prior reporting (Hartzler et al., J Subst Abuse Treat 46:429–438, 2014; Hartzler, Subst Abuse Treat Prev Policy 10:30, 2015) notes success in recruiting program staff for voluntary participation, durable impacts of CM training on staff-level outcomes, provisional setting implementation of the intervention, documentation of clinical effectiveness, and post-trial sustainment of CM.

Methods/design

Six pragmatic design features, and both scientific and practical bases for their inclusion in the trial, are presented: (1) a collaborative intervention design process, (2) voluntary recruitment of program staff for therapy training and implementation, (3) serial training outcome assessments, with quasi-experimental staff randomization to either single or multiple baseline assessment conditions, (4) designation of a 90-day period immediately after training in which the setting implemented the intervention on a provisional basis, (5) inclusive patient eligibility for receipt of the CM intervention, and (6) designation of two staff as local implementation leaders to oversee clinical/administrative issues in provisional implementation.

Discussion

Each pragmatic trial design feature is argued to have contributed to sustainment of CM. Contributions implicate the building of setting proprietorship for the CM intervention, culling of internal staff expertise in its delivery, iterative use of assessment methods that limited setting burden, documentation of setting-specific clinical effectiveness, expanded penetration of CM among staff during provisional implementation, and promotion of setting self-reliance in the oversight of sustainable implementation procedures. It is hoped this discussion offers ideas for how to impact local clinical and policy decisions via effective behavior therapy dissemination.