How are policy makers using evidence? Models of research utilisation and local NHS policy making

'Stopping the start': support for proposed tobacco control policies - a population-based survey in Great Britain 2021-2023

OBJECTIVES

This study assessed public support for four proposed tobacco control policies in Great Britain: (1) Raising the sales age of tobacco by 1 year every year (Smokefree Generation); (2) Raising the sales age of tobacco from 18 years to 21 years; (3) Providing prescription e-cigarettes as smoking cessation aids to adults who smoke; (4) Restricting e-cigarette advertising to prevent youth uptake.

DESIGN

Repeat cross-sectional population-based survey weighted to match the population of Great Britain.

SETTING

The survey was conducted in England, Scotland and Wales in September 2021, October 2022 and October 2023.

PARTICIPANTS

6541 adults living in Great Britain.

MAIN OUTCOME MEASURES

Support for each policy and year and prevalence ratios (PRs) comparing support between years and subgroups.

RESULTS

The most popular policy each year was restricting e-cigarette advertising (74%/79%/85%), followed by raising the sales age to 21 years (50%/58%/64%), providing prescription e-cigarettes (45%/44%/47%) and Smokefree Generation (34%/44%/49%). The largest increases were for policies about the age of sale (Smokefree Generation: 2021/2022 PR=1.28, 95% CI 1.18 to 1.40, 2022/2023 PR=1.12, 95% CI 1.04 to 1.20; raising the age to 21 years: 2021/2022 PR=1.16, 95% CI 1.09 to 1.23, 2022/2023 PR=1.11, 95% CI 1.05 to 1.17). Only 30% opposed Smokefree Generation in 2023 down from 41% in 2021.

CONCLUSIONS

Support for each policy increased each year, except for providing prescription e-cigarettes. Restricting e-cigarette advertising was the most popular policy, while support for age of sale policies, in particular for a Smokefree Generation, grew most.

TRIAL REGISTRATION

The study protocol was published on the Open Science Framework (https://osf.io/46z2c/) prior to starting the analysis.

Strategies for enacting health policy codesign: a scoping review and direction for research

BACKGROUND

Strategies for supporting evidence-informed health policy are a recognized but understudied area of policy dissemination and implementation science. Codesign describes a set of strategies potentially well suited to address the complexity presented by policy formation and implementation. We examine the health policy literature describing the use of codesign in initiatives intended to combine diverse sources of knowledge and evidence in policymaking.

METHODS

The search included PubMed, MEDLINE, PsychInfo, CINAHL, Web of Science, and Google Scholar in November 2022 and included papers published between 1996 and 2022. Terms included codesign, health, policy, and system terminology. Title and abstracts were reviewed in duplicate and included if efforts informed policy or system-level decision-making. Extracted data followed scoping review guidelines for location, evaluation method, health focus, codesign definition, description, level of health system user input, sectors involved, and reported benefits and challenges.

RESULTS

From 550 titles, 23 citations describing 32 policy codesign studies were included from multiple continents (Australia/New Zealand, 32%; UK/Europe, 32%; South America, 14%; Africa, 9%; USA/Canada 23%). Document type was primarily case study (77%). The area of health focus was widely distributed. Policy type was more commonly little p policy (47%), followed by big p policy (25%), and service innovations that included policy-enabled funding (25%). Models and frameworks originated from formal design (e.g., human-centered or participatory design (44%), political science (38%), or health service research (16%). Reported outcomes included community mobilization (50%), policy feasibility (41%), improved multisector alignment (31%), and introduction of novel ideas and critical thinking (47%). Studies engaging policy users in full decision-making roles self-reported higher levels of community mobilization and community needs than other types of engagement.

DISCUSSION

Policy codesign is theoretically promising and is gaining interest among diverse health sectors for addressing the complexity of policy formation and implementation. The maturity of the science is just emerging. We observed trends in the association of codesign strategies and outcomes that suggests a research agenda in this area could provide practical insights for tailoring policy codesign to respond to local contextual factors including values, needs, and resources.

Prostitutes, sailors and professionals - lived experiences of medical school students and staff with tattoos

Background: The aim of this study was to explore the lived experiences of medical school students and staff to uncover gaps in policy and its effects on those with tattoos. Methods: Adopting a phenomenological design, semi-structured interviews were conducted with ten medical teaching staff and students who had tattoos, within one university medical school. Five key themes emerged: tattoo motives, tattoo content, positive and negative views on tattoos and challenges for establishing policy. Results: Findings suggested that no existing stigma towards staff or students exists, however, there is an absence in existing policy regarding tattoos. Participants' motivations for obtaining tattoos mirrored those of the public, most commonly being artistic expression. Images of tattoos related to nature were common. Views on tattoos suggested the need for a full reform of 'dress code' policy for the profession. Several ethically laden scenarios were espoused, highlighting the need for an official stance on tattoos. Policy discussions flagged challenges for those who construct such documents; cultural and generational differences being commonly identified by participants. Conclusions: Universally there was uncertainty on current policy, with no individual able to recount specific guidance. The existence of a hidden curriculum surrounding tattoos within medical school further increases the need for guidance reform.

The usage of data in NHS primary care commissioning: a realist review

BACKGROUND

Primary care has been described as the 'bedrock' of the National Health Service (NHS) accounting for approximately 90% of patient contacts but is facing significant challenges. Against a backdrop of a rapidly ageing population with increasingly complex health challenges, policy-makers have encouraged primary care commissioners to increase the usage of data when making commissioning decisions. Purported benefits include cost savings and improved population health. However, research on evidence-based commissioning has concluded that commissioners work in complex environments and that closer attention should be paid to the interplay of contextual factors and evidence use. The aim of this review was to understand how and why primary care commissioners use data to inform their decision making, what outcomes this leads to, and understand what factors or contexts promote and inhibit their usage of data.

METHODS

We developed initial programme theory by identifying barriers and facilitators to using data to inform primary care commissioning based on the findings of an exploratory literature search and discussions with programme implementers. We then located a range of diverse studies by searching seven databases as well as grey literature. Using a realist approach, which has an explanatory rather than a judgemental focus, we identified recurrent patterns of outcomes and their associated contexts and mechanisms related to data usage in primary care commissioning to form context-mechanism-outcome (CMO) configurations. We then developed a revised and refined programme theory.

RESULTS

Ninety-two studies met the inclusion criteria, informing the development of 30 CMOs. Primary care commissioners work in complex and demanding environments, and the usage of data are promoted and inhibited by a wide range of contexts including specific commissioning activities, commissioners' perceptions and skillsets, their relationships with external providers of data (analysis), and the characteristics of data themselves. Data are used by commissioners not only as a source of evidence but also as a tool for stimulating commissioning improvements and as a warrant for convincing others about decisions commissioners wish to make. Despite being well-intentioned users of data, commissioners face considerable challenges when trying to use them, and have developed a range of strategies to deal with 'imperfect' data.

CONCLUSIONS

There are still considerable barriers to using data in certain contexts. Understanding and addressing these will be key in light of the government's ongoing commitments to using data to inform policy-making, as well as increasing integrated commissioning.

SciComm Optimizer for Policy Engagement: a randomized controlled trial of the SCOPE model on state legislators' research use in public discourse

BACKGROUND

While prior work has revealed conditions that foster policymakers' use of research evidence, few studies have rigorously investigated the effectiveness of theory-based practices. Specifically, policymakers are most apt to use research evidence when it is timely, relevant, brief, and messaged appropriately, as well as when it facilitates interactive engagement. This study sought to experimentally evaluate an enhanced research dissemination intervention, known as the SciComm Optimizer for Policy Engagement (SCOPE), implemented during the COVID-19 pandemic among US state legislators.

METHODS

State legislators assigned to health committees and their staff were randomized to receive the SCOPE intervention. This involved providing academic researchers with a pathway for translating and disseminating research relevant to current legislative priorities via fact sheets emailed directly to officials. The intervention occurred April 2020-March 2021. Research language was measured in state legislators' social media posts.

RESULTS

Legislators randomized to receive the intervention, relative to the control group, produced 24% more social media posts containing research language related to COVID-19. Secondary analyses revealed that these findings were driven by two different types of research language. Intervention officials produced 67% more COVID-related social media posts referencing technical language (e.g., statistical methods), as well as 28% more posts that referenced research-based concepts. However, they produced 31% fewer posts that referenced creating or disseminating new knowledge.

CONCLUSIONS

This study suggests that strategic, targeted science communication efforts may have the potential to change state legislators' public discourse and use of evidence. Strategic science communication efforts are particularly needed in light of the role government officials have played in communicating about the pandemic to the general public.

“You Can’t Look at an Orange and Draw a Banana”: Using Research Evidence to Develop Relevant Health Policy in Ghana

We explored inhibitors and enablers of using health policy and systems research to inform the policy process in Ghana. The findings suggest a myriad of factors influencing evidence-based policy development, including the strength of the relationships between policy makers and research producers.

We examined factors that either enabled or inhibited the process of evidence-based decision making regarding health policy in Ghana. We conducted qualitative interviews with 2 major groups of stakeholders: health policy and systems research producers (research producers [RPs]) and policy makers (PMs). In-depth interviews were conducted with 12 RPs, who were representatives from 11 health policy and systems research institutions; and 12 PMs working in various national health-related agencies, ministries, and departments. We analyzed the data using the thematic analysis approach. Interview results showed 5 recurring themes in their discussion of enablers and inhibitors of the evidence-to-policy process: (1) the quality, relevance, and quantity of available research evidence; (2) how findings are communicated to PMs; (3) the strength of relationships between RPs and PMs; (4) available structures that promote evidence-based policy making; and (5) the political context in which research and policy making occurs. These findings point to some specific areas for further collaboration and communication among Ghanaian stakeholders to ensure that appropriate health policies are developed from an evidence base.

Open science at the science-policy interface: bringing in the evidence?

Part of the current enthusiasm about open science stems from its promises to reform scientific practice in service of the common good, to ensure that scientific outputs will be found and reused more easily, and to enhance scientific impact on policy and society. With this article, we question this optimism by analysing the potential for open science practices to enhance research uptake at the science–policy interface. Science advice is critical to help policy-makers make informed decisions. Likewise, some interpretations of open science hold that making research processes and outputs more transparent and accessible will also enhance the uptake of results by policy and society at large. However, we argue that this hope is based on an unjustifiably simplistic understanding of the science–policy interface that leaves key terms (“impact”, “uptake”) undefined. We show that this understanding—based upon linear models of research uptake—likewise grounds the influential “evidence–policy gap” diagnosis which holds that to improve research uptake, communication and interaction between researchers and policy-makers need to be improved. The overall normative stance of both discussions has sidelined empirical description of the science–policy interface, ignoring questions about the underlying differences between the policy domain and academia. Importantly, both open science and literature on closing the evidence–policy gap recommend improving communication (in terms of either the content or the means) as a viable strategy. To correct some of these views, we combine insights from policy theory with a narrative review of the literature on the evidence–policy gap in the health domain and find that removing barriers to access by itself will not be enough to foster research uptake.

University-Based Researchers as Knowledge Brokers for Climate Policies and Action

Responding effectively to climate crisis requires strong science-policy links to be put in place. Past research on the research-policy interface indicates longstanding challenges that have become more acute in the case of climate science, since this requires multi-disciplinary approaches and faces distinctive political challenges in linking knowledge with policy. What can be learned from the experiences of university-based researchers seeking to influence policy as they try to operate in the brokering space? With this in mind, an empirical study was designed to capture the detailed views and experiences of forty researchers in four universities across four countries-Bangladesh, Germany, Uganda and UK. It found a wide range of different researcher attitudes to policy engagement, diverse methods of engaging, a preference for working with government and civil society over private sector policy actors, and a perceived need for more university support. The findings suggest a need to rethink conditions for engagement to create spaces for knowledge exchange and cooperation that can contribute to policies for societal transformation. More attention also needs to be paid to interdisciplinary research approaches, improving research connections with private sector actors, and strengthening university research links with local communities. Finally, the position of university based researchers in the Global South will require strengthening to improve North-South knowledge exchange, capacity development, and incentives for policy engagement.

Identifying competencies for integrated knowledge translation: a Delphi study

Background

Considerable progress has been made to advance the field of knowledge translation to address the knowledge-to-action gap in health care; however, there remains a growing concern that misalignments persist between research being conducted and the issues faced by knowledge users, such as clinicians and health policy makers, who make decisions in the health care context. Integrated knowledge translation (IKT) is a collaborative research model that has shown promise in addressing these concerns. It takes advantage of the unique and shared competencies amongst researchers and knowledge users to ensure relevance of the research process and its outcomes. To date, core competencies have already been identified to facilitate training in knowledge translation more generally but they have yet to be prioritized for IKT more specifically. The primary aim of this study was to recruit a group of researchers and knowledge users to identify and prioritize core competencies for researchers and knowledge users to engage with IKT.

Methods

We recruited health care knowledge users (KUs) and researchers with experience and knowledge of IKT for a quantitative, cross-sectional study. We employed a modified Delphi approach consisting of three e-survey rounds to establish consensus on competencies important to IKT for KUs and researchers based on mean rating of importance and agreement between participants.

Results

Nineteen (73%) of the initial 26 participants were researchers (response rate = 41% in the first round; retention in subsequent rounds > 80%). Participants identified a total of 46 competencies important for IKT (18 competencies for KUs, 28 competencies for researchers) under 3 broad domains. Technical research skills were deemed extremely important for researchers, while both groups require teamwork and knowledge translation skills.

Conclusions

This study provides important insight into distinct and overlapping IKT competencies for KUs and researchers. Future work could focus on how these can be further negotiated and contextualized for a wide range of IKT contexts, projects and teams. Greater attention could also be paid to establishing competencies of the entire team to support the research co-production process.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07107-7.

A Novel Method to Assess the Impact of a Government’s Water Strategy on Research: A Case Study of Azraq Basin, Jordan

Water scarcity drives governments in arid and semi-arid regions to promote strategies for improving water use efficiency. Water-related research generally also plays an important role in the same countries and for the same reason. However, it remains unclear how to link the implementation of new government strategies and water-related research. This article’s principal objective is to present a novel approach that defines water-related research gaps from the point of view of a government strategy. The proposed methodology is based on an extensive literature review, followed by a systematic evaluation of the topics covered both in grey and peer-reviewed literature. Finally, we assess if and how the different literature sources contribute to the goals of the water strategy. The methodology was tested by investigating the impact of the water strategy of Jordan’s government (2008–2022) on the research conducted in the Azraq Basin, considering 99 grey and peer-reviewed documents. The results showed an increase in the number of water-related research documents from 37 published between 1985 and 2007 to 62 published between 2008 and 2018. This increase should not, however, be seen as a positive impact of increased research activity from the development of Jordan’s water strategy. In fact, the increase in water-related research activity matches the increasing trend in research production in Jordan generally. Moreover, the results showed that only about 80% of the documents align with the goals identified in the water strategy. In addition, the distribution of the documents among the different goals of the strategy is heterogeneous; hence, research gaps can be identified, i.e., goals of the water-strategy that are not addressed by any of the documents sourced. To foster innovative and demand-based research in the future, a matrix was developed that linked basin-specific research focus areas (RFAs) with the MWI strategy topics. In doing so, the goals that are not covered by a particular RFA are highlighted. This analysis can inspire researchers to develop and apply new topics in the Azraq Basin to address the research gaps and strengthen the connection between the RFAs and the strategy topics and goals. Moreover, the application of the proposed methodology can motivate future research to become demand-driven, innovative, and contribute to solving societal challenges.

An embedded implementation research initiative to tackle service delivery bottlenecks in the expanded programme on immunisation in Pakistan: Overview and reflections

Background

Embedded implementation research (IR) can play a critical role in health systems strengthening by tackling systems and implementation bottlenecks of a program. To achieve this aim, with the financial support of GAVI, the Vaccine Alliance, in 2016, the Government of Pakistan, UNICEF and the Alliance for Health Policy and Systems Research (AHPSR) launched an Embedded IR for Immunisation Initiative (the Initiative) to explore health systems and implementation bottlenecks, and potential strategies to tackle such bottlenecks in the Expanded Programme on Immunisation (EPI) in Pakistan. In total, 10 research teams were involved in the Initiative, which was the first of its kind in the country. In this paper, we provided a brief overview of the Initiative's approach as well as the key learnings including challenges and successes of the research teams which could inform future embedded IR Initiatives.

Methods

Data were collected from members of the IR teams through an online survey. In addition, in-depth interviews were conducted via phone and in-person from IR team members to explore further the challenges they faced while conducting IR in Pakistan and recommendations for future IR initiatives. The qualitative information obtained from these sources was collated and categorized into themes reflecting some of the challenges, successes, and lessons learned, as well as teams' recommendations for future initiatives.

Results

The embedded IR Initiative in Pakistan followed several steps starting with a desk review to compile information on key implementation challenges of EPI and ended with a dissemination workshop where all the research teams shared their IR results with policymakers and implementers. Key factors that facilitated the successful and timely completion of the studies included appreciation by and leadership of implementers in generation and use of local knowledge, identification of research priorities jointly by EPI managers and researchers and provision of continuous and high-quality support from in-country research partners. Participants in the Initiative indicated that challenges included a lack of clarity on the role and responsibilities of each partner involved and need for further support to facilitate use and dissemination of research findings.

Conclusions

The Initiative established that an immunisation programme in a lower middle-income country can use small and time-bound embedded IR, based on partnerships between programme managers and local researchers, to generate information and evidence that can inform decision-making. Future embedded IR initiatives should strive to ensure effective coordination and active participation of all key stakeholders, a clear research utilisation plan from the outset, and efforts to strengthen research teams' capacity to foster utilisation of research findings.

Enhancing psychological safety in mental health services

Background

Psychological safety—speaking up about ideas and concerns, free from interpersonal risk—are essential to the high-risk environment, such as healthcare settings. Psychologically safe working is particularly important in mental health where recovery-oriented approaches rely on collaborative efforts of interprofessional teams to make complex decisions. Much research focuses on antecedents and outcomes associated with psychological safety, but little focus on the practical steps for how to increase psychological safety across and at different levels of a healthcare organisation.

Aims

We explore how a mental health organisation creates an organisation-wide plan for building the foundations of mental health and how to enhance psychological safety.

Methods

This review encompasses strategies across psychological safety and organisational culture change to increase psychological safety at an individual, team and organisational level.

Summary

We set out a comprehensive overview of the types of strategies and interventions for increasing the ethos of psychological safety and setting the foundations for delivering an organisation-wide programme on this topic. We also provide a list of key targeted areas in mental health that would maximally benefit from increasing psychological safety—both in clinical and non-clinical settings.

Conclusions

Psychological safety is a crucial determinant of safe and effective patient care in mental health services. This paper provides the key steps and considerations, creating a large-scale programme in psychological safety with a focus on mental health and drawing from the current literature, providing concrete steps for how our current understanding of psychological safety into practice.

Science in action? A critical view of UK blood donation deferral policy and men who have sex with men

The rules that govern blood donation vary globally. Some potential blood donors are ineligible for immediate blood donation, and as such are deferred until such time that they become eligible. This practice, the blood donation deferral period, is intended to reduce the risk of blood-borne infections being transfused into a blood product-recipient. As blood screening technologies improve, the risk of an infected blood product remaining undetected decreases-and so too have the deferral periods for certain donors. Much has been made of the importance of an evidence-based, scientific approach to protecting blood product-recipients. However, these deferrals are controversial. What exactly determines the blood donation deferral period? This article argues that blood donation deferral periods are not merely the result of enacting empirical data. Instead, the deferral periods represent a negotiation between scientific evidence, experts, politically expedient narratives, institutionalised risk aversion, as well as more mundane concerns such as operational feasibility. As a case study, I examine how the UK Advisory Committee on the Safety of Blood, Tissues and Organs changed the 12-month deferral period for blood donation from men who have sex with men to a 3-month deferral period.

Understanding collaborative interactions in relation to research impact in social sciences and humanities: A meta-ethnography

The number and type of collaborations between researchers and stakeholders has increased significantly. This responds to the demand from policymakers, funders, and citizens that researchers should help to tackle important social issues (e.g. climate change, healthy aging). However, there is little knowledge about how collaboration processes are experienced, how we can theoretically conceptualize them, and how in this way we can develop efficient collaboration methods that contribute to solving urgent societal problems. In this meta-ethnography, we gathered relevant knowledge from carefully selected qualitative studies. A title/abstract analysis of 3422 articles from Web of Science and ProQuest led to the interpretative analysis of qualitative data from eight publications. The results of this study show that despite the need for a better understanding of complex collaborative interactions, the differences (or asymmetry) between organizations to which various partners belong hinder efficient collaboration. Bridging figures (brokers) can play an essential role if they succeed in drawing diverse stakeholders out of their organizational context into a new dimension that allows creativity and mutual understanding, but that also allows conflict and distortion. In ideal circumstances, this leads to a quasi-automatic transfer of knowledge between partners that takes place naturally and in both directions (knowledge diffusion).

Should healthcare organisations offer ongoing rehabilitation services for patients undergoing haematopoietic cell transplant? A narrative review

Purpose

Hematopoietic stem cell transplant (HSCT) patients can suffer from long-term transplant-related complications that affect their quality of life and daily activities. This study, a narrative review, aims to report the impact of HCT complications, the benefits of rehabilitation intervention, the need for long-term care and highlights the research gap in clinical trials involving rehabilitation.

Design/methodology/approach

A comprehensive search strategy was performed on several databases to look for relevant articles published from 1998 to 2018. Articles published in English with the following terms were used: hematopoietic stem cell transplant, chronic graft-versus-host disease, rehabilitation, exercise, physical therapy, occupational therapy. A patient/population, intervention, comparison, and outcomes (PICO) framework was employed to ensure that the search strategies were structured and precise. Study year, design, outcome, intervention, sample demographics, setting and study results were extracted.

Findings

Of the 1,411 records identified, 51 studies underwent title/abstract screening for appropriateness, 30 were reviewed in full, and 19 studies were included in the review. The review found that, for the majority of patients who underwent HSCT and developed treatment-related complications, rehabilitation exercises had a positive impact on their overall quality of life. However, exercise prescription in this patient group has not always reflected the scientific approach; there is a lack of high-quality clinical trials in general. The review also highlights the need to educate healthcare policymakers and insurance companies responsible for rationing services to recognise the importance of offering long-term follow-up care for this patient group, including rehabilitation services.

Practical implications

A large number of HSCT patients require long-term follow-up from a multidisciplinary team, including rehabilitation specialists. It is important for healthcare policymakers and insurance companies to recognise this need and take the necessary steps to ensure that HSCT patients receive adequate long-term care. This paper also highlights the urgent need for high-quality rehabilitation trials to demonstrate the feasibility and importance of rehabilitation teams.

Originality/value

Healthcare policymakers and insurance companies need to recognise that transplant patients need ongoing physiotherapy for early identification of any functional impairments and appropriate timely intervention.

Uncovering the processes of knowledge transformation: the example of local evidence-informed policy-making in United Kingdom healthcare

Background

Healthcare policy-makers are expected to develop ‘evidence-based’ policies. Yet, studies have consistently shown that, like clinical practitioners, they need to combine many varied kinds of evidence and information derived from divergent sources. Working in the complex environment of healthcare decision-making, they have to rely on forms of (practical, contextual) knowledge quite different from that produced by researchers. It is therefore important to understand how and why they transform research-based evidence into the knowledge they ultimately use.

Methods

We purposively selected four healthcare-commissioning organisations working with external agencies that provided research-based evidence to assist with commissioning; we interviewed a total of 52 people involved in that work. This entailed 92 interviews in total, each lasting 20–60 minutes, including 47 with policy-making commissioners, 36 with staff of external agencies, and 9 with freelance specialists, lay representatives and local-authority professionals. We observed 25 meetings (14 within the commissioning organisations) and reviewed relevant documents. We analysed the data thematically using a constant comparison method with a coding framework and developed structured summaries consisting of 20–50 pages for each case-study site. We iteratively discussed and refined emerging findings, including cross-case analyses, in regular research team meetings with facilitated analysis. Further details of the study and other results have been described elsewhere.

Results

The commissioners’ role was to assess the available care provision options, develop justifiable arguments for the preferred alternatives, and navigate them through a tortuous decision-making system with often-conflicting internal and external opinion. In a multi-transactional environment characterised by interactive, pressurised, under-determined decisions, this required repeated, contested sensemaking through negotiation of many sources of evidence. Commissioners therefore had to subject research-based knowledge to multiple ‘knowledge behaviours’/manipulations as they repeatedly re-interpreted and recrafted the available evidence while carrying out their many roles. Two key ‘incorporative processes’ underpinned these activities, namely contextualisation of evidence and engagement of stakeholders. We describe five Active Channels of Knowledge Transformation – Interpersonal Relationships, People Placement, Product Deployment, Copy, Adapt and Paste, and Governance and Procedure – that provided the organisational spaces and the mechanisms for commissioners to constantly reshape research-based knowledge while incorporating it into the eventual policies that configured local health services.

Conclusions

Our new insights into the ways in which policy-makers and practitioners inevitably transform research-based knowledge, rather than simply translate it, could foster more realistic and productive expectations for the conduct and evaluation of research-informed healthcare provision.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies

BACKGROUND

Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

METHODS

A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included.

RESULTS

Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence.

CONCLUSION

This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

A scoping review of the uses and institutionalisation of knowledge for health policy in low- and middle-income countries

There is growing interest in how different forms of knowledge can strengthen policy-making in low- and middle-income country (LMIC) health systems. Additionally, health policy and systems researchers are increasingly aware of the need to design effective institutions for supporting knowledge utilisation in LMICs. To address these interwoven agendas, this scoping review uses the Arskey and O’Malley framework to review the literature on knowledge utilisation in LMIC health systems, using eight public health and social science databases. Articles that described the process for how knowledge was used in policy-making, specified the type of knowledge used, identified actors involved (individual, organisation or professional), and were set in specific LMICs were included. A total of 53 articles, from 1999 to 2016 and representing 56 countries, were identified. The majority of articles in this review presented knowledge utilisation as utilisation of research findings, and to a lesser extent routine health system data, survey data and technical advice. Most of the articles centered on domestic public sector employees and their interactions with civil society representatives, international stakeholders or academics in utilising epistemic knowledge for policy-making in LMICs. Furthermore, nearly all of the articles identified normative dimensions of institutionalisation. While there is some evidence of how different uses and institutionalisation of knowledge can strengthen health systems, the evidence on how these processes can ultimately improve health outcomes remains unclear. Further research on the ways in which knowledge can be effectively utilised and institutionalised is needed to advance the collective understanding of health systems strengthening and enhance evidence-informed policy formulation.

Organisational factors that facilitate research use in public health policy-making: a scoping review

Background

Although important syntheses and theoretical works exist in relation to understanding the organisational factors that facilitate research use, these contributions differ in their scope and object of study as well as their theoretical underpinnings. Therefore, from an exploratory angle, it may be useful to map out the current literature on organisational factors of research use in public health policy-making when revisiting existing theories and frameworks to gain further theoretical insights.

Methods

Herein, a scoping review technique and thematic content analysis were used to bring together findings from both synthesised and empirical studies of different types to map out the organisational factors that facilitate research use in public health policy-making.

Results

A total of 14 reviews and 40 empirical studies were included in the analysis. These were thematically coded and the intra-organisational factors reported as enabling research use were examined. Five main categories of organisational factors that advance research use in policy organisations – (1) individual factors, (2) the management of research integration, (3) organisational systems and infrastructures of research use, (4) institutional structures and rules for policy-making, and (5) organisational characteristics – were derived as well as 18 subcategories and a total of 64 specific factors, where 27 factors were well supported by research.

Conclusions

Using a scoping review methodology, the intra-organisational factors influencing research use in policy-making (including individual factors) were systematically mapped and the theories applied in this area of research were assessed. The review findings confirm the importance of an intra-organisational perspective when exploring research use, showing that many organisational factors are critical facilitators of research use but also that many factors and mechanisms are understudied. The synthesis shows a lack of studies on politicians and the need for more theoretically founded research. Despite increased efforts to update the existing evidential and theoretical basis of research use, we still need frameworks that combine different approaches and theories to help us grasp the complex organisational mechanisms that facilitate research use in policy settings.

Why is translating research into policy so hard? How theory can help public health researchers achieve impact?

OBJECTIVE

To describe how overly simple conceptualisations of how research is translated into public health policy impact impair effective translation. To suggest how alternative approaches to conceptualising impact, which incorporate recent developments in social and political sciences, can help stakeholders improve translation of high-quality public health research into policy impact.

STUDY DESIGN

Researchers often describe generating impact in terms of linear or cyclical models, in which the production of scientific findings alone compels action and leads to impact. However, such conceptualisations do not appear to have supported improved translation of research into policy and practice. Improving understanding of how research impact is achieved may identify areas stakeholders seeking to achieve impact could target.

METHODS

Overview of theoretical and practical approaches to achieving public health policy impact from research.

RESULTS

Despite much evidence that translating research into public health policy is more complex than linear and cyclical models suggest, stakeholders often revert to these heuristics, that is shorthand ways of thinking that allow simple but inaccurate answers to complex problems. This leads to potentially missing opportunities for impact, such as conducting research in collaboration with local policy makers and contributing ideas to the wider narrative through the media and public engagement.

CONCLUSION

The process of translating research into impact appears more complex than that suggested by linear and cyclical models. Success involves a planned approach targeting multiple routes to impact, sustained over time.

An inductive exploration of the implementation knowledge of research funders

Background

Healthcare research funders may undertake various roles to facilitate implementation of research findings. Their ability to enact such roles depends on several factors, knowledge of implementation being one essential requirement. However, previous studies do not assess the type or level of knowledge about implementation that research funders possess. This paper therefore presents findings from a qualitative, inductive study of the implementation knowledge of research funders. Three aspects of this knowledge are explored, namely how research funders define implementation, their level of self-assessed implementation knowledge and the factors influencing their self-assessment of implementation knowledge.

Methods

Research funders (n = 18) were purposefully selected from a sample of research funding organisations in Sweden (n = 10). In-depth semi-structured interviews were conducted, recorded and transcribed verbatim. An inductive method using a systematic coding procedure was employed to derive the findings.

Results

The research funders defined implementation as either an outcome or a process, with the majority believing that implementation of healthcare research results demands a process, although its complexity varied in the research funders’ view. They perceived their own level of implementation knowledge as either limited or substantial, with a majority regarding it as limited. Clinical research experience, clinical experience and task relevance were singled out as the clearest factors affecting the self-assessment of their own implementation knowledge.

Conclusions

This study, the first to focus on implementation knowledge of research funders, demonstrates that they are a category of policy-makers who may possess knowledge, based on their previous professional experience, that is comparable to some important findings from implementation research. Consequently, the findings not only pinpoint the relevance of professional experience, but also reveal a lack of awareness and knowledge of the results of implementation research among research funders in charge of healthcare research.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0472-8) contains supplementary material, which is available to authorized users.

How was research engaged with and used in the development of 131 policy documents? Findings and measurement implications from a mixed methods study

Background

Much has been written about the use of evidence in policy; however, there is still little known about whether and how research is engaged with and used in policy development or the impact of reported barriers and facilitators. This paper aims to (1) describe the characteristics of 131 policy documents, (2) describe the ways in which research was engaged with (e.g. was searched for, appraised or generated) and used (e.g. to clarify understanding, persuade others or inform a policy) in the development of these policy documents, and (3) identify the most commonly reported barriers and facilitators and describe their association with research engagement and use.

Methods

Six health policy and program development agencies based in Sydney, Australia, contributed four recently finalised policy documents for consideration over six measurement periods. Structured, qualitative interviews were conducted with the policymakers most heavily involved in developing each of the 131 policy documents. Interviews covered whether and how research was engaged with and used in the development of the policy product and any barriers or facilitators related to this. Interviews were scored using the empirically validated SAGE tool and thematically analysed. Descriptive statistics were calculated for all key variables and comparisons made between agencies. Multiple regression analyses were used to estimate the impact of specific barriers and facilitators on research engagement and use.

Results

Our data shows large variations between policy agencies in the types of policy documents produced and the characteristics of these documents. Nevertheless, research engagement and use was generally moderate across agencies. A number of barriers and facilitators to research use were identified. No barriers were significantly associated with any aspects of research engagement or use. Access to consultants and relationships with researchers were both associated with increased research engagement but not use. Thus, access to consultants and relationships with researchers may increase the extent and quality of the evidence considered in policy development.

Conclusions

Our findings suggest that those wishing to develop interventions and programs designed to improve the use of evidence in policy agencies might usefully target increasing access to consultants and relationships with researchers in order to increase the extent and quality of the research considered, but that a greater consideration of context might be required to develop strategies to increase evidence use.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0886-2) contains supplementary material, which is available to authorized users.

The dark side of coproduction: do the costs outweigh the benefits for health research?

Background

Coproduction, a collaborative model of research that includes stakeholders in the research process, has been widely advocated as a means of facilitating research use and impact. We summarise the arguments in favour of coproduction, the different approaches to establishing coproductive work and their costs, and offer some advice as to when and how to consider coproduction.

Debate

Despite the multiplicity of reasons and incentives to coproduce, there is little consensus about what coproduction is, why we do it, what effects we are trying to achieve, or the best coproduction techniques to achieve policy, practice or population health change. Furthermore, coproduction is not free risk or cost. Tensions can arise throughout coproduced research processes between the different interests involved. We identify five types of costs associated with coproduced research affecting the research itself, the research process, professional risks for researchers and stakeholders, personal risks for researchers and stakeholders, and risks to the wider cause of scholarship. Yet, these costs are rarely referred to in the literature, which generally calls for greater inclusion of stakeholders in research processes, focusing exclusively on potential positives. There are few tools to help researchers avoid or alleviate risks to themselves and their stakeholders.

Conclusions

First, we recommend identifying specific motivations for coproduction and clarifying exactly which outcomes are required for whom for any particular piece of research. Second, we suggest selecting strategies specifically designed to enable these outcomes to be achieved, and properly evaluated. Finally, in the absence of strong evidence about the impact and process of coproduction, we advise a cautious approach to coproduction. This would involve conscious and reflective research practice, evaluation of how coproduced research practices change outcomes, and exploration of the costs and benefits of coproduction. We propose some preliminary advice to help decide when coproduction is likely to be more or less useful.

Readiness to Adopt Physical Activity Policies in Rural Communities

The purpose was to explore community readiness to adopt physical activity (PA) policies by adapting and pilot testing an online survey (Physical Activity Readiness Survey [PARS]). PARS was adapted from the previously tested Community Readiness Survey-Short. In February/April 2016, key informants (N = 17) involved in PA activities from two rural communities were invited to complete the PARS, representing six dimensions: knowledge, leadership, resources, community climate, existing voluntary PA policies, and political climate. First, participants were asked to respond to a presurvey to screen for overall readiness for up to four evidence-based PA policies. A main survey readiness score (0-6) was determined by averaging the key informants' ratings across items: Raw scores were rescaled to range from 0 to 1, and dimension scores were summed. Participants identified two PA policies in the presurvey: neighborhood availability and point-of-decision prompts. For both policies, political climate had the highest dimension score (1.0) and the knowledge dimension scored lowest (0.05-0.38). Overall readiness scores ranged from 3.19 to 3.62, revealing the preparation stage for both policies. Readiness for the two PA policies were similar, but specific dimension scores varied by policy type and community, reinforcing the need for tailored interventions.

A content analysis of popular media reporting regarding increases in minimum ages of legal access for tobacco

BACKGROUND

In the late 20th century, US localities began increasing the minimum age of legal access (MLA) for tobacco from 18 to 21 years by enacting "Tobacco 21" ordinances. Although these policies have a strong evidence base and broad popular support, popular media coverage of tobacco control laws has not always been accurate. This study sought to determine if contemporaneous popular media reporting accurately reflected the scientific findings regarding increased tobacco MLAs.

METHODS

We searched LexisNexis for popular media reports that (1) addressed proposed or enacted Tobacco 21 ordinances and were (2) published in English, (3) drawn from a US news source, and (4) written after January 2004. We conducted a content analysis for quality based on a validated measure of accuracy of reporting, the Index of Scientific Quality (ISQ), which allows assessment of articles by assigning scores ranging from 1 (lowest) to 5 (highest).

RESULTS

Searches yielded 378 articles; after screening for relevance and duplicates, 98 were included in the review. All studies identified through the keyword searches addressed Tobacco 21 policies. The average global score identifying the scientific quality of the articles was 2.98 of 5. Over three-quarters of the popular media articles addressing Tobacco 21 laws were written after a systematic review of these policies was released by the Institute of Medicine and approximately 4 in 10 cited findings from that review.

CONCLUSIONS

Popular media reports on Tobacco 21 laws demonstrated average overall quality and relied on both anecdotal and scientific evidence, in contrast to previous studies found that popular media reports on tobacco issues demonstrated low overall quality and relied primarily on anecdotal evidence. The systematic review of increased MLAs for tobacco written by the Institute of Medicine diffused quickly into popular reporting, suggesting that this type of evidence might improve research translation.

Factors affecting engagement between academic faculty and decision-makers: learnings and priorities for a school of public health

Background

Schools of public health (SPHs) are increasingly being recognised as important contributors of human, social and intellectual capital relevant to health policy and decision-making. Few studies within the implementation science literature have systematically examined knowledge exchange experiences within this specific organisational context. The purpose of this study was therefore to elicit whether documented facilitators and barriers to engaging with government decision-makers resonates within an academic SPH context. We sought to understand the variations in such experiences at four different levels of government decision-making. Furthermore, we sought to elicit intervention priorities as identified by faculty.

Methods

Between May and December 2016, 211 (34%) of 627 eligible full-time faculty across one SPH in the United States of America participated in a survey on engagement with decision-makers at the city, state, federal and global government levels. Surveys were administered face-to-face or via Skype. Descriptive data as well as tests of association and logistic regression analyses were conducted using STATA.

Results

Over three-quarters of respondents identified colleagues with ties to decision-makers, institutional affiliation and conducting policy-relevant research as the highest facilitators. Several identified time constraints, academic incentives and financial support as important contributors to engagement. Faculty characteristics, such as research areas of expertise, career track and faculty rank, were found to be statistically significantly associated with facilitators. The top three intervention priorities that emerged were (1) creating incentives for engagement, (2) providing funding for engagement and (3) inculcating an institutional culture around engagement.

Conclusions

The data suggest that five principal categories of factors – individual characteristics, institutional environment, relational dynamics, research focus and funder policies – affect the willingness and ability of academic faculty to engage with government decision-makers. This study suggests that SPHs could enhance the relevance of their role in health policy decision-making by (1) periodically measuring engagement with decision-makers; (2) enhancing individual capacity in knowledge translation and communication, taking faculty characteristics into account; (3) institutionalising a culture that supports policies and practices for engagement in decision-making processes; and (4) creating a strategy to expand and nurture trusted, relevant networks and relationships with decision-makers.

Identifying public health policymakers' sources of information: comparing survey and network analyses

Background

Research suggests that policymakers often use personal contacts to find information and advice. However, the main sources of information for public health policymakers are not known. This study aims to describe policymakers' sources of information. A questionnaire survey of public health policymakers across Greater Manchester (GM) was carried out (response rate 48%). All policy actors above Director level involved in public health policy (finding, analyzing or producing information, producing or implementing policy) in GM were included in the sampling frame. Respondents were provided with a list of sources of information and asked which they used (categorical data) and to name specific individuals who acted as sources of information (network data). Data were analyzed using frequencies and network analysis. The most frequently chosen sources of information from the categorical data were NICE, government websites and Directors of Public Health. However, the network data showed that the main sources of information in the network were actually mid-level managers in the NHS, who had no direct expertise in public health. Academics and researchers did not feature in the network. Both survey and network analyses provide useful insights into how policymakers access information. Network analysis offers practical and theoretical contributions to the evidence-based policy debate. Identifying individuals who act as key users and producers of evidence allows academics to target actors likely to use and disseminate their work.

Do peer review models affect clinicians' trust in journals? A survey of junior doctors

Background

The aim of this survey was to determine the level of awareness and understanding of peer review and peer review models amongst junior hospital doctors and whether this influences clinical decision-making.

Methods

A 30-question online anonymous survey was developed aimed at determining awareness of peer review models and the purpose of peer review, perceived trustworthiness of different peer review models and the role of peer review in clinical decision-making. It was sent to 800 trainee doctors in medical specialties on the University College London Partners trainee database.

Results

The response rate was (178/800) 22%. Most respondents were specialist registrars. Checking that research is conducted correctly (152/178, 85%) and the data interpreted correctly (148/178, 83%) were viewed as the most important purposes of peer review. Most respondents were aware of open (133/178, 75%), double-blind (125/178, 70%) and single-blind peer review (121/178, 68%). 101/178 (57%) had heard of collaborative, 87/178 (49%) of post publication and 29/178 (16%) of decoupled peer review. Of those who were aware of double-blind, single-blind open and collaborative peer review, 85 (68%), 82 (68%), 74 (56%) and 24 (24%), respectively, understood how they worked. The NEJM, Lancet and The BMJ were deemed to have most trustworthy peer review, 137/178 (77%), 129/178 (72%) and 115/178 (65%), respectively. That peer review had taken place was important for a journal content to be used for clinical decision-making 152/178 (85%), but the ability to see peer review reports was not as important 22/178 (12%). Most felt there was a need for peer review training and that this should be at the specialist registrar stage of training.

Conclusions

Junior hospital doctors view peer review to be important as a means of quality control, but do not value the ability to scrutinize peer review themselves. The unquestioning acceptance of peer review as final validation in the field of medicine emphasises not only the responsibility held by medical journals to ensure peer review is done well but also the need to raise awareness amongst the medical community of the limitations of the current peer review process.

Development of a framework to improve the utilisation of malaria research for policy development in Malawi

BACKGROUND

The existing gap between research evidence and public health practice has attributed to the unmet Millennium Development Goals in Africa and consequently, has stimulated the development of frameworks to enhance knowledge translation. These efforts aim at maximising health research utilisation in policy and practice to address the world's disease burdens, including malaria. This study aimed at developing a contextual framework to improve the utilisation of malaria research for policy development in Malawi.

METHODS

The study used two approaches including: two case studies of policy analysis exploring the policy-making process in Malawi, utilisation of local malaria research, and the role of key stakeholders in policy formulation process; and the assessment of facilitating factors and barriers to malaria research utilisation for policy-making in Malawi.

RESULTS

From the case studies' lessons and elements identified during the assessment of facilitating factors and barriers, a framework is developed to promote an integrated approach to knowledge translation. In this framework the Ministry of Health is considered as the main user of knowledge from research through the demand created by the research directorate and the National Malaria Control Programme. Key documents identified as being particularly relevant to the Ministry of Health for purposes of knowledge translation include the National Health Research Agenda, Guidelines for Policy Development and Analysis, and Guidelines for Evidence Use in Policy-making. Institutions conducting academic and policy-relevant malaria research in Malawi are identified and a consolidation of their linkages with the users of research is established through the Knowledge Translation Unit, the Evidence Informed decision-making Centre, and the African Institute for Development Policy. Equally, key players in this framework are the funding partners for both research and programmes that need to see accountability and impact of their support. Independent advisors, partners, and consultants also have their vital role in the process.

CONCLUSION

The framework offers a practical basis for the factors identified and their linkages to promote a co-ordinated approach to malaria research utilisation in policy-making. Its applicability and success hinges on its wider dissemination and ownership by the government through the National Malaria Control Programme.

The research-policy-deliberation nexus: a case study approach

Background

Decision-makers tend to make connections with researchers far too late in the game of public policy, expecting to find a retail store in which researchers are busy filling shop-front shelves with a comprehensive set of all possible relevant studies that a decision-maker might some day drop by to purchase. This linear type of relation between research and policy needs to be replaced by a more interactive model that facilitates both researchers obtaining a better understanding of policy processes and policymakers being more aware and involved in the conceptualisation and conduct of research. This paper explores the role of governance in facilitating the research–policy nexus, testing a typology of research utilisation based on Murray’s (Soc Policy Society 10(4):459–70, 2011) analysis that considers various degrees of researcher–policymaker deliberation in decision-making processes. The projects were all part of various evaluation efforts carried out by the researchers to explore the use of governance in health promotion activities.

Methods

Three case studies were chosen to provide some specific examples that illustrate each level of Murray’s typology. The examples involve intersectoral health promotion collaborations that combine evidence-based research in health policy initiatives with various levels of researcher involvement. For all three projects, interview data was collated in the same way, coded thematically and analysed to consider the relationship between researchers and policymakers.

Results

Comparing the three models and their applicability to health promotion interventions, it could be observed that all programmes demonstrated successful examples of research translation. Strong governance imperatives structuring relationships led to more successful outcomes, whereby research was successfully translated into a public policy initiative that also led to improved health outcomes. The key idea across all of these models was that strong governance arrangements mitigated some of the barriers evidenced by the varying degrees of deliberation and researcher involvement in processes.

Conclusions

The paper demonstrates that successful research utilisation is related to strong governance agendas and that early and ongoing involvement of relevant decision-makers and researchers in the governance processes, that is both the conceptualisation and conduct of a study, tend to be the best predictors of success.

Using Obesity Research to Shape Obesity Policy in Minnesota: Stakeholder Insights and Feasibility of Recommendations

OBJECTIVE

Preventing childhood obesity requires innovative, evidence-based policy approaches. This study examines the use of research evidence by obesity policy stakeholders in Minnesota and develops pilot tools for communicating timely evidence to policymakers.

DESIGN, SETTING, AND PARTICIPANTS

From November 2012 to January 2013, semistructured interviews were conducted with 51 Minnesota stakeholders in childhood obesity prevention. Interviewees included 16 state legislators and staff; 16 personnel from the Minnesota Department of Education, Minnesota Department of Health, and Minnesota Department of Transportation; and 19 advocates for and against childhood obesity prevention legislation (response rate = 71%).

MAIN OUTCOME MEASURES

Participants were asked their views on 3 themes: (1) Whether and how they used research evidence in their current decision-making processes; (2) barriers to using research evidence for policymaking; and (3) suggestions for improving the evidence translation process. All interviews were audio-recorded and transcribed. A team approach to qualitative analysis was used to summarize themes, compare findings across interviewees' professional roles, and highlight unexpected findings, areas of tension, or illuminating quotes.

RESULTS

Stakeholders used research evidence to support policy decisions, educate the public, and overcome value-based arguments. Common challenges included the amount and complexity of research produced and limited relationships between researchers and decision makers. Responding to interviewee recommendations, we developed and assessed 2 pilot tools: a directory of research experts and a series of research webinars on topics related to childhood obesity. Stakeholders found these materials relevant and high-quality but expressed uncertainty about using them in making policy decisions.

CONCLUSIONS

Stakeholders believe that research evidence should inform the design of programs and policies for childhood obesity prevention; however, many lack the time and resources to consult research consistently. Future efforts to facilitate evidence-informed policymaking should emphasize approaches to designing and presenting research that better meets the needs of policy and programmatic decision makers.

Identifying and prioritising systematic review topics with public health stakeholders: A protocol for a modified Delphi study in Switzerland to inform future research agendas

INTRODUCTION

The Cochrane Collaboration aims to produce relevant and top priority evidence that responds to existing evidence gaps. Hence, research priority setting (RPS) is important to identify which potential research gaps are deemed most important. Moreover, RPS supports future health research to conform both health and health evidence needs. However, studies that are prioritising systematic review topics in public health are surprisingly rare. Therefore, to inform the research agenda of Cochrane Public Health Europe (CPHE), we introduce the protocol of a priority setting study on systematic review topics in several European countries, which is conceptualised as pilot.

METHODS AND ANALYSIS

We will conduct a two-round modified Delphi study in Switzerland, incorporating an anonymous web-based questionnaire, to assess which topics should be prioritised for systematic reviews in public health. In the first Delphi round public health stakeholders will suggest relevant assessment criteria and potential priority topics. In the second Delphi round the participants indicate their (dis)agreement to the aggregated results of the first round and rate the potential review topics with the predetermined criteria on a four-point Likert scale. As we invite a wide variety of stakeholders we will compare the results between the different stakeholder groups.

ETHICS AND DISSEMINATION

We have received ethical approval from the ethical board of the University of Bremen, Germany (principal investigation is conducted at the University of Bremen) and a certificate of non-objection from the Canton of Zurich, Switzerland (fieldwork will be conducted in Switzerland). The results of this study will be further disseminated through peer reviewed publication and will support systematic review author groups (i.a. CPHE) to improve the relevance of the groups´ future review work. Finally, the proposed priority setting study can be used as a framework by other systematic review groups when conducting a priority setting study in a different context.

Improving the capabilities of NHS organisations to use evidence: a qualitative study of redesign projects in Clinical Commissioning Groups

Background

Innovation driven by authoritative evidence is critical to the survival of England’s NHS. Clinical Commissioning Groups (CCGs) are central in NHS efforts to do more with less. Although decisions should be based on the ‘best available evidence’, this is often problematic, with frequent mismatches between the evidence ‘pushed’ by producers and that used in management work. Our concern, then, is to understand practices and conditions (which we term ‘capabilities’) that enable evidence use in commissioning work. We consider how research gets into CCGs (‘push’), how CCGs use evidence (‘pull’) and how this can be supported (toolkit development). We aim to contribute to evidence-based NHS innovation, and, more generally, to improved health-care service provision.

Method

Supported by the National Institute for Health Research (NIHR), we conducted semistructured ethnographic interviews in eight CCGs. We also conducted observations of redesign meetings in two of the CCGs. We used inductive and deductive coding to identify evidence used and capabilities for use from the qualitative data. We then compared across cases to understand variations in outcomes as a function of capabilities. To help improvements in commissioning, we collated our findings into a toolkit for use by stakeholders. We also conducted a small-scale case study of the production of evidence-based guidance to understand evidence ‘push’.

Results

Fieldwork indicated that different evidences inform CCG decision-making, which we categorise as ‘universal’, ‘local’, ‘expertise-based’ and ‘trans-local’. Fieldwork also indicated that certain practices and conditions (‘capabilities’) enable evidence use, including ‘sourcing and evaluating evidence’, ‘engaging experts’, ‘effective framing’, ‘managing roles and expectations’ and ‘managing expert collaboration’. Importantly, cases in which fewer capabilities were recorded tended to report more problems, relative to cases in which needed capabilities were applied. These latter cases were more likely to effectively use evidence, achieve objectives and maintain stakeholder satisfaction. We also found that various understandings of end-users are inscribed into products by evidence producers, which seems to reflect the evolving landscape of the production of authoritative evidence.

Conclusions

This was exploratory research on evidence use capabilities in commissioning decisions. The findings suggest that commissioning stakeholders need support to identify, understand and apply evidence. Support to develop capabilities for evidence may be one means of ensuring effective, evidence-based innovations in commissioning. Our work with evidence producers also shows variation in their perceptions of end users, which may inform the ‘push’/’pull’ gap between research and practice. There were also some limitations to our project, including a smaller than expected sample size and a time frame that did not allow us to capture full redesign projects in all CCGs.

Future work

With these findings in mind, future work may look more closely at how information comes to be treated as evidence and at the relationships of capabilities to project outcomes. Going forward, knowledge, especially that related to generalisability, may be built by means of a longer time and the study of redesign projects in different settings.

Funding

The NIHR Health Services and Delivery Research programme.

Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

There is extensive health and public health literature on the ‘evidence-policy gap’, exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems.

We identify insights from secondary data, namely systematic reviews, critical analysis and policy theories relevant to evidence-based policymaking. The studies are drawn primarily from countries such as the United States, United Kingdom, Canada, Australia and New Zealand. We combine empirical and normative elements to identify the ways in which scientists can, do and could influence policy.

We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.

We conclude that successful engagement in ‘evidence-based policymaking’ requires pragmatism, combining scientific evidence with governance principles, and persuasion to translate complex evidence into simple stories. To maximise the use of scientific evidence in health and public health policy, researchers should recognise the tendency of policymakers to base judgements on their beliefs, and shortcuts based on their emotions and familiarity with information; learn ‘where the action is’, and be prepared to engage in long-term strategies to be able to influence policy; and, in both cases, decide how far you are willing to go to persuade policymakers to act and secure a hierarchy of evidence underpinning policy. These are value-driven and political, not just ‘evidence-based’, choices.

Determinants of evidence use in public health policy making: Results from a study across six EU countries

The knowledge-practice gap in public health is widely known. The importance of using different types of evidence for the development of effective health promotion has also been emphasized. Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking the most effective approaches, thus limiting the impact of health promotion strategies. This article focuses on facilitators and barriers in the use of evidence in developing health enhancing physical activity policies. Data was collected in 2012 by interviewing 86 key stakeholders from six EU countries (FI, DK, UK, NL, IT, RO) using a common topic guide. Content analysis and concept mapping was used to construct a map of facilitators and barriers. Barriers and facilitators experienced by most stakeholders and policy context in each country are analysed. A lack of locally useful and concrete evidence, evidence on costs, and a lack of joint understanding were specific hindrances. Also users' characteristics and the role media play were identified as factors of influence. Attention for individual and social factors within the policy context might provide the key to enhance more sustainable evidence use. Developing and evaluating tailored approaches impacting on networking, personal relationships, collaboration and evidence coproduction is recommended.

Interpretation and use of evidence in state policymaking: a qualitative analysis

INTRODUCTION

Researchers advocating for evidence-informed policy have attempted to encourage policymakers to develop a greater understanding of research and researchers to develop a better understanding of the policymaking process. Our aim was to apply findings drawn from studies of the policymaking process, specifically the theory of policy windows, to identify strategies used to integrate evidence into policymaking and points in the policymaking process where evidence was more or less relevant.

METHODS

Our observational study relied on interviews conducted with 24 policymakers from the USA who had been trained to interpret scientific research in multiple iterations of an evidence-based workshop. Participants were asked to describe cases where they had been involved in making health policy and to provide examples in which research was used, either successfully or unsuccessfully. Interviews were transcribed, independently coded by multiple members of the study team and analysed for content using key words, concepts identified by participants and concepts arising from review of the texts.

RESULTS

Our results suggest that policymakers who focused on health issues used multiple strategies to encourage evidence-informed policymaking. The respondents used a strict definition of what constituted evidence, and relied on their experience with research to discourage the use of less rigorous research. Their experience suggested that evidence was less useful in identifying problems, encouraging political action or ensuring feasibility and more useful in developing policy alternatives.

CONCLUSIONS

Past research has suggested multiple strategies to increase the use of evidence in policymaking, including the development of rapid-response research and policy-oriented summaries of data. Our findings suggest that these strategies may be most relevant to the policymaking stream, which develops policy alternatives. In addition, we identify several strategies that policymakers and researchers can apply to encourage evidence-informed policymaking.

Health services research and development in practice: An Australian experience

While there is a growing literature on how health services research can inform health policy decisions, the practical challenge is for health services researchers to develop an effective interface with health policy-making processes and to produce outputs that lead to outcomes. The experience of the Centre for Health Service Development at the University of Wollongong, Australia, is used to illustrate the issues so commonly described in the literature and to reflect on our experience of trying to remain viable while producing relevant and valid research. A case study in a specific policy area - namely, the development of case-mix classifications and information systems to inform policy and funding in the subacute and non-acute hospital and community care sectors - is used as a practical example of the research-policy interface.

The path to impact of operational research on tuberculosis control policies and practices in Indonesia

Background

Operational research is currently one of the pillars of the global strategy to control tuberculosis. Indonesia initiated capacity building for operational research on tuberculosis over the last decade. Although publication of the research in peer-reviewed journals is an important indicator for measuring the success of this endeavor, the influence of operational research on policy and practices is considered even more important. However, little is known about the process by which operational research influences tuberculosis control policy and practices.

Objective

We aimed to investigate the influence of operational research on tuberculosis control policy and practice in Indonesia between 2004 and 2014.

Design

Using a qualitative study design, we conducted in-depth interviews of 50 researchers and 30 policy makers/program managers and performed document reviews. Transcripts of these interviews were evaluated while applying content analysis.

Results

Operational research contributed to tuberculosis control policy and practice improvements, including development of new policies, introduction of new practices, and reinforcement of current program policies and practices. However, most of these developments had limited sustainability. The path from the dissemination of research results and recommendations to policy and practice changes was long and complex. The skills, interests, and political power of researchers and policy makers, as well as health system response, could influence the process.

Conclusions

Operational research contributed to improving tuberculosis control policy and practices. A systematic approach to improve the sustainability of the impact of operational research should be explored.

State legislators' sources and use of information: bridging the gap between research and policy

Research can inform policymakers of public health issues and shape policy decisions, hopefully benefiting public health; thus, improving dissemination of research to policymakers is important for developing effective public health policies that improve health and health equity. However, the utilization of research among policymakers is often not fully realized. This study builds upon current knowledge about what types of information legislators seek when working on health issues and where they go for information. Further, it explores what kinds of information legislators find most helpful and if there are ways researchers could better provide this evidence. Key-informant interviews were conducted with 25 U.S. state legislators holding health committee leadership positions between July and November, 2010. Regarding types of information sought, most legislators discussed their desire for data and statistics when working on health-related issues. When asked about their most trusted sources of information, participants mentioned government sources as well as advocacy, lobby and industry groups. A few mentioned universities and healthcare professionals. Results from this study offer public health researchers and practitioners' insights into the types of information that may be most helpful to policymakers. Insights gathered may improve the dissemination of research and bridge the gap between knowledge users and knowledge producers.

Bridging evidence, policy, and practice to strengthen health systems for improved maternal and newborn health in Pakistan

Policy and decision making should be based on evidence, but translating evidence into policy and practice is often sporadic and slow. It is recognised that the relationship between research and policy uptake is complex and that dissemination of research findings is necessary, but insufficient, for policy uptake. Political, social, and economic context, use of (credible) data and dialogues between and across networks of researchers and policymakers play important roles in evidence uptake. Advocacy is the process of mobilising political and public opinions to achieve specific aims and its role is crucial in mobilising key actors to push for policy uptake. Advocacy and research groups (i.e. those who would like to see research evidence used by policymakers) may use different approaches and tools to stimulate the diffusion of research findings. The use of mass- and social media, communication with study participants, and the involvement of stakeholders at the early stages of research development are examples of the approaches that can be employed to stimulate diffusion of evidence and increase evidence uptake. The Research and Advocacy Fund (RAF) for Maternal and Newborn Health (MNH) worked within the health system context in Pakistan with the aim of espousing the principles of evidence, advocacy, and dissemination to improve MNH outcomes. The articles included in this special issue are outputs of RAF and highlight where RAF’s approaches contributed to MNH policy reforms. The papers discuss critical health system issues facing Pakistan, including service delivery components, demand creation, equitable access, transportation interventions for improved referrals, availability of medicines and equipment, and health workforce needs. In addition to these tangible elements, the health system ‘software’, i.e. the power and the political and social contexts, is also represented in the collection. These articles highlight three considerations for the future: the growing importance of implementation research, the crucial need for participation and ownership, and the recognition that policymaking can be ‘informed’ by rather than ‘based-on’ evidence. The future challenge will be to continue the momentum RAF has created and to welcome a new era of health, wealth, and growth for Pakistan.

Exploring the function and effectiveness of knowledge brokers as facilitators of knowledge translation in health-related settings: a systematic review and thematic analysis

Background

Knowledge brokers (KBs) work collaboratively with key stakeholders to facilitate the transfer and exchange of information in a given context. Currently, there is a perceived lack of evidence about the effectiveness of knowledge brokering and the factors that influence its success as a knowledge translation (KT) mechanism. Thus, the goal of this review was to systematically gather evidence regarding the nature of knowledge brokering in health-related settings and determine if KBs effectively contributed to KT in these settings.

Methods

A systematic review was conducted using a search strategy designed by a health research librarian. Eight electronic databases (MEDLINE, Embase, PsycINFO, CINAHL, ERIC, Scopus, SocINDEX, and Health Business Elite) and relevant grey literature sources were searched using English language restrictions. Two reviewers independently screened the abstracts, reviewed full-text articles, extracted data, and performed quality assessments. Analysis included a confirmatory thematic approach. To be included, studies must have occurred in a health-related setting, reported on an actual application of knowledge brokering, and be available in English.

Results

In total, 7935 records were located. Following removal of duplicates, 6936 abstracts were screened and 240 full-text articles were reviewed. Ultimately, 29 articles, representing 22 unique studies, were included in the thematic analysis. Qualitative (n = 18), quantitative (n = 1), and mixed methods (n = 6) designs were represented in addition to grey literature sources (n = 4). Findings indicated that KBs performed a diverse range of tasks across multiple health-related settings; results supported the KB role as a ‘knowledge manager’, ‘linkage agent’, and ‘capacity builder’. Our systematic review explored outcome data from a subset of studies (n = 8) for evidence of changes in knowledge, skills, and policies or practices related to knowledge brokering. Two studies met standards for acceptable methodological rigour; thus, findings were inconclusive regarding KB effectiveness.

Conclusions

As knowledge managers, linkage agents, and capacity builders, KBs performed many and varied tasks to transfer and exchange information across health-related stakeholders, settings, and sectors. How effectively they fulfilled their role in facilitating KT processes is unclear; further rigourous research is required to answer this question and discern the potential impact of KBs on education, practice, and policy.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-015-0351-9) contains supplementary material, which is available to authorized users.

Health systems research for policy change: lessons from the implementation of rapid assessment protocols for diabetes in low- and middle-income settings

Background

As many challenges exist for access to diabetes care in developing countries, the International Insulin Foundation developed a Rapid Assessment tool and implemented this approach to identify barriers to care and propose concrete recommendations for decision makers. The objective of this paper is to identify the factors that contributed to informing and influencing policymakers with regards to this work.

Methods

A documentary review comprised Stage 1. Stage 2 used an online questionnaire to gain insight from users of the Rapid Assessment results. Based on Stages 1 and 2, Stage 3 comprised in-depth interviews with a total of nine individuals (one individual each from the six participating countries; two individuals from the World Health Organization; one “Global Diabetes Advocate”). Interviews were analyzed based on a list of themes developed from Stage 2.

Results

Stage 1 led to the identification of various types of documents referring to the results. The online questionnaire had a response rate of 33%. Respondents directly involved in the assessment had a “Good” or “Very Good” appreciation of most aspects and scored these higher than those not directly involved. From the interviews, formalized methods and close collaboration between the international team and local partners were strengths. Trust and a relationship with local partners were also seen as assets. All stakeholders valued the results and the credibility of the data generated. Local partners felt that more could have been done for dissemination.

Conclusion

This study shows the importance of specific results from the different assessments. In addressing complex issues having external experts involved was seen as an advantage. The uptake of results was due to the credibility of the research which was influenced by a mix of the people involved, past assessments, trusted local partners, and the use of the results by knowledge brokers, such as the World Health Organization. Through these brokers, others gained ownership of the data. The methods used and the fact that this data was grounded in a local context also reinforced its value. Despite limitations, this study offers a unique perspective where a similar research approach was taken in six countries.

Using conjoint analysis to develop a system of scoring policymakers' use of research in policy and program development

Background

The importance of utilising the best available research evidence in the development of health policies, services, and programs is increasingly recognised, yet few standardised systems for quantifying policymakers’ research use are available. We developed a comprehensive measurement and scoring tool that assesses four domains of research use (i.e. instrumental, conceptual, tactical, and imposed). The scoring tool breaks down each domain into its key subactions like a checklist. Our aim was to develop a tool that assigned appropriate scores to each subaction based on its relative importance to undertaking evidence-informed health policymaking. In order to establish the relative importance of each research use subaction and generate this scoring system, we conducted conjoint analysis with a sample of knowledge translation experts.

Methods

Fifty-four experts were recruited to undertake four choice surveys. Respondents were shown combinations of research use subactions called profiles, and rated on a 1 to 9 scale whether each profile represented a limited (1–3), moderate (4–6), or extensive (7–9) example of research use. Generalised Estimating Equations were used to analyse respondents’ choice data, which calculated a utility coefficient for each subaction. A large utility coefficient indicated that a subaction was particularly influential in guiding experts’ ratings of extensive research use.

Results

Utility coefficients were calculated for each subaction, which became the points assigned to the subactions in the scoring system. The following subactions yielded the largest utilities and were regarded as the most important components of each research use domain: using research to directly influence the core of the policy decision; using research to inform alternative perspectives to deal with the policy issue; using research to persuade targeted stakeholders to support a predetermined decision; and using research because it was a mandated requirement by the policymaker’s organisation.

Conclusions

We have generated an empirically derived and context-sensitive means of measuring and scoring the extent to which policymakers used research to inform the development of a policy document. The scoring system can be used by organisations to not only quantify the extent of their research use, but also to provide them with insights into potential strategies to improve subsequent research use.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0022-y) contains supplementary material, which is available to authorized users.

How Health in All Policies are developed and implemented in a developing country? A case study of a HiAP initiative in Iran

Population health is influenced by many factors beyond the control of health system which should be addressed by other sectors through inter-sectoral collaboration (ISC). Countries have adopted diverse initiatives to operationalize ISC for health such as establishment of Councils of Health and Food Security (CHFSs) and development of provincial Health Master Plans (HMPs) in Iran. The literature, however, provides meager information on how these initiatives have been moved into the top policy agenda, how and by whom they have been formulated and what factors enable or inhibit their implementation. In addressing these knowledge gaps, we employed a qualitative case study approach, incorporating mixed methods: in-depth interviews and a textual analysis of policy documents. Iran founded the Supreme Council of Health and Food Security (SCHFS) at national level in 2006 followed by provincial and district CHFSs to ensure political commitment to ISC for health and Health in All Policies (HiAPs). In 2009, the SCHFS mandated all provincial CHFSs across the country to develop provincial HMP to operationalize the HiAP approach and Kerman was among the first provinces which responded to this call. We selected Kerman province HMP as a case study to investigate the research questions raised in this study. The study revealed two types of leverage, which played crucial role in agenda setting, policy formulation and implementation of HMP including politics (political commitment) and policy entrepreneurs. The multiple streams model was found to be informative for thinking about different stages of a policy cycle including agenda setting, policy formulation and policy implementation. It was also found to be a useful framework in analyzing HiAP initiatives as these policies do not smoothly and readily reach the policy agenda.

The barriers and facilitators to the implementation of clinical guidance in elective orthopaedic surgery: a qualitative study protocol

BACKGROUND

Clinical guidelines in orthopaedic surgery aim to improve the efficiency, quality and outcomes of patient care by ensuring that treatment recommendations are based on the best available evidence. The simple provision of guidelines, however, does not ensure fidelity or guarantee their uptake and use in surgical practice. Research exploring the factors that affect surgeons' use of evidence and guidelines has focused on understanding what evidence exists for current clinical decisions. This narrowed scope emphasises the technical, educational and accessibility issues but overlooks wider factors that help explain how and why guidelines are not implemented and used in surgery. It is also important to understand how we can encourage the implementation processes in practice. By taking a social science perspective to examine orthopaedic surgery, we move beyond the narrow focus and explore how and why clinical guidelines struggle to achieve full uptake. We aim to explore guideline uptake to discover the factors that contribute to, or complicate, appropriate implementation in this field. We need to go beyond traditional views and experimental methods to examine the barriers and facilitators of implementation in real-life NHS surgical practice. These could be multifactorial, linked to individual, organisational or contextual influences, which act on the guideline implementation process.

METHODS/DESIGN

We will use ethnographic methods to conduct case studies in three English NHS hospitals. Within each case, we will conduct observations, interviews and analysis of key documents to understand experiences, complex processes and decisions made and the role of clinical guidance and other sources of evidence within orthopaedic surgery. The data will be transcribed and analysed thematically. Comparisons will be made within cases and across cases.

DISCUSSION

Guidelines are a fundamental part of clinical practice, and various factors must be considered when preparing for their successful implementation into organisations. Understanding the views and experiences of a range of surgical, clerical and managerial staff across multiple orthopaedic departments will capture the complexity and variety of factors that can influence surgical decisions. The findings of our study will identify the specific features of orthopaedic practice to help guide the development of strategies to facilitate guideline uptake in everyday surgical work.