Evidence for public health policy on inequalities: 1: the reality according to policymakers

Process, pitfalls and profits: lessons from interviewing New Zealand policy-makers

Little has been written about interviewing policy-makers in health promotion and public health research. This article explores the process, pitfalls and profits of semi-structured interviews with policy-makers in 10 research projects conducted in New Zealand. Key members of each research team were surveyed about their research and findings verified against research publications. Key aspects of the process of policy-maker interviews include gaining ethical approval, navigating gatekeepers, using personal contacts and multiple research dissemination methods. Pitfalls of interviewing policy-makers include interviewers not having enough knowledge of the topic so efforts were made to use knowledgeable researchers or up-skill others. Interviews provide access to specialist knowledge of the policy process which cannot be obtained by other methods. While this study was conducted in one jurisdiction, it has implications for other countries. Effective policy-maker interviews in health promotion policy research could contribute to improvements in the quality of data collected and uptake of research by policy-makers.

Organisational capacity and its relationship to research use in six Australian health policy agencies

There are calls for policymakers to make greater use of research when formulating policies. Therefore, it is important that policy organisations have a range of tools and systems to support their staff in using research in their work. The aim of the present study was to measure the extent to which a range of tools and systems to support research use were available within six Australian agencies with a role in health policy, and examine whether this was related to the extent of engagement with, and use of research in policymaking by their staff. The presence of relevant systems and tools was assessed via a structured interview called ORACLe which is conducted with a senior executive from the agency. To measure research use, four policymakers from each agency undertook a structured interview called SAGE, which assesses and scores the extent to which policymakers engaged with (i.e., searched for, appraised, and generated) research, and used research in the development of a specific policy document. The results showed that all agencies had at least a moderate range of tools and systems in place, in particular policy development processes; resources to access and use research (such as journals, databases, libraries, and access to research experts); processes to generate new research; and mechanisms to establish relationships with researchers. Agencies were less likely, however, to provide research training for staff and leaders, or to have evidence-based processes for evaluating existing policies. For the majority of agencies, the availability of tools and systems was related to the extent to which policymakers engaged with, and used research when developing policy documents. However, some agencies did not display this relationship, suggesting that other factors, namely the organisation's culture towards research use, must also be considered.

Does engagement predict research use? An analysis of The Conversation Annual Survey 2016

The impact of research on the world beyond academia has increasingly become an area of focus in research performance assessments internationally. Impact assessment is expected to incentivise researchers to increase engagement with industry, government and the public more broadly. Increased engagement is in turn expected to increase translation of research so decision-makers can use research to inform development of policies, programs, practices, processes, products, and other mechanisms, through which impact can be realised. However, research has shown that various factors affect research use, and evidence on 'what works' to increase decision-makers' use of research is limited. The Conversation is an open access research communication platform, published under Creative Commons licence, which translates research into news articles to engage a general audience, aiming to improve understanding of current issues and complex social problems. To identify factors that predict use of academic research and expertise reported in The Conversation, regression analyses were performed using The Conversation Australia 2016 Annual Survey data. A broad range of factors predicted use, with engagement actions being the most common. Interestingly, different types of engagement actions predicted different types of use. This suggests that to achieve impact through increased engagement, a deeper understanding of how and why different engagement actions elicit different types of use is needed. Findings also indicate The Conversation is overcoming some of the most commonly identified barriers to the use of research: access, relevance, actionable outcomes, and timeliness. As such, The Conversation offers an effective model for providing access to and communicating research in a way that enables use, a necessary precursor to achieving research impact.

Competing for space in an already crowded market: a mixed methods study of why an online community of practice (CoP) for alcohol harm reduction failed to generate interest amongst the group of public health professionals at which it was aimed

BACKGROUND

Improving mechanisms for knowledge translation (KT) and connecting decision-makers to each other and the information and evidence they consider relevant to their work remains a priority for public health. Virtual communities of practices (CoPs) potentially offer an affordable and flexible means of encouraging connection and sharing of evidence, information and learning among the public health community in ways that transgress traditional geographical, professional, institutional and time boundaries. The suitability of online CoPs in public health, however, has rarely been tested. This paper explores the reasons why particular online CoP for alcohol harm reduction hosted by the UK Health Forum failed to generate sufficient interest from the group of public health professionals at which it was aimed.

METHODS

The study utilises online web-metrics demonstrating a lack of online activity on the CoP. One hundred and twenty seven responses to an online questionnaire were used to explore whether the lack of activity could be explained by the target audience's existing information and evidence practices and needs. Qualitative interviews with 10 members describe in more detail the factors that shape and inhibit use of the virtual CoP by those at which it was targeted.

RESULTS

Quantitative and qualitative data confirm that the target audience had an interest in the kind of information and evidence the CoP was set up to share and generate discussion about, but also that participants considered themselves to already have relatively good access to the information and evidence they needed to inform their work. Qualitative data revealed that the main barriers to using the CoP were a proliferation of information sources meaning that participants preferred to utilise trusted sources that were already established within their daily routines and a lack of time to engage with new online tools that required any significant commitment.

CONCLUSIONS

Specialist online CoPs are competing for space in an already crowded market. A target audience that regards itself as busy and over-supplied is unlikely to commit to a new service without the assurance that the service will provide unique and valuable well-summarised information, which would reduce the need to spend time accessing competing resources.

Academic advocacy in public health: Disciplinary 'duty' or political 'propaganda'?

The role of 'advocacy' within public health attracts considerable debate but is rarely the subject of empirical research. This paper reviews the available literature and presents data from qualitative research (interviews and focus groups conducted in the UK in 2011-2013) involving 147 professionals (working in academia, the public sector, the third sector and policy settings) concerned with public health in the UK. It seeks to address the following questions: (i) What is public health advocacy and how does it relate to research?; (ii) What role (if any) do professionals concerned with public health feel researchers ought to play in advocacy?; and (iii) For those researchers who do engage in advocacy, what are the risks and challenges and to what extent can these be managed/mitigated? In answering these questions, we argue that two deeply contrasting conceptualisations of 'advocacy' exist within public health, the most dominant of which ('representational') centres on strategies for 'selling' public health goals to decision-makers and the wider public. This contrasts with an alternative (less widely employed) conceptualisation of advocacy as 'facilitational'. This approach focuses on working with communities whose voices are often unheard/ignored in policy to enable their views to contribute to debates. We argue that these divergent ways of thinking about advocacy speak to a more fundamental challenge regarding the role of the public in research, policy and practice and the activities that connect these various strands of public health research.

Considerations and guidance in designing equity-relevant clinical trials

Health research has documented disparities in health and health outcomes within and between populations. When these disparities are unfair and avoidable they may be referred to as health inequities. Few trials attend to factors related to health inequities, and there is limited understanding about how to build consideration of health inequities into trials. Due consideration of health inequities is important to inform the design, conduct and reporting of trials so that research can build evidence to more effectively address health inequities and importantly, ensure that inequities are not aggravated. In this paper, we discuss approaches to integrating health equity-considerations in randomized trials by using the PROGRESS Plus framework (Place of residence, Race/ethnicity/culture/language, Occupation, Gender, Religion, Education, Socio-economic status, Social capital and "Plus" that includes other context specific factors) and cover: (i) formulation of research questions, (ii) two specific scenarios relevant to trials about health equity and (iii) describe how the PROGRESS Plus characteristics may influence trial design, conduct and analyses. This guidance is intended to support trialists designing equity-relevant trials and lead to better design, conduct, analyses and reporting, by addressing two main issues: how to avoid aggravating inequity among research participants and how to produce information that is useful to decision-makers who are concerned with health inequities.

Reversing the pipeline? Implementing public health evidence-based guidance in english local government

BACKGROUND

In the UK, responsibility for many public health functions was transferred in 2013 from the National Health Service (NHS) to local government; a very different political context and one without the NHS history of policy and practice being informed by evidence-based guidelines. A problem this move presented was whether evidence-based guidelines would be seen as relevant, useful and implementable within local government. This study investigates three aspects of implementing national evidence-based recommendations for public health within a local government context: influences on implementation, how useful guidelines are perceived to be and whether the linear evidence-guidelines-practice model is considered relevant.

METHODS

Thirty-one councillors, public health directors and deputy directors and officers and other local government employees were interviewed about their experiences implementing evidence-based guidelines. Interviews were informed and analysed using a theoretical model of behaviour (COM-B; Capability, Opportunity, Motivation-Behaviour).

RESULTS

Contextual issues such as budget, capacity and political influence were important influences on implementation. Guidelines were perceived to be of limited use, with concerns expressed about recommendations being presented in the abstract, lacking specificity and not addressing the complexity of situations or local variations. Local evidence was seen as the best starting point, rather than evidence-based guidance produced by the traditional linear 'evidence-guidelines-practice' model. Local evidence was used to not only provide context for recommendations but also replace recommendations when they conflicted with local evidence.

CONCLUSIONS

Local government users do not necessarily consider national guidelines to be fit for purpose at local level, with the consequence that local evidence tends to trump evidence-based guidelines. There is thus a tension between the traditional model of guideline development and the needs of public health decision-makers and practitioners working in local government. This tension needs to be addressed to facilitate implementation. One way this might be achieved, and participants supported this approach, would be to reverse or re-engineer the traditional pipeline of guideline development by starting with local need and examples of effective local practice rather than starting with evidence of effectiveness synthesised from the international scientific literature. Alternatively, and perhaps in addition, training about the relevance of research evidence should become a routine for local government staff and councillors.

L’évaluation d’impact sur la santé en Montérégie : un processus appuyé sur le courtage de connaissances

Le présent article fait état du modèle de soutien à la prise de décision appliqué à l’évaluation d’impact sur la santé (EIS) en Montérégie, au Canada. Pour répondre à la volonté de soutenir l’élaboration de politiques et projets municipaux favorables à la santé et fondés sur les données probantes, la Direction de santé publique a mis en œuvre une démarche d’EIS dont les fondements pratiques s’inspirent des expériences internationales en matière d’EIS et dont les assises théoriques reposent sur le partage et le courtage de connaissances. L’expérience montérégienne démontre que la stratégie de courtage de connaissances appliquées à l’EIS encourage un partage respectueux des responsabilités et rôles de chacune des parties prenantes et favorise l’utilisation des connaissances de santé publique dans la prise de décision municipale. Le courtage des connaissances permet de prendre en compte les enjeux locaux dans les analyses d’impact et dans les recommandations aux décideurs. En conclusion, il s’avère que l’implication des décideurs municipaux à la recherche de solutions favorables à la santé permet d’inscrire les données probantes à l’intérieur des processus décisionnels et d’observer leur utilisation à plusieurs niveaux. L’évaluation des premières expériences d’EIS révèle en effet que les connaissances partagées répondent à des besoins immédiats engendrant leur utilisation directe, contribuent à des changements de perception et de pratique à plus large échelle et font de la santé une valeur reconnue par les décideurs publics et promue auprès de leurs citoyens pour accroître l’acceptabilité des initiatives municipales.

Policymaker experiences with rapid response briefs to address health-system and technology questions in Uganda

Background

Health service and systems researchers have developed knowledge translation strategies to facilitate the use of reliable evidence for policy, including rapid response briefs as timely and responsive tools supporting decision making. However, little is known about users’ experience with these newer formats for presenting evidence. We sought to explore Ugandan policymakers’ experience with rapid response briefs in order to develop a format acceptable for policymakers.

Methods

We used existing research regarding evidence formats for policymakers to inform the initial version of rapid response brief format. We conducted user testing with healthcare policymakers at various levels of decision making in Uganda, employing a concurrent think-aloud method, collecting data on elements including usability, usefulness, understandability, desirability, credibility and value of the document. We modified the rapid response briefs format based on the results of the user testing and sought feedback on the new format.

Results

The participants generally found the format of the rapid response briefs usable, credible, desirable and of value. Participants expressed frustrations regarding several aspects of the document, including the absence of recommendations, lack of clarity about the type of document and its potential uses (especially for first time users), and a crowded front page. Participants offered conflicting feedback on preferred length of the briefs and use and placement of partner logos. Users had divided preferences for the older and newer formats.

Conclusion

Although the rapid response briefs were generally found to be of value, there are major and minor frustrations impeding an optimal user experience. Areas requiring further research include how to address policymakers’ expectations of recommendations in these briefs and their optimal length.

Can health indicators help policy-makers? Experience from European system of urban health indicators (EURO-URHIS)

Background

This paper proposes that Population Impact Measures (PIMs), the Population Impact Number of Eliminating a Risk Factor over a time period (PIN-ER-t) and the number of events prevented in your population (NEPP), can assist in policy making as they include relevant information which describes the impact or benefits to the population of risk factors and interventions. In this study, we explore the utilization of the indicators from European System of Urban Health Indicators System to produce the two PIMs. We identified from the indicators list the health determinants, health status and health interventions which can be linked, and searched Medline for evidence of association. We then investigated whether the type of frequency measure available for the indicator match with the measure used in PIMs, and explored data availability for the City of Manchester (UK) as an urban area. Of the 39 indicators relevant to socio-economic factors, health determinants and health status, it was possible to calculate the population impact of a risk factor, i.e. the PIN-ER-t, for only six associations, and the population impact of health interventions, i.e. NEPP, for only one out of the three listed indicators, as the relevant health conditions were not included. The results of this study suggest that if an indicator system is intended to play a part in the policy making process, then the method of presentation to policy-makers should be decided before setting up the system, as it is likely that some indicators which would be essential might not be available.

Determinants of evidence use in public health policy making: Results from a study across six EU countries

The knowledge-practice gap in public health is widely known. The importance of using different types of evidence for the development of effective health promotion has also been emphasized. Nevertheless, in practice, intervention decisions are often based on perceived short-term opportunities, lacking the most effective approaches, thus limiting the impact of health promotion strategies. This article focuses on facilitators and barriers in the use of evidence in developing health enhancing physical activity policies. Data was collected in 2012 by interviewing 86 key stakeholders from six EU countries (FI, DK, UK, NL, IT, RO) using a common topic guide. Content analysis and concept mapping was used to construct a map of facilitators and barriers. Barriers and facilitators experienced by most stakeholders and policy context in each country are analysed. A lack of locally useful and concrete evidence, evidence on costs, and a lack of joint understanding were specific hindrances. Also users' characteristics and the role media play were identified as factors of influence. Attention for individual and social factors within the policy context might provide the key to enhance more sustainable evidence use. Developing and evaluating tailored approaches impacting on networking, personal relationships, collaboration and evidence coproduction is recommended.

Use of science to guide city planning policy and practice: how to achieve healthy and sustainable future cities

Land-use and transport policies contribute to worldwide epidemics of injuries and non-communicable diseases through traffic exposure, noise, air pollution, social isolation, low physical activity, and sedentary behaviours. Motorised transport is a major cause of the greenhouse gas emissions that are threatening human health. Urban and transport planning and urban design policies in many cities do not reflect the accumulating evidence that, if policies would take health effects into account, they could benefit a wide range of common health problems. Enhanced research translation to increase the influence of health research on urban and transport planning decisions could address many global health problems. This paper illustrates the potential for such change by presenting conceptual models and case studies of research translation applied to urban and transport planning and urban design. The primary recommendation of this paper is for cities to actively pursue compact and mixed-use urban designs that encourage a transport modal shift away from private motor vehicles towards walking, cycling, and public transport. This Series concludes by urging a systematic approach to city design to enhance health and sustainability through active transport and a move towards new urban mobility. Such an approach promises to be a powerful strategy for improvements in population health on a permanent basis.

The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review

Background

Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which make them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision makers. This systematic review aimed to (1) assess the effectiveness of evidence summaries on policymakers’ use of the evidence and (2) identify the most effective summary components for increasing policymakers’ use of the evidence. We present an overview of the available evidence on systematic review derivative products.

Methods

We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of “evidence summary,” “policy brief,” or other products derived from systematic reviews that presented evidence in a summarized form. The primary outcomes were the (1) use of systematic review summaries in decision-making (e.g., self-reported use of the evidence in policymaking and decision-making) and (2) policymakers’ understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g., format) of the summaries.

Results

Our database search combined with our gray literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text, and we included six studies (reported in seven papers) as well as protocols from two ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (four studies), and perceived usefulness or usability (three studies). Summary of findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for summary of findings tables, certain elements, such as reporting study event rates and absolute differences, were preferred as well as avoiding the use of footnotes.

Conclusions

Evidence summaries are likely easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Trial registration

The protocol was published in the journal Systematic Reviews (2015;4:122)

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0530-3) contains supplementary material, which is available to authorized users.

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists

BACKGROUND

Currently, little is known about the types of evidence used by policy makers. This study aimed to investigate how policy makers in the health domain use and evaluate evidence and how this differs from academic epidemiologists. By having a better understanding of how policy makers select, evaluate, and use evidence, academics can tailor the way in which that evidence is produced, potentially leading to more effective knowledge translation.

METHODS

An exploratory mixed-methods study design was used. Quantitative measures were collected via an anonymous online survey (n = 28), with sampling from three health-related government and non-government organizations. Semi-structured interviews with policy makers (n = 20) and epidemiologists (n = 6) were conducted to gather qualitative data.

RESULTS

Policy makers indicated systematic reviews were the preferred research resource (19%), followed closely by qualitative research (16%). Neither policy makers nor epidemiologists used grading instruments to evaluate evidence. In the web survey, policy makers reported that consistency and strength of evidence (93%), the quality of data (93%), bias in the evidence (79%), and recency of evidence (79%) were the most important factors taken into consideration when evaluating the available evidence. The same results were found in the qualitative interviews. Epidemiologists focused on the methodology used in the study. The most cited barriers to using robust evidence, according to policy makers, were political considerations (60%), time limitations (55%), funding (50%), and research not being applicable to current policies (50%).

CONCLUSION

The policy maker's investigation did not report a systematic approach to evaluating evidence. Although there was some overlap between what policy makers and epidemiologists identified as high-quality evidence, there was also some important differences. This suggests that the best scientific evidence may not routinely be used in the development of policy. In essence, the policy-making process relied on other jurisdictions' policies and the opinions of internal staff members as primary evidence sources to inform policy decisions. Findings of this study suggest that efforts should be directed toward making scientific information more systematically available to policy makers.

Consideration of health inequalities in systematic reviews: a mapping review of guidance

BACKGROUND

Given that we know that interventions shown to be effective in improving the health of a population may actually widen the health inequalities gap while others reduce it, it is imperative that all systematic reviewers consider how the findings of their reviews may impact (reduce or increase) on the health inequality gap. This study reviewed existing guidance on incorporating considerations of health inequalities in systematic reviews in order to examine the extent to which they can help reviewers to incorporate such issues.

METHODS

A mapping review was undertaken to identify guidance documents that purported to inform reviewers on whether and how to incorporate considerations of health inequalities. Searches were undertaken in Medline, CINAHL and The Cochrane Library Methodology Register. Review guidance manuals prepared by international organisations engaged in undertaking systematic reviews, and their associated websites were scanned. Studies were included if they provided an overview or discussed the development and testing of guidance for dealing with the incorporation of considerations of health inequalities in evidence synthesis. Results are summarised in narrative and tabular forms.

RESULTS

Twenty guidance documents published between 2009 and 2016 were included. Guidance has been produced to inform considerations of health inequalities at different stages of the systematic review process. The Campbell and Cochrane Equity Group have been instrumental in developing and promoting such guidance. Definitions of health inequalities and guidance differed across the included studies. All but one guidance document were transparent in their method of production. Formal methods of evaluation were reported for six guidance documents. Most of the guidance was operationalised in the form of examples taken from published systematic reviews. The number of guidance items to operationalise ranges from 3 up to 26 with a considerable overlap noted.

CONCLUSIONS

Adhering to the guidance will require more work for the reviewers. It requires a deeper understanding of how reviewers can operationalise the guidance taking into consideration the barriers and facilitators involved. This has implications not only for understanding the usefulness and burden of the guidance but also for the uptake of guidance and its ultimate goal of improving health inequalities considerations in systematic reviews.

Acceptability of financial incentives for health behaviour change to public health policymakers: a qualitative study

Background

Providing financial incentives contingent on healthy behaviours is one way to encourage healthy behaviours. However, there remains substantial concerns with the acceptability of health promoting financial incentives (HPFI). Previous research has studied acceptability of HPFI to the public, recipients and practitioners. We are not aware of any previous work that has focused particularly on the views of public health policymakers.

Our aim was to explore the views of public health policymakers on whether or not HPFI are acceptable; and what, if anything, could be done to maximise acceptability of HPFI.

Methods

We recruited 21 local, regional and national policymakers working in England via gatekeepers and snowballing. We conducted semi-structured in-depth interviews with participants exploring experiences of, and attitudes towards, HPFI. We analysed data using the Framework approach.

Results

Public health policymakers working in England acknowledged that HPFI could be a useful behaviour change tool, but were not overwhelmingly supportive of them. In particular, they raised concerns about effectiveness and cost-effectiveness, potential ‘gaming’, and whether or not HPFI address the underlying causes of unhealthy behaviours. Shopping voucher rewards, of smaller value, targeted at deprived groups were particularly acceptable to policymakers. Participants were particularly concerned about the response of other stakeholders to HPFI – including the public, potential recipients, politicians and the media.

Overall, the interviews reflected three tensions. Firstly, a tension between wanting to trust individuals and promote responsibility; and distrust around the potential for ‘gaming the system’. Secondly, a tension between participants’ own views about HPFI; and their concerns about the possible views of other stakeholders. Thirdly, a tension between participants’ personal distaste of HPFI; and their professional view that they could be a valuable behaviour change tool.

Conclusions

There are aspects of design that influence acceptability of financial incentive interventions to public health policymakers. However, it is not clear that even interventions designed to maximise acceptability would be acceptable enough to be recommended for implementation. Further work may be required to help policymakers understand the potential responses of other stakeholder groups to financial incentive interventions.

Learning from doing: the case for combining normalisation process theory and participatory learning and action research methodology for primary healthcare implementation research

Background

The implementation of research findings is not a straightforward matter. There are substantive and recognised gaps in the process of translating research findings into practice and policy. In order to overcome some of these translational difficulties, a number of strategies have been proposed for researchers. These include greater use of theoretical approaches in research focused on implementation, and use of a wider range of research methods appropriate to policy questions and the wider social context in which they are placed. However, questions remain about how to combine theory and method in implementation research. In this paper, we respond to these proposals.

Discussion

Focussing on a contemporary social theory, Normalisation Process Theory, and a participatory research methodology, Participatory Learning and Action, we discuss the potential of their combined use for implementation research. We note ways in which Normalisation Process Theory and Participatory Learning and Action are congruent and may therefore be used as heuristic devices to explore, better understand and support implementation. We also provide examples of their use in our own research programme about community involvement in primary healthcare.

Conclusions

Normalisation Process Theory alone has, to date, offered useful explanations for the success or otherwise of implementation projects post-implementation. We argue that Normalisation Process Theory can also be used to prospectively support implementation journeys. Furthermore, Normalisation Process Theory and Participatory Learning and Action can be used together so that interventions to support implementation work are devised and enacted with the expertise of key stakeholders. We propose that the specific combination of this theory and methodology possesses the potential, because of their combined heuristic force, to offer a more effective means of supporting implementation projects than either one might do on its own, and of providing deeper understandings of implementation contexts, rather than merely describing change.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1587-z) contains supplementary material, which is available to authorized users.

Similarities and differences between stakeholders' opinions on using Health Technology Assessment (HTA) information across five European countries: results from the EQUIPT survey

Background

The European-study on Quantifying Utility of Investment in Protection from Tobacco (EQUIPT) project aimed to study transferability of economic evidence by co-creating the Tobacco Return On Investment (ROI) tool, previously developed in the United Kingdom, for four sample countries (Germany, Hungary, Spain and the Netherlands). The EQUIPT tool provides policymakers and stakeholders with customized information about the economic and wider returns on the investment in evidence-based tobacco control, including smoking cessation interventions. A Stakeholder Interview Survey was developed to engage with the stakeholders in early phases of the development and country adaptation of the ROI tool. The survey assessed stakeholders’ information needs, awareness about underlying principles used in economic analyses, opinion about the importance, effectiveness and cost-effectiveness of tobacco control interventions, and willingness to use a Health Technology Assessment (HTA) tool such as the ROI tool.

Methods

A cross sectional study using a mixed method approach was conducted among participating stakeholders in the sample countries and the United Kingdom. The individual questionnaire contained open-ended questions as well as single choice and 7- or 3-point Likert-scale questions. The results corresponding to the priority and needs assessment and to the awareness of stakeholders about underlying principles used in economic analysis are analysed by country and stakeholder categories.

Results

Stakeholders considered it important that the decisions on the investments in tobacco control interventions should be supported by scientific evidence, including prevalence of smoking, cost of smoking, quality of life, mortality due to smoking, and effectiveness, cost-effectiveness and budget impact of smoking cessation interventions. The proposed ROI tool was required to provide this granularity of information. The majority of the stakeholders were aware of the general principles of economic analyses used in decision making contexts but they did not appear to have in-depth knowledge about specific technical details. Generally, stakeholders’ answers showed larger variability by country than by stakeholder category.

Conclusions

Stakeholders across different European countries viewed the use of HTA evidence to be an important factor in their decision-making process. Further, they considered themselves to be capable of interpreting the results from a ROI tool and were highly motivated to use it.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-016-0110-7) contains supplementary material, which is available to authorized users.

Reducing the Strength: a mixed methods evaluation of alcohol retailers' willingness to voluntarily reduce the availability of low cost, high strength beers and ciders in two UK local authorities

BACKGROUND

Reducing the Strength is an increasingly popular intervention in which local authorities ask retailers to stop selling 'super-strength' beers and ciders. The intervention cannot affect alcohol availability, nor consumption, unless retailers participate. In this paper, we ask whether and why retailers choose or refuse to self-impose restrictions on alcohol sales in this way.

METHODS

Mixed method assessment of retailers' participation in Reducing the Strength in two London (UK) local authorities. Compliance rates and the cheapest available unit of alcohol at each store were assessed. Qualitative interviews with retailer managers and staff (n = 39) explored attitudes towards the intervention and perceptions of its impacts.

RESULTS

Shops selling super-strength across both areas fell from 78 to 25 (18 % of all off-licences). The median price of the cheapest unit of alcohol available across all retailers increased from £0.29 to £0.33 and in shops that participated in Reducing the Strength it rose from £0.33 to £0.43. The project received a mixed response from retailers. Retailers said they participated to deter disruptive customers, reduce neighbourhood disruptions and to maintain a good relationship with the local authority. Reducing the Strength participants and non-participants expressed concern about its perceived financial impact due to customers shopping elsewhere for super-strength. Some felt that customers' ability to circumvent the intervention would limit its effectiveness and that a larger scale compulsory approach would be more effective.

CONCLUSIONS

Reducing the Strength can achieve high rates of voluntary compliance, reduce availability of super-strength and raise the price of the cheapest available unit of alcohol in participating shops. Questions remain over the extent to which voluntary interventions of this type can achieve wider social or health goals if non-participating shops attract customers from those who participate.

The ambition of Health in All Policies in Norway: The role of political leadership and bureaucratic change

This paper presents and discusses status, challenges and future developments of Health in All Policies (HiAP) in Norway. Within the frames of the identified challenge of creating coordinated and durable policies and practices in local government, it discusses The Norwegian HiAP policy. More specifically, the paper identifies status and challenges for instituting firmer political and administrative attention to population health and health equity across administrative sectors and levels, and discusses how national authorities may stimulate more coordinated and durable HiAP policies and practices in the future.

Evidence for Health III: Making evidence-informed decisions that integrate values and context

Making evidence-informed decisions with the aim of improving the health of individuals or populations can be facilitated by using a systematic approach. While a number of algorithms already exist, and while there is no single ‘right’ way of summarizing or ordering the various elements that should be involved in making such health-related decisions, an algorithm is presented here that lays out many of the key issues that should be considered, and which adds a special emphasis on balancing the values of individual patients and entire populations, as well as the importance of incorporating contextual considerations. Indeed many different types of evidence and value judgements are needed during the decision-making process to answer a wide range of questions, including (1) What is the priority health problem? (2) What causes this health problem? (3) What are the different strategies or interventions that can be used to address this health problem? (4) Which of these options, as compared to the status quo, has an added benefit that outweighs the harms? (5) Which options would be acceptable to the individuals or populations involved? (6) What are the costs and opportunity costs? (7) Would these options be feasible and sustainable in this specific context? (8) What are the ethical, legal and social implications of choosing one option over another? (9) What do different stakeholders stand to gain or lose from each option? and (10) Taking into account the multiple perspectives and considerations involved, which option is most likely to improve health while minimizing harms? This third and final article in the ‘Evidence for Health’ series will go through each of the steps in the algorithm in greater detail to promote more evidence-informed decisions that aim to improve health and reduce inequities.

Desperately seeking reductions in health inequalities: perspectives of UK researchers on past, present and future directions in health inequalities research

Following government commitments to reducing health inequalities from 1997 onwards, the UK has been recognised as a global leader in health inequalities research and policy. Yet health inequalities have continued to widen by most measures, prompting calls for new research agendas and advocacy to facilitate greater public support for the upstream policies that evidence suggests are required. However, there is currently no agreement as to what new research might involve or precisely what public health egalitarians ought to be advocating. This article presents an analysis of discussions among 52 researchers to consider the feasibility that research-informed advocacy around particular solutions to health inequalities may emerge in the UK. The data indicate there is a consensus that more should be been done to learn from post-1997 efforts to reduce health inequalities, and an obvious desire to provide clearer policy guidance in future.However, discussions as to where researchers should now focus their efforts and with whom researchers ought to be engaging reveal three distinct ways of approaching health inequalities, each of which has its own epistemological foundations. Such differences imply that a consensus on reducing health inequalities is unlikely to materialise. Instead, progress seems most likely if all three approaches are simultaneously enabled.

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy

Background

Evidence-informed policymaking is more likely if organisations have cultures that promote research use and invest in resources that facilitate staff engagement with research. Measures of organisations’ research use culture and capacity are needed to assess current capacity, identify opportunities for improvement, and examine the impact of capacity-building interventions. The aim of the current study was to develop a comprehensive system to measure and score organisations’ capacity to engage with and use research in policymaking, which we entitled ORACLe (Organisational Research Access, Culture, and Leadership).

Method

We used a multifaceted approach to develop ORACLe. Firstly, we reviewed the available literature to identify key domains of organisational tools and systems that may facilitate research use by staff. We interviewed senior health policymakers to verify the relevance and applicability of these domains. This information was used to generate an interview schedule that focused on seven key domains of organisational capacity. The interview was pilot-tested within four Australian policy agencies. A discrete choice experiment (DCE) was then undertaken using an expert sample to establish the relative importance of these domains. This data was used to produce a scoring system for ORACLe.

Results

The ORACLe interview was developed, comprised of 23 questions addressing seven domains of organisational capacity and tools that support research use, including (1) documented processes for policymaking; (2) leadership training; (3) staff training; (4) research resources (e.g. database access); and systems to (5) generate new research, (6) undertake evaluations, and (7) strengthen relationships with researchers. From the DCE data, a conditional logit model was estimated to calculate total scores that took into account the relative importance of the seven domains. The model indicated that our expert sample placed the greatest importance on domains (2), (3) and (4).

Conclusion

We utilised qualitative and quantitative methods to develop a system to assess and score organisations’ capacity to engage with and apply research to policy. Our measure assesses a broad range of capacity domains and identifies the relative importance of these capacities. ORACLe data can be used by organisations keen to increase their use of evidence to identify areas for further development.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0069-9) contains supplementary material, which is available to authorized users.

A review of policy dissemination and implementation research funded by the National Institutes of Health, 2007-2014

BACKGROUND

Policy has a tremendous potential to improve population health when informed by research evidence. Such evidence, however, typically plays a suboptimal role in policymaking processes. The field of policy dissemination and implementation research (policy D&I) exists to address this challenge. The purpose of this study was to: (1) determine the extent to which policy D&I was funded by the National Institutes of Health (NIH), (2) identify trends in NIH-funded policy D&I, and (3) describe characteristics of NIH-funded policy D&I projects.

METHODS

The NIH Research Portfolio Online Reporting Tool was used to identify all projects funded through D&I-focused funding announcements. We screened for policy D&I projects by searching project title, abstract, and term fields for mentions of "policy," "policies," "law," "legal," "legislation," "ordinance," "statute," "regulation," "regulatory," "code," or "rule." A project was classified as policy D&I if it explicitly proposed to conduct research about the content of a policy, the process through which it was developed, or outcomes it produced. A coding guide was iteratively developed, and all projects were independently coded by two researchers. ClinicalTrials.gov and PubMed were used to obtain additional project information and validate coding decisions. Descriptive statistics--stratified by funding mechanism, Institute, and project characteristics--were produced.

RESULTS

Between 2007 and 2014, 146 projects were funded through the D&I funding announcements, 12 (8.2 %) of which were policy D&I. Policy D&I funding totaled $16,177,250, equivalent to 10.5 % of all funding through the D&I funding announcements. The proportion of funding for policy D&I projects ranged from 14.6 % in 2007 to 8.0 % in 2012. Policy D&I projects were primarily focused on policy outcomes (66.7 %), implementation (41.7 %), state-level policies (41.7 %), and policies within the USA (83.3 %). Tobacco (33.3 %) and cancer (25.0 %) control were the primary topics of focus. Many projects combined survey (58.3 %) and interview (33.3 %) methods with analysis of archival data sources.

CONCLUSIONS

NIH has made an initial investment in policy D&I research, but the level of support has varied between Institutes. Policy D&I researchers have utilized a variety of designs, methods, and data sources to investigate the development processes, content, and outcomes of public and private policies.

Assessing the usefulness of systematic reviews for policymakers in public health: A case study of overweight and obesity prevention interventions

OBJECTIVE

Systematic reviews (SRs) should include policy-relevant information in order to more readily inform policy and practice. We investigated whether SRs of overweight and obesity prevention interventions are framed in such a way that maximises their usefulness for policymakers.

METHOD

We conducted a systematic review of SRs of overweight and obesity prevention interventions published in 4 databases any time up to December 2014. We analysed the SRs for their usefulness to policymakers, using a coding frame developed based on literature around what policymakers want and need from systematic reviews. Systematic reviews were assessed for a) policy links and framing; b) quality assessment and conflict of interest statements: and c) discussion of policy implications.

RESULTS

Of the 153 SRs that met the inclusion criteria, very few (7%) had authors from policy-based organisations, 48% had funding from such organisations, and almost a third (31%) framed their introduction or aims around policy. Most (69%) discussed issues affecting generalisability of the SR findings but only a quarter (24%) discussed cost or cost-effectiveness of the intervention under investigation. Less than a third (29%) of SRs discussed the policy implications of their findings. SRs that were policy-framed were significantly more likely to discuss costs (PR=1.8, 95%CI 1.0-3.0) and policy implications (PR=2.5, 95%CI 1.5-4.0).

CONCLUSION

SRs should discuss the policy and practice implications of their findings to maximise the influence of SRs on policy making. It is recommended that SR guidelines are updated to include generalisability and discussion of policy and practice implications as a requirement.

Protocol for the development of a CONSORT-equity guideline to improve reporting of health equity in randomized trials

BACKGROUND

Health equity concerns the absence of avoidable and unfair differences in health. Randomized controlled trials (RCTs) can provide evidence about the impact of an intervention on health equity for specific disadvantaged populations or in general populations; this is important for equity-focused decision-making. Previous work has identified a lack of adequate reporting guidelines for assessing health equity in RCTs. The objective of this study is to develop guidelines to improve the reporting of health equity considerations in RCTs, as an extension of the Consolidated Standards of Reporting Trials (CONSORT).

METHODS/DESIGN

A six-phase study using integrated knowledge translation governed by a study executive and advisory board will assemble empirical evidence to inform the CONSORT-equity extension. To create the guideline, the following steps are proposed: (1) develop a conceptual framework for identifying "equity-relevant trials," (2) assess empirical evidence regarding reporting of equity-relevant trials, (3) consult with global methods and content experts on how to improve reporting of health equity in RCTs, (4) collect broad feedback and prioritize items needed to improve reporting of health equity in RCTs, (5) establish consensus on the CONSORT-equity extension: the guideline for equity-relevant trials, and (6) broadly disseminate and implement the CONSORT-equity extension.

DISCUSSION

This work will be relevant to a broad range of RCTs addressing questions of effectiveness for strategies to improve practice and policy in the areas of social determinants of health, clinical care, health systems, public health, and international development, where health and/or access to health care is a primary outcome. The outcomes include a reporting guideline (CONSORT-equity extension) for equity-relevant RCTs and a knowledge translation strategy to broadly encourage its uptake and use by journal editors, authors, and funding agencies.

Extending the PRISMA statement to equity-focused systematic reviews (PRISMA-E 2012): explanation and elaboration

BACKGROUND

The promotion of health equity, the absence of avoidable and unfair differences in health outcomes, is a global imperative. Systematic reviews are an important source of evidence for health decision-makers, but have been found to lack assessments of the intervention effects on health equity. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) is a 27 item checklist intended to improve transparency and reporting of systematic reviews. We developed an equity extension for PRISMA (PRISMA-E 2012) to help systematic reviewers identify, extract, and synthesise evidence on equity in systematic reviews.

METHODS AND FINDINGS

In this explanation and elaboration paper we provide the rationale for each extension item. These items are additions or modifications to the existing PRISMA Statement items, in order to incorporate a focus on equity. An example of good reporting is provided for each item as well as the original PRISMA item.

CONCLUSIONS

This explanation and elaboration document is intended to accompany the PRISMA-E 2012 Statement and the PRISMA Statement to improve understanding of the reporting guideline for users. The PRISMA-E 2012 reporting guideline is intended to improve transparency and completeness of reporting of equity-focused systematic reviews. Improved reporting can lead to better judgement of applicability by policy makers which may result in more appropriate policies and programs and may contribute to reductions in health inequities. To encourage wide dissemination of this article it is accessible on the International Journal for Equity in Health, Journal of Clinical Epidemiology, and Journal of Development Effectiveness web sites.

Do evidence summaries increase policy-makers' use of evidence from systematic reviews: A systematic review protocol

BACKGROUND

Systematic reviews are important for decision-makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. There are many organizations that develop and disseminate derivative products, such as evidence summaries, from systematic reviews for different populations or subsets of decision-makers. This systematic review will assess the effectiveness of systematic review summaries on increasing policymakers' use of systematic review evidence and to identify the components or features of these summaries that are most effective.

METHODS/DESIGN

We will include studies of policy-makers at all levels as well as health-system managers. We will include studies examining any type of "evidence summary," "policy brief," or other products derived from systematic reviews that present evidence in a summarized form. The primary outcomes are the following: (1) use of systematic review summaries decision-making (e.g., self-reported use of the evidence in policy-making, decision-making) and (2) policy-maker understanding, knowledge, and/or beliefs (e.g., changes in knowledge scores about the topic included in the summary). We will conduct a systematic review of randomized controlled trials (RCTs), non-randomized controlled trials (NRCTs), controlled before-after studies (CBA), and interrupted time series (ITS) studies.

DISCUSSION

The results of this review will inform the development of future systematic review summaries to ensure that systematic review evidence is accessible to and used by policy-makers making health-related decisions.

Taking knowledge users' knowledge needs into account in health: an evidence synthesis framework

The increased demand for evidence-based practice in health policy in recent years has provoked a parallel increase in diverse evidence-based outputs designed to translate knowledge from researchers to policy makers and practitioners. Such knowledge translation ideally creates user-friendly outputs, tailored to meet information needs in a particular context for a particular audience. Yet matching users’ knowledge needs to the most suitable output can be challenging. We have developed an evidence synthesis framework to help knowledge users, brokers, commissioners and producers decide which type of output offers the best ‘fit’ between ‘need’ and ‘response’. We conducted a four-strand literature search for characteristics and methods of evidence synthesis outputs using databases of peer reviewed literature, specific journals, grey literature and references in relevant documents. Eight experts in synthesis designed to get research into policy and practice were also consulted to hone issues for consideration and ascertain key studies. In all, 24 documents were included in the literature review. From these we identified essential characteristics to consider when planning an output—Readability, Relevance, Rigour and Resources—which we then used to develop a process for matching users’ knowledge needs with an appropriate evidence synthesis output. We also identified 10 distinct evidence synthesis outputs, classifying them in the evidence synthesis framework under four domains: key features, utility, technical characteristics and resources, and in relation to six primary audience groups—professionals, practitioners, researchers, academics, advocates and policy makers. Users’ knowledge needs vary and meeting them successfully requires collaborative planning. The Framework should facilitate a more systematic assessment of the balance of essential characteristics required to select the best output for the purpose.

Advocacy for health equity: a synthesis review

Context

Health inequalities are systematic differences in health among social groups that are caused by unequal exposure to—and distributions of—the social determinants of health (SDH). They are persistent between and within countries despite action to reduce them. Advocacy is a means of promoting policies that improve health equity, but the literature on how to do so effectively is dispersed. The aim of this review is to synthesize the evidence in the academic and gray literature and to provide a body of knowledge for advocates to draw on to inform their efforts.

Methods

This article is a systematic review of the academic literature and a fixed-length systematic search of the gray literature. After applying our inclusion criteria, we analyzed our findings according to our predefined dimensions of advocacy for health equity. Last, we synthesized our findings and made a critical appraisal of the literature.

Findings

The policy world is complex, and scientific evidence is unlikely to be conclusive in making decisions. Timely qualitative, interdisciplinary, and mixed-methods research may be valuable in advocacy efforts. The potential impact of evidence can be increased by “packaging” it as part of knowledge transfer and translation. Increased contact between researchers and policymakers could improve the uptake of research in policy processes. Researchers can play a role in advocacy efforts, although health professionals and disadvantaged people, who have direct contact with or experience of hardship, can be particularly persuasive in advocacy efforts. Different types of advocacy messages can accompany evidence, but messages should be tailored to advocacy target. Several barriers hamper advocacy efforts. The most frequently cited in the academic literature are the current political and economic zeitgeist and related public opinion, which tend to blame disadvantaged people for their ill health, even though biomedical approaches to health and political short-termism also act as barriers. These barriers could be tackled through long-term actions to raise public awareness and understanding of the SDH and through training of health professionals in advocacy. Advocates need to take advantage of “windows of opportunity,” which open and close quickly, and demonstrate expertise and credibility.

Conclusions

This article brings together for the first time evidence from the academic and the gray literature and provides a building block for efforts to advocate for health equity. Evidence regarding many of the dimensions is scant, and additional research is merited, particularly concerning the applicability of findings outside the English-speaking world. Advocacy organizations have a central role in advocating for health equity, given the challenges bridging the worlds of civil society, research, and policy.

Acceptability of financial incentives for breastfeeding: thematic analysis of readers' comments to UK online news reports

BACKGROUND

Whilst it is recommended that babies are breastfed exclusively for the first six months, many mothers do not maintain breastfeeding for this length of time. Previous research confirms that women and midwives value financial incentives for breastfeeding, but limited research has explored the wider acceptability of these interventions to the general public. This paper examines opinion towards financial incentives for breastfeeding using reader responses to UK on-line media coverage of a study undertaken in this area.

METHODS

This study used netnography to undertake a thematic analysis of 3,373 reader comments posted in response to thirteen articles, published in November 2013, which reported findings from a feasibility study of financial incentives for breastfeeding. All articles were published on one of six UK news websites that achieved a monthly audience of at least five million viewers across laptop and desktop computers and mobile devices during April-May 2013.

RESULTS

Nine analytical themes were identified, with a majority view that financial incentives for breastfeeding are unacceptable. These themes cover a range of opinions: from negligent parents unable to take responsibility for their own actions; through to psychologically vulnerable members of society who should be protected from coercion and manipulation; to capable and responsible women who can, and should be allowed to, make their own decisions. Many views focused on the immediate costs of the intervention, concluding that this was something that was currently unaffordable to fund (e.g. by the NHS). Others contrasted the value of the incentive against other 'costs' of breastfeeding. There was some consideration of the issue of cost-effectiveness and cost-saving, where the potential future benefit from initial investment was identified. Many commenters identified that financial incentives do not address the many structural and cultural barriers to breastfeeding.

CONCLUSIONS

Overall, those commenting on the on-line UK news articles viewed financial incentives for breastfeeding as unacceptable and that alternative, structural, interventions were likely to be more effective. Further consideration of how best to conduct internet-based qualitative research to elicit opinion towards public health issues is required.

Working for the public health: politics, localism and epistemologies of practice

The recent move of public health back to English local government has reignited debates about the role of a medicalised public health profession. The explicit policy rationale for the move was that local government is the arena in which the social determinants of health can be addressed, and that public health specialists could provide neutral evidence to support action on these. However, if a discourse of 'evidence-based' policy is in principle (if not practice) relatively unproblematic within the health arena, within the more overtly politicised local government space, rather different policy imperatives come to the fore. Responding to calls for research on evidence in practice, this article draws on ethnographic data of local authorities in the first year of the reorganised public health function. Focusing on alcohol policy, we explore how decisions that affect public health are rationalised and enacted through discourses of localism, empiricism and holism. These frame policy outcomes as inevitably plural and contingent: a framing which sits uneasily with normative discourses of evidence-based policy. We argue that locating public health in local government necessitates a refocusing of how evidence for public health is conceptualised, to incorporate multiple, and political, understandings of health and wellbeing.

Using conjoint analysis to develop a system to score research engagement actions by health decision makers

Background

Effective use of research to inform policymaking can be strengthened by policymakers undertaking various research engagement actions (e.g., accessing, appraising, and applying research). Consequently, we developed a thorough measurement and scoring tool to assess whether and how policymakers undertook research engagement actions in the development of a policy document. This scoring tool breaks down each research engagement action into its key ‘subactions’ like a checklist. The primary aim was to develop the scoring tool further so that it assigned appropriate scores to each subaction based on its effectiveness for achieving evidence-informed policymaking. To establish the relative effectiveness of these subactions, we conducted a conjoint analysis, which was used to elicit the opinions and preferences of knowledge translation experts.

Method

Fifty-four knowledge translation experts were recruited to undertake six choice surveys. Respondents were exposed to combinations of research engagement subactions called ‘profiles’, and rated on a 1–9 scale whether each profile represented a limited (1–3), moderate (4–6), or extensive (7–9) example of each research engagement action. Generalised estimating equations were used to analyse respondents’ choice data, where a utility coefficient was calculated for each subaction. A large utility coefficient indicates that a subaction was influential in guiding experts’ ratings of extensive engagement with research.

Results

The calculated utilities were used as the points assigned to the subactions in the scoring system. The following subactions yielded the largest utilities and were regarded as the most important components of engaging with research: searching academic literature databases, obtaining systematic reviews and peer-reviewed research, appraising relevance by verifying its applicability to the policy context, appraising quality by evaluating the validity of the method and conclusions, engaging in thorough collaborations with researchers, and undertaking formal research projects to inform the policy in question.

Conclusions

We have generated an empirically-derived and context-sensitive method of measuring and scoring the extent to which policymakers engaged with research to inform policy development. The scoring system can be used by organisations to quantify staff research engagement actions and thus provide them with insights into what types of training, systems, and tools might improve their staff’s research use capacity.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0013-z) contains supplementary material, which is available to authorized users.

Usage of an online tool to help policymakers better engage with research: Web CIPHER

BACKGROUND

There is a need to develop innovations that help policymakers better engage with research in order to increase its use in policymaking. As part of the Centre for Informing Policy in Health with Evidence from Research (CIPHER), we established Web CIPHER, an online tool with dynamic interactive elements such as hot topics, research summaries, blogs from trusted figures in health policy and research, a community bulletin board, multimedia section and research portal. The aim of this study was to examine policymakers' use of the website, and determine which sections were key drivers of use.

METHODS

Google Analytics (GA) was used to gather usage data during a 16-month period. Analysis was restricted to Web CIPHER members from policy agencies. We examined descriptive statistics including mean viewing times, number of page visits and bounce rates for each section and performed analyses of variance to compare usage between sections. Repeated measures analyses were undertaken to examine whether a weekly reminder email improved usage of Web CIPHER, particularly for research-related content.

RESULTS

During the measurement period, 223 policymakers from more than 32 organisations joined Web CIPHER. Users viewed eight posts on average per visit and stayed on the site for approximately 4 min. The bounce rate was less than 6%. The Blogs and Community sections received more unique views than all other sections. Blogs relating to improving policymakers' skills in applying research to policy were particularly popular. The email reminder had a positive effect on improving usage, particularly for research-related posts.

CONCLUSIONS

The data indicated a relatively small number of users. However, this sample may not be representative of policymakers since membership to the site and usage was completely voluntarily. Nonetheless, those who used the site appeared to engage well with it. The findings suggest that providing blog-type content written by trusted experts in health policy and research as well as regular email reminders may provide an effective means of disseminating the latest research to policymakers through an online web portal.

Trading quality for relevance: non-health decision-makers' use of evidence on the social determinants of health

OBJECTIVES

Local government services and policies affect health determinants across many sectors such as planning, transportation, housing and leisure. Researchers and policymakers have argued that decisions affecting wider determinants of health, well-being and inequalities should be informed by evidence. This study explores how information and evidence are defined, assessed and utilised by local professionals situated beyond the health sector, but whose decisions potentially affect health: in this case, practitioners working in design, planning and maintenance of the built environment.

DESIGN

A qualitative study using three focus groups. A thematic analysis was undertaken.

SETTING

The focus groups were held in UK localities and involved local practitioners working in two UK regions, as well as in Brazil, USA and Canada.

PARTICIPANTS

UK and international practitioners working in the design and management of the built environment at a local government level.

RESULTS

Participants described a range of data and information that constitutes evidence, of which academic research is only one part. Built environment decision-makers value empirical evidence, but also emphasise the legitimacy and relevance of less empirical ways of thinking through narratives that associate their work to art and philosophy. Participants prioritised evidence on the acceptability, deliverability and sustainability of interventions over evidence of longer term outcomes (including many health outcomes). Participants generally privileged local information, including personal experiences and local data, but were less willing to accept evidence from contexts perceived to be different from their own.

CONCLUSIONS

Local-level built environment practitioners utilise evidence to make decisions, but their view of 'best evidence' appears to prioritise local relevance over academic rigour. Academics can facilitate evidence-informed local decisions affecting social determinants of health by working with relevant practitioners to improve the quality of local data and evaluations, and by advancing approaches to improve the external validity of academic research.

Perceptions and beliefs of public policymakers in a Southern European city

Introduction

Socio-economic inequalities in health are large in urban areas; however, local municipal governments may plan, manage and provide services and policies which can reduce these. The objective of this study was to describe the beliefs and perceptions of public policymakers in a European city, Barcelona. They are the key actors in designing and implementing urban public policies.

Methods

A qualitative research study describing policymakers’ beliefs on health inequalities. The study population were twelve policymakers. These were politicians or officers from the city council. Informant profiles were selected using a theoretical sample. Semi-structured individual interviews were performed to collect the data and a thematic content analysis was carried out.

Results

Politicians were aware of health inequalities in their city and identified diverse social causes. They viewed reducing inequalities as a priority for the city’s government. Officers were less knowledgeable and described less efforts in addressing health inequalities. It was stated by some that reducing inequalities in non-health sectors helped to reduce health inequalities indirectly and there was some collaboration between two sectors. The most frequent barriers encountered when implementing policies were funding and the cities’ limited authority.

Conclusions

Officers and policymakers had different levels of awareness and access to information on health and its socials determinants. Officers referred to specific causes of health inequalities and policies which related to their sectors and politicians were more familiar with upstream determinants and policies. Some participants explained that policies and programmes needed to be evaluated and very little intersectoral action was said to be carried out. More efforts should be made to provide all policymakers with information on the social determinants of health inequalities. Research on health inequalities and policy should engage with policymakers and promote health as a cross cutting issue in the city council in liaison with the third sector.

Development and evaluation of 'briefing notes' as a novel knowledge translation tool to aid the implementation of sex/gender analysis in systematic reviews: a pilot study

Background

There is increasing recognition of sex/gender differences in health and the importance of identifying differential effects of interventions for men and women. Yet, to whom the research evidence does or does not apply, with regard to sex/gender, is often insufficiently answered. This is also true for systematic reviews which synthesize results of primary studies. A lack of analysis and reporting of evidence on sex/gender raises concerns about the applicability of systematic reviews. To bridge this gap, this pilot study aimed to translate knowledge about sex/gender analysis (SGA) into a user-friendly ‘briefing note’ format and evaluate its potential in aiding the implementation of SGA in systematic reviews.

Methods

Our Sex/Gender Methods Group used an interactive process to translate knowledge about sex/gender into briefing notes, a concise communication tool used by policy and decision makers. The briefing notes were developed in collaboration with three Cochrane Collaboration review groups (HIV/AIDS, Hypertension, and Musculoskeletal) who were also the target knowledge users of the briefing notes. Briefing note development was informed by existing systematic review checklists, literature on sex/gender, in-person and virtual meetings, and consultation with topic experts. Finally, we held a workshop for potential users to evaluate the notes.

Results

Each briefing note provides tailored guidance on considering sex/gender to reviewers who are planning or conducting systematic reviews and includes the rationale for considering sex/gender, with examples specific to each review group’s focus. Review authors found that the briefing notes provided welcome guidance on implementing SGA that was clear and concise, but also identified conceptual and implementation challenges.

Conclusions

Sex/gender briefing notes are a promising knowledge translation tool. By encouraging sex/gender analysis and equity considerations in systematic reviews, the briefing notes can assist systematic reviewers in ensuring the applicability of research evidence, with the goal of improved health outcomes for diverse populations.

Protocol for the process evaluation of a complex intervention designed to increase the use of research in health policy and program organisations (the SPIRIT study)

Background

Process evaluation is vital for understanding how interventions function in different settings, including if and why they have different effects or do not work at all. This is particularly important in trials of complex interventions in ‘real world’ organisational settings where causality is difficult to determine. Complexity presents challenges for process evaluation, and process evaluations that tackle complexity are rarely reported. This paper presents the detailed protocol for a process evaluation embedded in a randomised trial of a complex intervention known as SPIRIT (Supporting Policy In health with Research: an Intervention Trial). SPIRIT aims to build capacity for using research in health policy and program agencies.

Methods

We describe the flexible and pragmatic methods used for capturing, managing and analysing data across three domains: (a) the intervention as it was implemented; (b) how people participated in and responded to the intervention; and (c) the contextual characteristics that mediated this relationship and may influence outcomes. Qualitative and quantitative data collection methods include purposively sampled semi-structured interviews at two time points, direct observation and coding of intervention activities, and participant feedback forms. We provide examples of the data collection and data management tools developed.

Discussion

This protocol provides a worked example of how to embed process evaluation in the design and evaluation of a complex intervention trial. It tackles complexity in the intervention and its implementation settings. To our knowledge, it is the only detailed example of the methods for a process evaluation of an intervention conducted as part of a randomised trial in policy organisations. We identify strengths and weaknesses, and discuss how the methods are functioning during early implementation. Using ‘insider’ consultation to develop methods is enabling us to optimise data collection while minimising discomfort and burden for participants. Embedding the process evaluation within the trial design is facilitating access to data, but may impair participants’ willingness to talk openly in interviews. While it is challenging to evaluate the process of conducting a randomised trial of a complex intervention, our experience so far suggests that it is feasible and can add considerably to the knowledge generated.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-014-0113-0) contains supplementary material, which is available to authorized users.

What kinds of policies to reduce health inequalities in the UK do researchers support?

BACKGROUND

Despite a wealth of research and policy initiatives, progress in tackling the UK's health inequalities has been limited. This article explores whether there appears to be consensus among researchers about the kinds of policies likely to reduce health inequalities.

METHODS

Ninety-nine proposals for addressing health inequalities were identified from multiple sources. Forty-one researchers participated in a survey assessing the extent to which they believed each proposal would reduce health inequalities, based on three criteria. The 20 proposals generating most support were employed in a second stage, in which 92 researchers indicated which proposals they felt would have the greatest impact on reducing health inequalities.

RESULTS

Some consensus exists among researchers about the policy approaches likely to reduce UK health inequalities: a more progressive distribution of income/wealth, greater investment in services for deprived communities, plus regulatory policies to limit the impact of lifestyle-behavioural risks. However, researchers' support for proposals varies depending whether they are asked to express their expert opinion or to comment on the strength of the available evidence.

CONCLUSIONS

When consulting researchers about health inequalities, policymakers need to consider whether they are seeking research-informed expertise or assessments of the available evidence; these questions are likely to yield different responses.

Foundation of evidence-based decision making for health care managers-part II: meta-analysis and applying the evidence

This article is a continuation of a 2-part series that seeks to build the skills of health care managers and supervisors in critically reading and interpreting systematic reviews and meta-analyses. The first article, part I, defined the types of systematic reviews and outlined the process of conducting them. This article, part II, focuses on meta-analysis, a type of systematic review. This article explains the additional procedures associated with meta-analyses and describes the potential shortcomings of both systematic reviews and meta-analyses. Finally, this article provides a way for health care managers and supervisors to appraise systematic reviews and meta-analyses and to determine whether the evidence applies to their problems and settings.

New directions in evidence-based policy research: a critical analysis of the literature

Despite 40 years of research into evidence-based policy (EBP) and a continued drive from both policymakers and researchers to increase research uptake in policy, barriers to the use of evidence are persistently identified in the literature. However, it is not clear what explains this persistence - whether they represent real factors, or if they are artefacts of approaches used to study EBP. Based on an updated review, this paper analyses this literature to explain persistent barriers and facilitators. We critically describe the literature in terms of its theoretical underpinnings, definitions of 'evidence', methods, and underlying assumptions of research in the field, and aim to illuminate the EBP discourse by comparison with approaches from other fields. Much of the research in this area is theoretically naive, focusing primarily on the uptake of research evidence as opposed to evidence defined more broadly, and privileging academics' research priorities over those of policymakers. Little empirical data analysing the processes or impact of evidence use in policy is available to inform researchers or decision-makers. EBP research often assumes that policymakers do not use evidence and that more evidence - meaning research evidence - use would benefit policymakers and populations. We argue that these assumptions are unsupported, biasing much of EBP research. The agenda of 'getting evidence into policy' has side-lined the empirical description and analysis of how research and policy actually interact in vivo. Rather than asking how research evidence can be made more influential, academics should aim to understand what influences and constitutes policy, and produce more critically and theoretically informed studies of decision-making. We question the main assumptions made by EBP researchers, explore the implications of doing so, and propose new directions for EBP research, and health policy.

Lost in translation? Theory, policy and practice in systems-based environmental approaches to obesity prevention in the Healthy Towns programme in England

This paper explores how system-wide approaches to obesity prevention were 'theorised' and translated into practice in the 'Healthy Towns' programme implemented in nine areas in England. Semi-structured interviews with 20 informants, purposively selected to represent national and local programme development, management and delivery were undertaken. Results suggest that informants articulated a theoretical understanding of a system-wide approach to obesity prevention, but simplifying this complex task in the context of uncertainty over programme aims and objectives, and absence of a clear direction from the central government, resulted in local programmes relying on traditional multi-component approaches to programme delivery. The development of clear, practical guidance on implementation should form a central part of future system-wide approaches to obesity prevention.