SUPPORT Tools for evidence-informed health Policymaking (STP) 17: Dealing with insufficient research evidence

Developing Framework and Strategies for Capacity Building to Apply Evidence-Informed Health Policy-Making in Iran: Mixed Methods Study of SAHSHA Project

BACKGROUND

SASHA, which stands for "evidence-informed health policy-making (EIHP)" in Persian, is a national project to draw a roadmap for strengthening EIHP in Iran. As a part of SASHA, this research aimed to develop evidence-based and context-aware policy options for increasing the capacity of decision-makers to apply EIHP in Iran.

METHODS

This was a qualitative study, which was informed by a literature review of pull efforts' capacity building programs. Based on the review, we developed policy options and validated them through an expert panel that involved twelve experts. Data were analyzed using a content analysis method.

RESULTS

We extracted data from 11 articles. The objectives of capacity building programs were: single-skill development, personal/professional development, and organizational development. According to these objectives, the contents and training methods of the programs vary. Capacity building programs have shown positive impacts on individual knowledge/attitudes to use EIHP. However, the impacts of programs at the organizational or the health system level remain under-researched. We followed several threads from the literature review through to the expert panel that included training the management team, instead of training managers, training for problem-solving skills, and designing tailored programs. Barriers of capacity building for EIHP regard the context of the health system (weak accountability and the widespread conflict of interest) and healthcare organizational structures (decision support systems, knowledge management infrastructures, and lack of management team). Experts suggested interventions on the barriers, particularly on resolving the conflict of interests before launching new programs. A proposed framework to increase the capacity of health policy-makers incorporates strategies at three levels: capacity building program, organizational structure, and health system context.

CONCLUSION

To prepare the context of Iranian healthcare organizations for capacity building programs, the conflict of interests needs to be resolved, decision-makers should be made more accountable, and healthcare organizations need to provide more knowledge management infrastructures and decision support systems.

Supporting the use of research evidence in decision-making in crisis zones in low- and middle-income countries: a critical interpretive synthesis

BACKGROUND

Decision-makers in crisis zones are faced with the challenge of having to make health-related decisions under limited time and resource constraints and in light of the many factors that can influence their decisions, of which research evidence is just one. To address a key gap in the research literature about how best to support the use of research evidence in such situations, we conducted a critical interpretive synthesis approach to develop a conceptual framework that outlines the strategies that leverage the facilitators and address the barriers to evidence use in crisis zones.

METHODS

We systematically reviewed both empirical and non-empirical literature and used an interpretive analytic approach to synthesise the results and develop the conceptual framework. We used a 'compass' question to create a detailed search strategy and conducted electronic searches in CINAHL, EMBASE, MEDLINE, SSCI and Web of Science. A second reviewer was assigned to a representative sample of articles. We purposively sampled additional papers to fill in conceptual gaps.

RESULTS

We identified 21 eligible papers to be analysed and purposively sampled an additional 6 to fill conceptual gaps. The synthesis resulted in a conceptual framework that focuses on evidence use in crisis zones examined through the lens of four systems - political, health, international humanitarian aid and health research. Within each of the four systems, the framework identifies the most actionable strategies that leverage the facilitators and address the barriers to evidence use.

CONCLUSIONS

This study presents a new conceptual framework that outlines strategies that leverage the facilitators and address the barriers to evidence use in crisis zones within different systems. This study expands on the literature pertaining to evidence-informed decision-making.

The governmental health policy-development process for Syrian refugees: an embedded qualitative case studies in Lebanon and Ontario

BACKGROUND

The unprecedented amount of resources dedicated to humanitarian aid has led many stakeholders to demand the use of reliable evidence in humanitarian aid decisions to ensure that desired impacts are achieved at acceptable costs. However, little is known about the factors that influence the use of research evidence in the policy development in humanitarian crises. We examined how research evidence was used to inform two humanitarian policies made in response to the Syrian refugee crisis.

METHODS

We identified two policies as rich potential case studies to examine the use of evidence in humanitarian aid policy decision-making: Lebanon's 2016 Health Response Strategy and Ontario's 2016 Phase 2: Health System Action Plan, Syrian Refugees. To study each, we used an embedded qualitative case study methodology and recruited senior decision-makers, policy advisors, and healthcare providers who were involved with the development of each policy. We reviewed publicly available documents and media articles that spoke to the factors that influence the process. We used the analytic technique of explanation building to understand the factors that influence the use of research evidence in the policy-development process in crisis zones.

RESULTS

We interviewed eight informants working in government and six in international agencies in Lebanon, and two informants working in healthcare provider organizations and two in non-governmental organizations in Ontario, for a total of 18 key informants. Based on our interviews and documentary analysis, we identified that there was limited use of research evidence and that four broad categories of factors helped to explain the policy-development process for Syrian refugees - development of health policies without significant chance for derailment from other government bodies (Lebanon) or opposition parties (Ontario) (i.e., facing no veto points), government's engagement with key societal actors to inform the policy-development process, the values underpinning the process, and external factors significantly influencing the policy-development process.

CONCLUSIONS

This study suggests that use of research evidence in the policy-development process for Syrian refugees was subordinate to key political factors, resulting in limited influence of research evidence in the development of both the Lebanese and Ontarian policy.

The GRADE Evidence to Decision (EtD) framework for health system and public health decisions

OBJECTIVE

To describe a framework for people making and using evidence-informed health system and public health recommendations and decisions.

BACKGROUND

We developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests.

DEVELOPING THE FRAMEWORK

Building on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels.

FINDINGS

All the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline.

DISCUSSION

This framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

Factors Associated With Unhealthy Snacks Consumption Among Adolescents in Iran's Schools

Background: Well-informed interventions are needed if school-based health promotion is to be effective. Among other aims, the Iranian Health Promoting School (IHPS) program that was launched in 2011, has an important aim of promoting dietary behaviors of adolescents. The present study, therefore, aimed to investigate the factors affecting unhealthy snacking of adolescents and provide evidence for a more effective IHPS program.

Methods: In a cross-sectional study design, 1320 students from 40 schools in Kerman city were selected using a proportional stratified random sampling method. A modified qualitative Food Frequency Questionnaire (FFQ) was used to gather data about unhealthy snacking behavior. Data about intrapersonal and environmental factors were obtained using a validated and reliable questionnaire. A mixed-effects negative-binomial regression model was used to analyze the data.

Results: Taste and sensory perception (prevalence rate ratio [PRR]=1.18; 95% CI: 1.09-1.27), being a male (PRR=1.20; 95% CI: 1.05-1.38) and lower nutritional knowledge (PRR=0.96; 95% CI: 0.91-0.99) were associated with higher weekly unhealthy snaking. Perceived self-efficacy (PRR=0.95; 95% CI: 0.91-1.00) negatively influenced the frequency of unhealthy snaking, with this approaching significance (P<.06). In case of environmental factors, high socio-economic status (SES) level (PRR=1.45; 95% CI: 1.26-1.67), single-parent family (PRR=1.14; 95% CI: 1.01-1.30), more social norms pressure (PRR=1.08; 95% CI: 1.01-1.17), pocket money allowance (PRR=1.21; 95% CI: 1.09-1.34), easy accessibility (PRR=1.06; 95% CI:1.01-1.11), and less perceived parental control (PRR=0.96; 95% CI: 0.92-0.99) all had a role in higher consumption of unhealthy snacks. Interestingly, larger school size was associated with less unhealthy snacking (PRR=0.79; 95% CI: 0.68-0.92).

Conclusion: Unhealthy snacking behavior is influenced by individual, socio-cultural and physical-environmental influences, namely by factors relating to poor parenting practices, high SES level, family characteristics, improper social norms pressure, and less knowledge and self-efficacy of students. This evidence can be used to inform a more evidencebased IHPS program through focusing on supportive strategies at the home, school, and local community levels.

A Qualitative Assessment of the Evidence Utilization for Health Policy-Making on the Basis of SUPPORT Tools in a Developing Country

Background: SUPPORT tools consist of 18 articles addressing the health policy-makers so that they can learn how to make evidence-informed health policies. These tools have been particularly recommended for developing countries. The present study tries to explain the process of evidence utilization for developing policy documents in the Iranian Ministry of Health and Medical Education (MoHME) and to compare the findings with those of SUPPORT tools.

Methods: A qualitative research was conducted, using the framework analysis approach. Participants consisted of senior managers and technicians in MoHME. Purposeful sampling was done, with a maximum variety, for the selection of research participants: individuals having at least 5 years of experience in preparing evidence-based policy documents. Face-to-face interviews were conducted for data collection. As a guideline for the interviews, ‘the Utilization of Evidence in Policy-Making Organizations’ procedure was used. The data were analyzed through the analysis of the framework method using MAXQDA 10 software.

Results: The participants acquired the research evidence in a topic-based form, and they were less likely to search on the basis of the evidence pyramid. To assess the quality of evidence, they did not use standard critical tools; to adapt the evidence and interventions with the local setting, they did not use the ideas and experiences of all stakeholders, and in preparing the evidence-based policy documents, they did not take into consideration the window of opportunity, did not refrain from using highly technical terms, did not write user-friendly summaries, and did not present alternative policy options. In order to develop health policies, however, they used the following innovations: attention to the financial burden of policy issues on the agenda, sensitivity analysis of the preferred policy option on the basis of technical, sociopolitical, and economic feasibility, advocacy from other scholars, using the multi-criteria decision-making models for the prioritization of policy options, implementation of policy based on the degree of readiness of policy-implementing units, and the classification of policy documents on the basis of different conditions of policy-making (urgent, short-term, and long-term).

Conclusion: Findings showed that the process of evidence utilization in IR-MoH enjoys some innovations for the support of health policy development. The present study provides IR-MoH with considerable opportunities for the improvement of evidence-informed health policy-making. Moreover, the SUPPORT process and tools are recommended to be used in developing countries.

Tobacco Use during Pregnancy and Its Associated Factors in a Mountain District of Eastern Nepal: A Cross-Sectional Questionnaire Survey

Background

Tobacco using among women is more prevalent in Nepal as compared to other South-East Asian countries. The effect of its use is seen not only on the pregnant women, but also health of the growing fetus is compromised. Currently, little is known about the tobacco use among women especially during pregnancy in Nepal. This study explored the tobacco use prevalence and its associated factors during pregnancy.

Materials and methods

A cross-sectional study was conducted in Sankhuwasabha, a mountain district of eastern Nepal. Representative sample of 436 women of reproductive age group with infant were selected by stratified simple random sampling. Data were collected by face-to-face interviews of selected participants. Data were analyzed with SPSS version 16.0. Binary logistic regression was used to analyze the relationship among variables.

Results

The study revealed that the prevalence of tobacco use during pregnancy was 17.2%. Only one fifth of the research participants were asked to quit tobacco by health workers during last pregnancy. Multivariable analyses revealed that illiteracy (AOR: 2.31, CI: 1.18–4.52), more than two parity (AOR: 2.45, CI: 1.19–5.07), alcohol use during last pregnancy (AOR: 3.99, CI: 1.65–9.68), and having tobacco user within family (AOR: 2.05, CI: 1.11–3.78) are more likely to use tobacco during pregnancy.

Conclusion

Tobacco use during pregnancy was widely prevalent. Tobacco-focused interventions are required for antenatal women to promote cessation among user and prevent initiation with focus on overcoming problems like illiteracy, high parity, alcohol use, and having other tobacco user family members in family.

An interpretive review of consensus statements on clinical guideline development and their application in the field of traditional and complementary medicine

BACKGROUND

Despite ongoing consumer demand and an emerging scientific evidence-base for traditional and complementary medicine (T&CM), there remains a paucity of reliable information in standard clinical guidelines about their use. Often T&CM interventions are not mentioned, or the recommendations arising from these guidelines are unhelpful to end-users (i.e. patients, practitioners and policy makers). Insufficient evidence of efficacy may be a contributing factor; however, often informative recommendations could still be made by drawing on relevant information from other avenues. In light of this, the aim of this research was to review national and internationally endorsed consensus statements for clinical guideline developers, and to interpret how to apply these methods when making recommendations regarding the use of T&CM.

METHOD

The critical interpretive review method was used to identify and appraise relevant consensus statements published between 1995 and 2015. The statements were identified using a purposive sampling technique until data saturation was reached. The most recent edition of a statement was included in the analysis. The content, scope and themes of the statements were compared and interpreted within the context of the T&CM setting; including history, regulation, use, emerging scientific evidence-base and existing guidelines.

RESULTS

Eight consensus statements were included in the interpretive review. Searching stopped at this stage as no new major themes were identified. The five themes relevant to the challenges of developing T&CM guidelines were: (1) framing the question; (2) the limitations of using an evidence hierarchy; (3) strategies for dealing with insufficient, high quality evidence; (4) the importance of qualifying a recommendation; and (5) the need for structured consensus development.

CONCLUSION

Evidence regarding safety, efficacy and cost effectiveness are not the only information required to make recommendations for clinical guidelines. Modifying factors such as burden of disease, magnitude of effect, current use, demand, equity and ease of integration should also be considered. Uptake of the recommendations arising from this review are expected to result in the development of higher quality clinical guidelines that offer greater assistance to those seeking answers about the appropriate use of T&CM.

Developments in Impact Assessment of New Diagnostic Algorithms for Tuberculosis Control

A modified presentation of the impact assessment framework is proposed that improves accessibility while continuing to provide a checklist of the evidence needed to support policy decisions on the implementation of new tools for the diagnosis of tuberculosis.

Evidence for informing health policy development in Low-income Countries (LICs): perspectives of policy actors in Uganda

BACKGROUND

Although there is a general agreement on the benefits of evidence informed health policy development given resource constraints especially in Low-Income Countries (LICs), the definition of what evidence is, and what evidence is suitable to guide decision-making is still unclear. Our study is contributing to filling this knowledge gap. We aimed to explore health policy actors' views regarding what evidence they deemed appropriate to guide health policy development.

METHODS

Using exploratory qualitative methods, we conducted interviews with 51 key informants using an in-depth interview guide. We interviewed a diverse group of stakeholders in health policy development and knowledge translation in the Uganda health sector. Data were analyzed using inductive content analysis techniques.

RESULTS

Different stakeholders lay emphasis on different kinds of evidence. While donors preferred international evidence and Ministry of Health (MoH) officials looked to local evidence, district health managers preferred local evidence, evidence from routine monitoring and evaluation, and reports from service providers. Service providers on the other hand preferred local evidence and routine monitoring and evaluation reports whilst researchers preferred systematic reviews and clinical trials. Stakeholders preferred evidence covering several aspects impacting on decision-making highlighting the fact that although policy actors look for factual information, they also require evidence on context and implementation feasibility of a policy decision.

CONCLUSION

What LICs like Uganda categorize as evidence suitable for informing policy encompasses several types with no consensus on what is deemed as most appropriate. Evidence must be of high quality, applicable, acceptable to the users, and informing different aspects of decision-making.

Understanding evidence: a statewide survey to explore evidence-informed public health decision-making in a local government setting

Background

The value placed on types of evidence within decision-making contexts is highly dependent on individuals, the organizations in which the work and the systems and sectors they operate in. Decision-making processes too are highly contextual. Understanding the values placed on evidence and processes guiding decision-making is crucial to designing strategies to support evidence-informed decision-making (EIDM). This paper describes how evidence is used to inform local government (LG) public health decisions.

Methods

The study used mixed methods including a cross-sectional survey and interviews. The Evidence-Informed Decision-Making Tool (EvIDenT) survey was designed to assess three key domains likely to impact on EIDM: access, confidence, and organizational culture. Other elements included the usefulness and influence of sources of evidence (people/groups and resources), skills and barriers, and facilitators to EIDM. Forty-five LGs from Victoria, Australia agreed to participate in the survey and up to four people from each organization were invited to complete the survey (n = 175). To further explore definitions of evidence and generate experiential data on EIDM practice, key informant interviews were conducted with a range of LG employees working in areas relevant to public health.

Results

In total, 135 responses were received (75% response rate) and 13 interviews were conducted. Analysis revealed varying levels of access, confidence and organizational culture to support EIDM. Significant relationships were found between domains: confidence, culture and access to research evidence. Some forms of evidence (e.g. community views) appeared to be used more commonly and at the expense of others (e.g. research evidence). Overall, a mixture of evidence (but more internal than external evidence) was influential in public health decision-making in councils. By comparison, a mixture of evidence (but more external than internal evidence) was deemed to be useful in public health decision-making.

Conclusions

This study makes an important contribution to understanding how evidence is used within the public health LG context.

Trial registration

ACTRN12609000953235.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-014-0188-7) contains supplementary material, which is available to authorized users.

Knowledge translation strategies to improve the use of evidence in public health decision making in local government: intervention design and implementation plan

BACKGROUND

Knowledge translation strategies are an approach to increase the use of evidence within policy and practice decision-making contexts. In clinical and health service contexts, knowledge translation strategies have focused on individual behavior change, however the multi-system context of public health requires a multi-level, multi-strategy approach. This paper describes the design of and implementation plan for a knowledge translation intervention for public health decision making in local government.

METHODS

Four preliminary research studies contributed findings to the design of the intervention: a systematic review of knowledge translation intervention effectiveness research, a scoping study of knowledge translation perspectives and relevant theory literature, a survey of the local government public health workforce, and a study of the use of evidence-informed decision-making for public health in local government. A logic model was then developed to represent the putative pathways between intervention inputs, processes, and outcomes operating between individual-, organizational-, and system-level strategies. This formed the basis of the intervention plan.

RESULTS

The systematic and scoping reviews identified that effective and promising strategies to increase access to research evidence require an integrated intervention of skill development, access to a knowledge broker, resources and tools for evidence-informed decision making, and networking for information sharing. Interviews and survey analysis suggested that the intervention needs to operate at individual and organizational levels, comprising workforce development, access to evidence, and regular contact with a knowledge broker to increase access to intervention evidence; develop skills in appraisal and integration of evidence; strengthen networks; and explore organizational factors to build organizational cultures receptive to embedding evidence in practice. The logic model incorporated these inputs and strategies with a set of outcomes to measure the intervention's effectiveness based on the theoretical frameworks, evaluation studies, and decision-maker experiences.

CONCLUSION

Documenting the design of and implementation plan for this knowledge translation intervention provides a transparent, theoretical, and practical approach to a complex intervention. It provides significant insights into how practitioners might engage with evidence in public health decision making. While this intervention model was designed for the local government context, it is likely to be applicable and generalizable across sectors and settings.

TRIAL REGISTRATION

Australia New Zealand Clinical Trials Register ACTRN12609000953235.

Knowledge translation of research findings

BACKGROUND

One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?

DISCUSSION

We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.

SUMMARY

There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.

Knowledge translation of research findings

BACKGROUND

One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?

DISCUSSION

We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.

SUMMARY

There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.

Missing and accounted for: gaps and areas of wealth in the public health review literature

BACKGROUND

High-quality review evidence is useful for informing and influencing public health policy and practice decisions. However, certain topic areas lack representation in terms of the quantity and quality of review literature available. The objectives of this paper are to identify the quantity, as well as quality, of review-level evidence available on the effectiveness of public health interventions for public health decision makers.

METHODS

Searches conducted on http://www.health-evidence.ca produced an inventory of public health review literature in 21 topic areas. Gaps and areas of wealth in the review literature, as well as the proportion of reviews rated methodologically strong, moderate, or weak were identified. The top 10 topic areas of interest for registered users and visitors of http://www.health-evidence.ca were extracted from user profile data and Google Analytics.

RESULTS

Registered users' top three interests included: 1) healthy communities, 2) chronic diseases, and 3) nutrition. The top three preferences for visitors included: 1) chronic diseases, 2) physical activity, and 3) addiction/substance use. All of the topic areas with many (301+) available reviews were of interest to registered users and/or visitors (mental health, physical activity, addiction/substance use, adolescent health, child health, nutrition, adult health, and chronic diseases). Conversely, the majority of registered users and/or visitors did not have preference for topic areas with few (≤ 150) available reviews (food safety and inspection, dental health, environmental health) with the exception of social determinants of health and healthy communities. Across registered users' and visitors' topic areas of preference, 80.2% of the reviews were of well-done methodological quality, with 43.5% of reviews having a strong quality rating and 36.7% a moderate review quality rating.

CONCLUSIONS

In topic areas in which many reviews are available, higher level syntheses are needed to guide policy and practice. For other topic areas with few reviews, it is necessary to determine whether primary study evidence exists, or is needed, so that reviews can be conducted in the future. Considering that less than half of the reviews available on http://www.health-evidence.ca are of strong methodological quality, the quality of the review-level evidence needs to improve across the range of public health topic areas.

Where there is no evidence: use of expert consensus methods to fill the evidence gap in low-income countries and cultural minorities

BACKGROUND

In both developing countries and in relation to cultural minorities there have been calls to scale up mental health services and for evidence-informed policy and practice.

EVIDENCE BASED MEDICINE

The evidence based medicine movement has had a major influence in improving practice. However, implementation of this approach has some major difficulties. One that has been neglected is the situation where there is no relevant evidence. This situation is more likely to occur for healthcare decisions in developing countries or for cultural minorities within developed countries, because resources do not exist for expensive research studies.

CONSENSUS METHODS

Consensus methods, such as the Delphi process, can be useful in providing an evidence base in situations where there is insufficient evidence. They provide a way of systematically tapping the expertise of people working in the area and give evidence that is readily applicable for a particular country and culture. Although consensus methods are often thought of as low in the hierarchy of evidence, consensus is central to the scientific process. We present four examples where the Delphi method was used to assess expert consensus in situations where no other evidence existed: estimating the prevalence of dementia in developing countries, developing mental health first aid guidelines in Asian countries, mental health first aid guidelines for Australian Aboriginal people, and modification of the concept of 'recovery' for Australian immigrant communities.

CONCLUSION

Consensus methods can provide a basis for decision-making and considered action when there is no evidence or when there are doubts about the applicability of evidence that has been generated from other populations or health system settings.

SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. The reliability of systematic reviews of the effects of health interventions is variable. Consequently, policymakers and others need to assess how much confidence can be placed in such evidence. The use of systematic and transparent processes to determine such decisions can help to prevent the introduction of errors and bias in these judgements. In this article, we suggest five questions that can be considered when deciding how much confidence to place in the findings of a systematic review of the effects of an intervention. These are: 1. Did the review explicitly address an appropriate policy or management question? 2. Were appropriate criteria used when considering studies for the review? 3. Was the search for relevant studies detailed and reasonably comprehensive? 4. Were assessments of the studies' relevance to the review topic and of their risk of bias reproducible? 5. Were the results similar from study to study?

SUPPORT tools for evidence-informed policymaking in health 18: Planning monitoring and evaluation of policies

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

The term monitoring is commonly used to describe the process of systematically collecting data to inform policymakers, managers and other stakeholders whether a new policy or programme is being implemented in accordance with their expectations. Indicators are used for monitoring purposes to judge, for example, if objectives are being achieved, or if allocated funds are being spent appropriately. Sometimes the term evaluation is used interchangeably with the term monitoring, but the former usually suggests a stronger focus on the achievement of results. When the term impact evaluation is used, this usually implies that there is a specific attempt to try to determine whether the observed changes in outcomes can be attributed to a particular policy or programme. In this article, we suggest four questions that can be used to guide the monitoring and evaluation of policy or programme options. These are: 1. Is monitoring necessary? 2. What should be measured? 3. Should an impact evaluation be conducted? 4. How should the impact evaluation be done?

SUPPORT Tools for evidence-informed health Policymaking (STP) 16: Using research evidence in balancing the pros and cons of policies

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address the use of evidence to inform judgements about the balance between the pros and cons of policy and programme options. We suggest five questions that can be considered when making these judgements. These are: 1. What are the options that are being compared? 2. What are the most important potential outcomes of the options being compared? 3. What is the best estimate of the impact of the options being compared for each important outcome? 4. How confident can policymakers and others be in the estimated impacts? 5. Is a formal economic model likely to facilitate decision making?

SUPPORT Tools for evidence-informed health Policymaking (STP) 12: Finding and using research evidence about resource use and costs

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address considerations about resource use and costs. The consequences of a policy or programme option for resource use differ from other impacts (both in terms of benefits and harms) in several ways. However, considerations of the consequences of options for resource use are similar to considerations related to other impacts in that policymakers and their staff need to identify important impacts on resource use, acquire and appraise the best available evidence regarding those impacts, and ensure that appropriate monetary values have been applied. We suggest four questions that can be considered when assessing resource use and the cost consequences of an option. These are: 1. What are the most important impacts on resource use? 2. What evidence is there for important impacts on resource use? 3. How confident is it possible to be in the evidence for impacts on resource use? 4. Have the impacts on resource use been valued appropriately in terms of their true costs?

SUPPORT Tools for evidence-informed health Policymaking (STP)

This article is the Introduction to a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

Knowing how to find and use research evidence can help policymakers and those who support them to do their jobs better and more efficiently. Each article in this series presents a proposed tool that can be used by those involved in finding and using research evidence to support evidence-informed health policymaking. The series addresses four broad areas: 1. Supporting evidence-informed policymaking 2. Identifying needs for research evidence in relation to three steps in policymaking processes, namely problem clarification, options framing, and implementation planning 3. Finding and assessing both systematic reviews and other types of evidence to inform these steps, and 4. Going from research evidence to decisions. Each article begins with between one and three typical scenarios relating to the topic. These scenarios are designed to help readers decide on the level of detail relevant to them when applying the tools described. Most articles in this series are structured using a set of questions that guide readers through the proposed tools and show how to undertake activities to support evidence-informed policymaking efficiently and effectively. These activities include, for example, using research evidence to clarify problems, assessing the applicability of the findings of a systematic review about the effects of options selected to address problems, organising and using policy dialogues to support evidence-informed policymaking, and planning policy monitoring and evaluation. In several articles, the set of questions presented offers more general guidance on how to support evidence-informed policymaking. Additional information resources are listed and described in every article. The evaluation of ways to support evidence-informed health policymaking is a developing field and feedback about how to improve the series is welcome.