SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

Interventions promoting uptake of water, sanitation and hygiene (WASH) technologies in low- and middle-income countries: An evidence and gap map of effectiveness studies

Background

Lack of access to and use of water, sanitation and hygiene (WASH) cause 1.6 million deaths every year, of which 1.2 million are due to gastrointestinal illnesses like diarrhoea and acute respiratory infections like pneumonia. Poor WASH access and use also diminish nutrition and educational attainment, and cause danger and stress for vulnerable populations, especially for women and girls. The hardest hit regions are sub-Saharan Africa and South Asia. Sustainable Development Goal (SDG) 6 calls for the end of open defecation, and universal access to safely managed water and sanitation facilities, and basic hand hygiene, by 2030. WASH access and use also underpin progress in other areas such as SDG1 poverty targets, SDG3 health and SDG4 education targets. Meeting the SDG equity agenda to "leave none behind" will require WASH providers prioritise the hardest to reach including those living remotely and people who are disadvantaged.

Objectives

Decision makers need access to high-quality evidence on what works in WASH promotion in different contexts, and for different groups of people, to reach the most disadvantaged populations and thereby achieve universal targets. The WASH evidence map is envisioned as a tool for commissioners and researchers to identify existing studies to fill synthesis gaps, as well as helping to prioritise new studies where there are gaps in knowledge. It also supports policymakers and practitioners to navigate the evidence base, including presenting critically appraised findings from existing systematic reviews.

Methods

This evidence map presents impact evaluations and systematic reviews from the WASH sector, organised according to the types of intervention mechanisms, WASH technologies promoted, and outcomes measured. It is based on a framework of intervention mechanisms (e.g., behaviour change triggering or microloans) and outcomes along the causal pathway, specifically behavioural outcomes (e.g., handwashing and food hygiene practices), ill-health outcomes (e.g., diarrhoeal morbidity and mortality), nutrition and socioeconomic outcomes (e.g., school absenteeism and household income). The map also provides filters to examine the evidence for a particular WASH technology (e.g., latrines), place of use (e.g., home, school or health facility), location (e.g., global region, country, rural and urban) and group (e.g., people living with disability). Systematic searches for published and unpublished literature and trial registries were conducted of studies in low- and middle-income countries (LMICs). Searches were conducted in March 2018, and searches for completed trials were done in May 2020. Coding of information for the map was done by two authors working independently. Impact evaluations were critically appraised according to methods of conduct and reporting. Systematic reviews were critically appraised using a new approach to assess theory-based, mixed-methods evidence synthesis.

Results

There has been an enormous growth in impact evaluations and systematic reviews of WASH interventions since the International Year of Sanitation, 2008. There are now at least 367 completed or ongoing rigorous impact evaluations in LMICs, nearly three-quarters of which have been conducted since 2008, plus 43 systematic reviews. Studies have been done in 83 LMICs, with a high concentration in Bangladesh, India, and Kenya. WASH sector programming has increasingly shifted in focus from what technology to supply (e.g., a handwashing station or child's potty), to the best way in which to do so to promote demand. Research also covers a broader set of intervention mechanisms. For example, there has been increased interest in behaviour change communication using psychosocial "triggering", such as social marketing and community-led total sanitation. These studies report primarily on behavioural outcomes. With the advent of large-scale funding, in particular by the Bill & Melinda Gates Foundation, there has been a substantial increase in the number of studies on sanitation technologies, particularly latrines. Sustaining behaviour is fundamental for sustaining health and other quality of life improvements. However, few studies have been done of intervention mechanisms for, or measuring outcomes on sustained adoption of latrines to stop open defaecation. There has also been some increase in the number of studies looking at outcomes and interventions that disproportionately affect women and girls, who quite literally carry most of the burden of poor water and sanitation access. However, most studies do not report sex disaggregated outcomes, let alone integrate gender analysis into their framework. Other vulnerable populations are even less addressed; no studies eligible for inclusion in the map were done of interventions targeting, or reporting on outcomes for, people living with disabilities. We were only able to find a single controlled evaluation of WASH interventions in a health care facility, in spite of the importance of WASH in health facilities in global policy debates. The quality of impact evaluations has improved, such as the use of controlled designs as standard, attention to addressing reporting biases, and adequate cluster sample size. However, there remain important concerns about quality of reporting. The quality and usefulness of systematic reviews for policy is also improving, which draw clearer distinctions between intervention mechanisms and synthesise the evidence on outcomes along the causal pathway. Adopting mixed-methods approaches also provides information for programmes on barriers and enablers affecting implementation.

Conclusion

Ensuring everyone has access to appropriate water, sanitation, and hygiene facilities is one of the most fundamental of challenges for poverty elimination. Researchers and funders need to consider carefully where there is the need for new primary evidence, and new syntheses of that evidence. This study suggests the following priority areas:Impact evaluations incorporating understudied outcomes, such as sustainability and slippage, of WASH provision in understudied places of use, such as health care facilities, and of interventions targeting, or presenting disaggregated data for, vulnerable populations, particularly over the life-course and for people living with a disability;Improved reporting in impact evaluations, including presentation of participant flow diagrams; andSynthesis studies and updates in areas with sufficient existing and planned impact evaluations, such as for diarrhoea mortality, ARIs, WASH in schools and decentralisation. These studies will preferably be conducted as mixed-methods systematic reviews that are able to answer questions about programme targeting, implementation, effectiveness and cost-effectiveness, and compare alternative intervention mechanisms to achieve and sustain outcomes in particular contexts, preferably using network meta-analysis.

Economic Aspects of Delivering Primary Care Services: An Evidence Synthesis to Inform Policy and Research Priorities

Policy Points The 2018 Declaration of Astana reemphasized the importance of primary health care and its role in achieving universal health coverage. While there is a large amount of literature on the economic aspects of delivering primary care services, there is a need for more comprehensive overviews of this evidence. In this article, we offer such an overview. Evidence suggests that there are several strategies involving coverage, financing, service delivery, and governance arrangements which can, if implemented, have positive economic impacts on the delivery of primary care services. These include arrangements such as worker task-shifting and telemedicine. The implementation of any such arrangements, based on positive economic evidence, should carefully account for potential impacts on overall health care access and quality. There are many opportunities for further research, with notable gaps in evidence on the impacts of increasing primary care funding or the overall supply of primary care services.

CONTEXT

The 2018 Declaration of Astana reemphasized the importance of primary health care and its role in achieving universal health coverage. To strengthen primary health care, policymakers need guidance on how to allocate resources in a manner that maximizes its economic benefits.

METHODS

We collated and synthesized published systematic reviews of evidence on the economic aspects of different models of delivering primary care services. Building on previous efforts, we adapted existing taxonomies of primary care components to classify our results according to four categories: coverage, financing, service delivery, and governance.

FINDINGS

We identified and classified 109 reviews that met our inclusion criteria according to our taxonomy of primary care components: coverage, financing, service delivery, and governance arrangements. A significant body of evidence suggests that several specific primary care arrangements, such as health workers' task shifting and telemedicine, can have positive economic impacts (such as lower overall health care costs). Notably absent were reviews on the impact of increasing primary care funding or the overall supply of primary care services.

CONCLUSIONS

There is a great opportunity for further research to systematically examine the broader economic impacts of investing in primary care services. Despite progress over the last decade, significant evidence gaps on the economic implications of different models of primary care services remain, which could help inform the basis of future research efforts.

Protocol: mixed-methods study of how implementation of US state medical cannabis laws affects treatment of chronic non-cancer pain and adverse opioid outcomes

BACKGROUND

Thirty-three US states and Washington, D.C., have enacted medical cannabis laws allowing patients with chronic non-cancer pain to use cannabis, when recommended by a physician, to manage their condition. However, clinical guidelines do not recommend cannabis for treatment of chronic non-cancer pain due to limited and mixed evidence of effectiveness. How state medical cannabis laws affect delivery of evidence-based treatment for chronic non-cancer pain is unclear. These laws could lead to substitution of cannabis in place of clinical guideline-discordant opioid prescribing, reducing risk of opioid use disorder and overdose. Conversely, state medical cannabis laws could lead to substitution of cannabis in place of guideline-concordant treatments such as topical analgesics or physical therapy. This protocol describes a mixed-methods study examining the implementation and effects of state medical cannabis laws on treatment of chronic non-cancer pain. A key contribution of the study is the examination of how variation in state medical cannabis laws' policy implementation rules affects receipt of chronic non-cancer pain treatments.

METHODS

The study uses a concurrent-embedded design. The primary quantitative component of the study employs a difference-in-differences design using a policy trial emulation approach. Quantitative analyses will evaluate state medical cannabis laws' effects on treatment for chronic non-cancer pain as well as on receipt of treatment for opioid use disorder, opioid overdose, cannabis use disorder, and cannabis poisoning among people with chronic non-cancer pain. Secondary qualitative and survey methods will be used to characterize implementation of state medical cannabis laws through interviews with state leaders and representative surveys of physicians who treat, and patients who experience, chronic non-cancer pain in states with medical cannabis laws.

DISCUSSION

This study will examine the effects of medical cannabis laws on patients' receipt of guideline-concordant non-opioid, non-cannabis treatments for chronic non-cancer pain and generate new evidence on the effects of state medical cannabis laws on adverse opioid outcomes. Results will inform the dynamic policy environment in which numerous states consider, enact, and/or amend medical cannabis laws each year.

Guidance for producing a Campbell evidence and gap map

Evidence and Gap Maps (EGMs) are a systematic evidence synthesis product which display the available evidence relevant to a specific research question. EGMs are produced following the same principles as a systematic reviews, that is: specify a PICOS, a comprehensive search, screening against explicit inclusion and exclusion criteria, and systematic coding, analysis and reporting. This paper provides guidance on producing EGMs for publication in Campbell Systematic Reviews.

The impact of an insecticide treated bednet campaign on all-cause child mortality: A geospatial impact evaluation from the Democratic Republic of Congo

OBJECTIVE

To test the impact of a nationwide Long-Lasting Insecticidal Nets [LLINs] distribution program in the Democratic Republic of Congo [DRC] on all-cause under-five child mortality exploiting subnational variation in malaria endemicity and the timing in the scale-up of the program across provinces.

DESIGN

Geospatial Impact Evaluation using a difference-in-differences approach.

SETTING

Democratic Republic of the Congo.

PARTICIPANTS

52,656 children sampled in the 2007 and 2013/2014 DRC Demographic and Health Surveys.

INTERVENTIONS

The analysis provides plausibly causal estimates of both average treatment effects of the LLIN distribution campaign and geospatial heterogeneity in these effects based on malaria endemicity. It compares the under-five, all-cause mortality for children pre- and post-LLIN campaign relative to children in those areas that had not yet been exposed to the campaign using a difference-in-differences model and controlling for year- and province-fixed effects, and province-level trends in mortality.

RESULTS

We find that the campaign led to a 41% decline [3.7 percentage points, 95% CI 1.3 to 6.0] in under-5 mortality risk among children living in rural areas with malaria ecology above the sample median. Results were robust to controlling for household assets and the presence of other health aid programs. No effect was detected in children living in areas with malaria ecology below the median.

CONCLUSION

The findings of this paper make important contributions to the evidence base for the effectiveness of large scale-national LLIN campaigns against malaria. We found that the program was effective in areas of the DRC with the highest underlying risk of malaria. Targeting bednets to areas with greatest underlying risk for malaria may help to increase the efficiency of increasingly limited malaria resources but should be balanced against other malaria control concerns.

Putting research in place: an innovative approach to providing contextualized evidence synthesis for decision makers

BACKGROUND

The Contextualized Health Research Synthesis Program (CHRSP), developed in 2007 by the Newfoundland and Labrador Centre for Applied Health Research, produces contextualized knowledge syntheses for health-system decision makers. The program provides timely, relevant, and easy-to-understand scientific evidence; optimizes evidence uptake; and, most importantly, attunes research questions and evidence to the specific context in which knowledge users must apply the findings.

METHODS

As an integrated knowledge translation (KT) method, CHRSP: Involves intensive partnerships with senior healthcare decision makers who propose priority research topics and participate on research teams; Considers local context both in framing the research question and in reporting the findings; Makes economical use of resources by utilizing a limited number of staff; Uses a combination of external and local experts; and Works quickly by synthesizing high-level systematic review evidence rather than primary studies. Although it was developed in the Canadian province of Newfoundland and Labrador, the CHRSP methodology is adaptable to a variety of settings with distinctive features, such as those in rural, remote, and small-town locations.

RESULTS

CHRSP has published 25 syntheses on priority topics chosen by the provincial healthcare system, including: Clinical and cost-effectiveness: telehealth, rural renal dialysis, point-of-care testing; Community-based health services: helping seniors age in place, supporting seniors with dementia, residential treatment centers for at-risk youth; Healthcare organization/service delivery: reducing acute-care length of stay, promoting flu vaccination among health workers, safe patient handling, age-friendly acute care; and Health promotion: diabetes prevention, promoting healthy dietary habits. These studies have been used by decision makers to inform local policy and practice decisions.

CONCLUSIONS

By asking the health system to identify its own priorities and to participate directly in the research process, CHRSP fully integrates KT among researchers and knowledge users in healthcare in Newfoundland and Labrador. This high level of decision-maker buy-in has resulted in a corresponding level of uptake. CHRSP studies have directly informed a number of policy and practice directions, including the design of youth residential treatment centers, a provincial policy on single-use medical devices, and most recently, the opening of the province's first Acute Care for the Elderly hospital unit.

Governance arrangements for health systems in low-income countries: an overview of systematic reviews

BACKGROUND

Governance arrangements include changes in rules or processes that determine authority and accountability for health policies, organisations, commercial products and health professionals, as well as the involvement of stakeholders in decision-making. Changes in governance arrangements can affect health and related goals in numerous ways, generally through changes in authority, accountability, openness, participation and coherence. A broad overview of the findings of systematic reviews can help policymakers, their technical support staff and other stakeholders to identify strategies for addressing problems and improving the governance of their health systems.

OBJECTIVES

To provide an overview of the available evidence from up-to-date systematic reviews about the effects of governance arrangements for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on governance arrangements and informing refinements of the framework for governance arrangements outlined in the overview.

METHODS

We searched Health Systems Evidence in November 2010 and PDQ Evidence up to 17 December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of governance arrangements on patient outcomes (health and health behaviours), the quality or utilisation of healthcare services, resource use (health expenditures, healthcare provider costs, out-of-pocket payments, cost-effectiveness), healthcare provider outcomes (such as sick leave), or social outcomes (such as poverty, employment) and that were published after April 2005. We excluded reviews with limitations that were important enough to compromise the reliability of the findings of the review. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries.

MAIN RESULTS

We identified 7272 systematic reviews and included 21 of them in this overview (19 primary reviews and 2 supplementary reviews). We focus here on the results of the 19 primary reviews, one of which had important methodological limitations. The other 18 were reliable (with only minor limitations).We grouped the governance arrangements addressed in the reviews into five categories: authority and accountability for health policies (three reviews); authority and accountability for organisations (two reviews); authority and accountability for commercial products (three reviews); authority and accountability for health professionals (seven reviews); and stakeholder involvement (four reviews).Overall, we found desirable effects for the following interventions on at least one outcome, with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects. Decision-making about what is covered by health insurance- Placing restrictions on the medicines reimbursed by health insurance systems probably decreases the use of and spending on these medicines (moderate-certainty evidence). Stakeholder participation in policy and organisational decisions- Participatory learning and action groups for women probably improve newborn survival (moderate-certainty evidence).- Consumer involvement in preparing patient information probably improves the quality of the information and patient knowledge (moderate-certainty evidence). Disclosing performance information to patients and the public- Disclosing performance data on hospital quality to the public probably encourages hospitals to implement quality improvement activities (moderate-certainty evidence).- Disclosing performance data on individual healthcare providers to the public probably leads people to select providers that have better quality ratings (moderate-certainty evidence).

AUTHORS' CONCLUSIONS

Investigators have evaluated a wide range of governance arrangements that are relevant for low-income countries using sound systematic review methods. These strategies have been targeted at different levels in health systems, and studies have assessed a range of outcomes. Moderate-certainty evidence shows desirable effects (with no undesirable effects) for some interventions. However, there are important gaps in the availability of systematic reviews and primary studies for the all of the main categories of governance arrangements.

An interpretive review of consensus statements on clinical guideline development and their application in the field of traditional and complementary medicine

BACKGROUND

Despite ongoing consumer demand and an emerging scientific evidence-base for traditional and complementary medicine (T&CM), there remains a paucity of reliable information in standard clinical guidelines about their use. Often T&CM interventions are not mentioned, or the recommendations arising from these guidelines are unhelpful to end-users (i.e. patients, practitioners and policy makers). Insufficient evidence of efficacy may be a contributing factor; however, often informative recommendations could still be made by drawing on relevant information from other avenues. In light of this, the aim of this research was to review national and internationally endorsed consensus statements for clinical guideline developers, and to interpret how to apply these methods when making recommendations regarding the use of T&CM.

METHOD

The critical interpretive review method was used to identify and appraise relevant consensus statements published between 1995 and 2015. The statements were identified using a purposive sampling technique until data saturation was reached. The most recent edition of a statement was included in the analysis. The content, scope and themes of the statements were compared and interpreted within the context of the T&CM setting; including history, regulation, use, emerging scientific evidence-base and existing guidelines.

RESULTS

Eight consensus statements were included in the interpretive review. Searching stopped at this stage as no new major themes were identified. The five themes relevant to the challenges of developing T&CM guidelines were: (1) framing the question; (2) the limitations of using an evidence hierarchy; (3) strategies for dealing with insufficient, high quality evidence; (4) the importance of qualifying a recommendation; and (5) the need for structured consensus development.

CONCLUSION

Evidence regarding safety, efficacy and cost effectiveness are not the only information required to make recommendations for clinical guidelines. Modifying factors such as burden of disease, magnitude of effect, current use, demand, equity and ease of integration should also be considered. Uptake of the recommendations arising from this review are expected to result in the development of higher quality clinical guidelines that offer greater assistance to those seeking answers about the appropriate use of T&CM.

Developments in Impact Assessment of New Diagnostic Algorithms for Tuberculosis Control

A modified presentation of the impact assessment framework is proposed that improves accessibility while continuing to provide a checklist of the evidence needed to support policy decisions on the implementation of new tools for the diagnosis of tuberculosis.

Using conjoint analysis to develop a system to score research engagement actions by health decision makers

Background

Effective use of research to inform policymaking can be strengthened by policymakers undertaking various research engagement actions (e.g., accessing, appraising, and applying research). Consequently, we developed a thorough measurement and scoring tool to assess whether and how policymakers undertook research engagement actions in the development of a policy document. This scoring tool breaks down each research engagement action into its key ‘subactions’ like a checklist. The primary aim was to develop the scoring tool further so that it assigned appropriate scores to each subaction based on its effectiveness for achieving evidence-informed policymaking. To establish the relative effectiveness of these subactions, we conducted a conjoint analysis, which was used to elicit the opinions and preferences of knowledge translation experts.

Method

Fifty-four knowledge translation experts were recruited to undertake six choice surveys. Respondents were exposed to combinations of research engagement subactions called ‘profiles’, and rated on a 1–9 scale whether each profile represented a limited (1–3), moderate (4–6), or extensive (7–9) example of each research engagement action. Generalised estimating equations were used to analyse respondents’ choice data, where a utility coefficient was calculated for each subaction. A large utility coefficient indicates that a subaction was influential in guiding experts’ ratings of extensive engagement with research.

Results

The calculated utilities were used as the points assigned to the subactions in the scoring system. The following subactions yielded the largest utilities and were regarded as the most important components of engaging with research: searching academic literature databases, obtaining systematic reviews and peer-reviewed research, appraising relevance by verifying its applicability to the policy context, appraising quality by evaluating the validity of the method and conclusions, engaging in thorough collaborations with researchers, and undertaking formal research projects to inform the policy in question.

Conclusions

We have generated an empirically-derived and context-sensitive method of measuring and scoring the extent to which policymakers engaged with research to inform policy development. The scoring system can be used by organisations to quantify staff research engagement actions and thus provide them with insights into what types of training, systems, and tools might improve their staff’s research use capacity.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-015-0013-z) contains supplementary material, which is available to authorized users.

Evaluation of an integrated system for classification, assessment and comparison of services for long-term care in Europe: the eDESDE-LTC study

Background

The harmonization of European health systems brings with it a need for tools to allow the standardized collection of information about medical care. A common coding system and standards for the description of services are needed to allow local data to be incorporated into evidence-informed policy, and to permit equity and mobility to be assessed. The aim of this project has been to design such a classification and a related tool for the coding of services for Long Term Care (DESDE-LTC), based on the European Service Mapping Schedule (ESMS).

Methods

The development of DESDE-LTC followed an iterative process using nominal groups in 6 European countries. 54 researchers and stakeholders in health and social services contributed to this process. In order to classify services, we use the minimal organization unit or “Basic Stable Input of Care” (BSIC), coded by its principal function or “Main Type of Care” (MTC). The evaluation of the tool included an analysis of feasibility, consistency, ontology, inter-rater reliability, Boolean Factor Analysis, and a preliminary impact analysis (screening, scoping and appraisal).

Results

DESDE-LTC includes an alpha-numerical coding system, a glossary and an assessment instrument for mapping and counting LTC. It shows high feasibility, consistency, inter-rater reliability and face, content and construct validity. DESDE-LTC is ontologically consistent. It is regarded by experts as useful and relevant for evidence-informed decision making.

Conclusion

DESDE-LTC contributes to establishing a common terminology, taxonomy and coding of LTC services in a European context, and a standard procedure for data collection and international comparison.

How to achieve universal coverage of cataract surgical services in developing countries: lessons from systematic reviews of other services

PURPOSE

Since the Declaration of Alma Ata, universal coverage has been at the heart of international health. The purpose of this study was to review the evidence on factors and interventions which are effective in promoting coverage and access to cataract and other health services, focusing on developing countries.

METHODS

A thorough literature search for systematic reviews was conducted. Information resources searched were Medline, The Cochrane Library and the Health System Evidence database. Medline was searched from January 1950 to June 2010. The Cochrane Library search consisted of identifying all systematic reviews produced by the Cochrane Eyes and Vision Group and the Cochrane Effective Practice and Organisation of Care. These reviews were assessed for potential inclusion in the review. The Health Systems Evidence database hosted by MacMaster University was searched to identify overviews of systematic reviews.

RESULTS

No reviews met the inclusion criteria for cataract surgery. The literature search on other health sectors identified 23 systematic reviews providing robust evidence on the main factors facilitating universal coverage. The main enabling factors influencing access to services in developing countries were peer education, the deployment of staff to rural areas, task shifting, integration of services, supervision of health staff, eliminating user fees and scaling up of health insurance schemes.

CONCLUSION

There are significant research gaps in eye care. There is a pressing need for further high quality primary research on health systems-related factors to understand how the delivery of eye care services and health systems' capacities are interrelated.

Implementation research evidence uptake and use for policy-making

A major obstacle to the progress of the Millennium Development Goals has been the inability of health systems in many low- and middle-income countries to effectively implement evidence-informed interventions. This article discusses the relationships between implementation research and knowledge translation and identifies the role of implementation research in the design and execution of evidence-informed policy. After a discussion of the benefits and synergies needed to translate implementation research into action, the article discusses how implementation research can be used along the entire continuum of the use of evidence to inform policy. It provides specific examples of the use of implementation research in national level programmes by looking at the scale up of zinc for the treatment of childhood diarrhoea in Bangladesh and the scaling up of malaria treatment in Burkina Faso. A number of tested strategies to support the transfer of implementation research results into policy-making are provided to help meet the standards that are increasingly expected from evidence-informed policy-making practices.

Knowledge translation of research findings

BACKGROUND

One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?

DISCUSSION

We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.

SUMMARY

There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.

Knowledge translation of research findings

BACKGROUND

One of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?

DISCUSSION

We suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.

SUMMARY

There is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.

Guidance for evidence-informed policies about health systems: assessing how much confidence to place in the research evidence

In the third paper in a three-part series on health systems guidance, Simon Lewin and colleagues explore the challenge of assessing how much confidence to place in evidence on health systems interventions.

SUPPORT Tools for evidence-informed health Policymaking (STP) 13: Preparing and using policy briefs to support evidence-informed policymaking

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

Policy briefs are a relatively new approach to packaging research evidence for policymakers. The first step in a policy brief is to prioritise a policy issue. Once an issue is prioritised, the focus then turns to mobilising the full range of research evidence relevant to the various features of the issue. Drawing on available systematic reviews makes the process of mobilising evidence feasible in a way that would not otherwise be possible if individual relevant studies had to be identified and synthesised for every feature of the issue under consideration. In this article, we suggest questions that can be used to guide those preparing and using policy briefs to support evidence-informed policymaking. These are: 1. Does the policy brief address a high-priority issue and describe the relevant context of the issue being addressed? 2. Does the policy brief describe the problem, costs and consequences of options to address the problem, and the key implementation considerations? 3. Does the policy brief employ systematic and transparent methods to identify, select, and assess synthesised research evidence? 4. Does the policy brief take quality, local applicability, and equity considerations into account when discussing the synthesised research evidence? 5. Does the policy brief employ a graded-entry format? 6. Was the policy brief reviewed for both scientific quality and system relevance?

SUPPORT tools for evidence-informed policymaking in health 6: Using research evidence to address how an option will be implemented

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. After a policy decision has been made, the next key challenge is transforming this stated policy position into practical actions. What strategies, for instance, are available to facilitate effective implementation, and what is known about the effectiveness of such strategies? We suggest five questions that can be considered by policymakers when implementing a health policy or programme. These are: 1. What are the potential barriers to the successful implementation of a new policy? 2. What strategies should be considered in planning the implementation of a new policy in order to facilitate the necessary behavioural changes among healthcare recipients and citizens? 3. What strategies should be considered in planning the implementation of a new policy in order to facilitate the necessary behavioural changes in healthcare professionals? 4. What strategies should be considered in planning the implementation of a new policy in order to facilitate the necessary organisational changes? 5. What strategies should be considered in planning the implementation of a new policy in order to facilitate the necessary systems changes?

SUPPORT Tools for evidence-informed health Policymaking (STP) 9: Assessing the applicability of the findings of a systematic review

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

Differences between health systems may often result in a policy or programme option that is used in one setting not being feasible or acceptable in another. Or these differences may result in an option not working in the same way in another setting, or even achieving different impacts in another setting. A key challenge that policymakers and those supporting them must face is therefore the need to understand whether research evidence about an option can be applied to their setting. Systematic reviews make this task easier by summarising the evidence from studies conducted in a variety of different settings. Many systematic reviews, however, do not provide adequate descriptions of the features of the actual settings in which the original studies were conducted. In this article, we suggest questions to guide those assessing the applicability of the findings of a systematic review to a specific setting. These are: 1. Were the studies included in a systematic review conducted in the same setting or were the findings consistent across settings or time periods? 2. Are there important differences in on-the-ground realities and constraints that might substantially alter the feasibility and acceptability of an option? 3. Are there important differences in health system arrangements that may mean an option could not work in the same way? 4. Are there important differences in the baseline conditions that might yield different absolute effects even if the relative effectiveness was the same? 5. What insights can be drawn about options, implementation, and monitoring and evaluation? Even if there are reasonable grounds for concluding that the impacts of an option might differ in a specific setting, insights can almost always be drawn from a systematic review about possible options, as well as approaches to the implementation of options and to monitoring and evaluation.

SUPPORT Tools for evidence-informed health Policymaking (STP) 7: Finding systematic reviews

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. Systematic reviews are increasingly seen as a key source of information in policymaking, particularly in terms of assisting with descriptions of the impacts of options. Relative to single studies they offer a number of advantages related to understanding impacts and are also seen as a key source of information for clarifying problems and providing complementary perspectives on options. Systematic reviews can be undertaken to place problems in comparative perspective and to describe the likely harms of an option. They also assist with understanding the meanings that individuals or groups attach to a problem, how and why options work, and stakeholder views and experiences related to particular options. A number of constraints have hindered the wider use of systematic reviews in policymaking. These include a lack of awareness of their value and a mismatch between the terms employed by policymakers, when attempting to retrieve systematic reviews, and the terms used by the original authors of those reviews. Mismatches between the types of information that policymakers are seeking, and the way in which authors fail to highlight (or make obvious) such information within systematic reviews have also proved problematic. In this article, we suggest three questions that can be used to guide those searching for systematic reviews, particularly reviews about the impacts of options being considered. These are: 1. Is a systematic review really what is needed? 2. What databases and search strategies can be used to find relevant systematic reviews? 3. What alternatives are available when no relevant review can be found?

SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers. Policymakers and those supporting them may find themselves in one or more of the following three situations that will require them to characterise the costs and consequences of options to address a problem. These are: 1. A decision has already been taken and their role is to maximise the benefits of an option, minimise its harms, optimise the impacts achieved for the money spent, and (if there is substantial uncertainty about the likely costs and consequences of the option) to design a monitoring and evaluation plan, 2. A policymaking process is already underway and their role is to assess the options presented to them, or 3. A policymaking process has not yet begun and their role is therefore to identify options, characterise the costs and consequences of these options, and look for windows of opportunity in which to act. In situations like these, research evidence, particularly about benefits, harms, and costs, can help to inform whether an option can be considered viable. In this article, we suggest six questions that can be used to guide those involved in identifying policy and programme options to address a high-priority problem, and to characterise the costs and consequences of these options. These are: 1. Has an appropriate set of options been identified to address a problem? 2. What benefits are important to those who will be affected and which benefits are likely to be achieved with each option? 3. What harms are important to those who will be affected and which harms are likely to arise with each option? 4. What are the local costs of each option and is there local evidence about their cost-effectiveness? 5. What adaptations might be made to any given option and could they alter its benefits, harms and costs? 6. Which stakeholder views and experiences might influence an option's acceptability and its benefits, harms, and costs?

SUPPORT Tools for evidence-informed health Policymaking (STP) 12: Finding and using research evidence about resource use and costs

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address considerations about resource use and costs. The consequences of a policy or programme option for resource use differ from other impacts (both in terms of benefits and harms) in several ways. However, considerations of the consequences of options for resource use are similar to considerations related to other impacts in that policymakers and their staff need to identify important impacts on resource use, acquire and appraise the best available evidence regarding those impacts, and ensure that appropriate monetary values have been applied. We suggest four questions that can be considered when assessing resource use and the cost consequences of an option. These are: 1. What are the most important impacts on resource use? 2. What evidence is there for important impacts on resource use? 3. How confident is it possible to be in the evidence for impacts on resource use? 4. Have the impacts on resource use been valued appropriately in terms of their true costs?

SUPPORT Tools for evidence-informed health Policymaking (STP) 17: Dealing with insufficient research evidence

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

In this article, we address the issue of decision making in situations in which there is insufficient evidence at hand. Policymakers often have insufficient evidence to know with certainty what the impacts of a health policy or programme option will be, but they must still make decisions. We suggest four questions that can be considered when there may be insufficient evidence to be confident about the impacts of implementing an option. These are: 1. Is there a systematic review of the impacts of the option? 2. Has inconclusive evidence been misinterpreted as evidence of no effect? 3. Is it possible to be confident about a decision despite a lack of evidence? 4. Is the option potentially harmful, ineffective or not worth the cost?

SUPPORT Tools for evidence-informed health Policymaking (STP)

This article is the Introduction to a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers.

Knowing how to find and use research evidence can help policymakers and those who support them to do their jobs better and more efficiently. Each article in this series presents a proposed tool that can be used by those involved in finding and using research evidence to support evidence-informed health policymaking. The series addresses four broad areas: 1. Supporting evidence-informed policymaking 2. Identifying needs for research evidence in relation to three steps in policymaking processes, namely problem clarification, options framing, and implementation planning 3. Finding and assessing both systematic reviews and other types of evidence to inform these steps, and 4. Going from research evidence to decisions. Each article begins with between one and three typical scenarios relating to the topic. These scenarios are designed to help readers decide on the level of detail relevant to them when applying the tools described. Most articles in this series are structured using a set of questions that guide readers through the proposed tools and show how to undertake activities to support evidence-informed policymaking efficiently and effectively. These activities include, for example, using research evidence to clarify problems, assessing the applicability of the findings of a systematic review about the effects of options selected to address problems, organising and using policy dialogues to support evidence-informed policymaking, and planning policy monitoring and evaluation. In several articles, the set of questions presented offers more general guidance on how to support evidence-informed policymaking. Additional information resources are listed and described in every article. The evaluation of ways to support evidence-informed health policymaking is a developing field and feedback about how to improve the series is welcome.