Policymaker experiences with rapid response briefs to address health-system and technology questions in Uganda

Research-to-Policy Partnerships for Evidence-Informed Resource Allocation in Health Systems in Africa: An Example Using the Thanzi Programme

OBJECTIVES

Empirical data on the impact of research-to-policy interventions are scant, with the few attempts mainly focusing on ensuring policymakers' timely access to evidence and evidence-informed dialogs.

METHODS

This article reflects on how the Thanzi Programme cultivates an approach of research-to-policy engagement in health economics. The program is structured around 3 interrelated pillars comprising research evidence generation, capacity and capability building, and research-and-policy engagement. Each pillar is described and examples from the Thanzi Programme are given, including illustrating how each pillar informs the other. Limitations and challenges of the approach are discussed, with examples of a way forward.

RESULTS

This program supports health system strengthening through addressing gaps identified by program partners. This includes providing health economics training and research and strengthened partnerships between in-country researchers and health policymakers, as well as between national and international researchers. Platforms bringing together researchers and policymakers to shape the research agenda, disseminate evidence, and foster an evidence-based dialog are institutionalized at country and regional levels. Health Economics and Policy Units have been established, which sit between the Ministries of Health and Universities, to augment policymakers and health economics researchers' engagements on priority health policy matters and determine researchable policy questions. The establishment of the Health Economics Community of Practice as a substantive expert committee under the East Central and Southern Africa Health Community bolsters the contribution of health economics evidence in policy processes at the regional level.

CONCLUSIONS

The Thanzi Programme is an example of how a research-and-policy partnership framework is being used to support evidence-informed health resource allocation decisions in Africa. It uses a combination of high-quality multidisciplinary research, sustained research and policymakers' engagement and capacity strengthening to use research evidence to guide and support policy makers more effectively.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Paper 3: Selecting rapid review methods for complex questions related to health policy and system issues

Approaches for rapid reviews that focus on streamlining systematic review methods are not always suitable for exploring complex policy questions, as developing and testing theories to explain these complexities requires configuring diverse qualitative, quantitative, and mixed methods studies. Our objective was therefore to provide a guide to selecting approaches for rapidly (i.e., within days to months) addressing complex questions related to health policy and system issues.We provide a two-stage, transdisciplinary collaborative process to select a rapid review approach to address complex policy questions, which consists of scoping the breadth and depth of the literature and then selecting an optimal approach to synthesis. The first stage (scoping the literature) begins with a discussion with the stakeholders requesting evidence to identify and refine the question for the review, which is then used to conduct preliminary searches and conceptually map the documents identified. In the second stage (selection of an optimal approach), further stakeholder consultation is required to refine and tailor the question and approach to identifying relevant documents to include. The approach to synthesizing the included documents is then guided by the final question, the breadth and depth of the literature, and the time available and can include a static or evolving conceptual framework to code and analyze a range of evidence. For areas already covered extensively by existing systematic reviews, the focus can be on summarizing and integrating the review findings, resynthesizing the primary studies, or updating the search and reanalyzing one or more of the systematic reviews.The choice of approaches for conducting rapid reviews is intertwined with decisions about how to manage projects, the amount of work to be done, and the knowledge already available, and our guide offers support to help make these strategic decisions.

Lessons learned from descriptions and evaluations of knowledge translation platforms supporting evidence-informed policy-making in low- and middle-income countries: a systematic review

BACKGROUND

Knowledge translation (KT) platforms are organisations, initiatives and networks that focus on supporting evidence-informed policy-making at least in part about the health-system arrangements that determine whether the right programmes, services and products get to those who need them. Many descriptions and evaluations of KT platforms in low- and middle-income countries have been produced but, to date, they have not been systematically reviewed.

METHODS

We identified potentially relevant studies through a search of five electronic databases and a variety of approaches to identify grey literature. We used four criteria to select eligible empirical studies. We extracted data about seven characteristics of included studies and about key findings. We used explicit criteria to assess study quality. In synthesising the findings, we gave greater attention to themes that emerged from multiple studies, higher-quality studies and different contexts.

RESULTS

Country was the most common jurisdictional focus of KT platforms, EVIPNet the most common name and high turnover among staff a common infrastructural feature. Evidence briefs and deliberative dialogues were the activities/outputs that were the most extensively studied and viewed as helpful, while rapid evidence services were the next most studied but only in a single jurisdiction. None of the summative evaluations used a pre-post design or a control group and, with the exception of the evaluations of the influence of briefs and dialogues on intentions to act, none of the evaluations achieved a high quality score.

CONCLUSIONS

A large and growing volume of research evidence suggests that KT platforms offer promise in supporting evidence-informed policy-making in low- and middle-income countries. KT platforms should consider as next steps expanding their current, relatively limited portfolio of activities and outputs, building bridges to complementary groups, and planning for evaluations that examine 'what works' for 'what types of issues' in 'what types of contexts'.

Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews

OBJECTIVES

To develop methods guidance to support the conduct of rapid reviews (RRs) produced within Cochrane and beyond, in response to requests for timely evidence syntheses for decision-making purposes including urgent health issues of high priority.

STUDY DESIGN AND SETTING

Interim recommendations were informed by a scoping review of the underlying evidence, primary methods studies conducted, and a survey sent to 119 representatives from 20 Cochrane entities, who were asked to rate and rank RR methods across stages of review conduct. Discussions among those with expertise in RR methods further informed the list of recommendations with accompanying rationales provided.

RESULTS

Based on survey results from 63 respondents (53% response rate), 26 RR methods recommendations are presented for which there was a high or moderate level of agreement or scored highest in the absence of such agreement. Where possible, how recommendations align with Cochrane methods guidance for systematic reviews is highlighted.

CONCLUSION

The Cochrane Rapid Reviews Methods Group offers new, interim guidance to support the conduct of RRs. Because best practice is limited by the lack of currently available evidence for some RR methods shortcuts taken, this guidance will need to be updated as additional abbreviated methods are evaluated.

Assessing how information is packaged in rapid reviews for policy-makers and other stakeholders: a cross-sectional study

Background

Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice.

Methods

We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere.

Results

Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read.

Conclusions

Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored.

Protocol availability

The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Development of a checklist for people communicating evidence-based information about the effects of healthcare interventions: a mixed methods study

OBJECTIVES

To make informed decisions about healthcare, patients and the public, health professionals and policymakers need information about the effects of interventions. People need information that is based on the best available evidence; that is presented in a complete and unbiased way; and that is relevant, trustworthy and easy to use and to understand. The aim of this paper is to provide guidance and a checklist to those producing and communicating evidence-based information about the effects of interventions intended to inform decisions about healthcare.

DESIGN

To inform the development of this checklist, we identified research relevant to communicating evidence-based information about the effects of interventions. We used an iterative, informal consensus process to synthesise our recommendations. We began by discussing and agreeing on some initial recommendations, based on our own experience and research over the past 20-30 years. Subsequent revisions were informed by the literature we examined and feedback. We also compared our recommendations to those made by others. We sought structured feedback from people with relevant expertise, including people who prepare and use information about the effects of interventions for the public, health professionals or policymakers.

RESULTS

We produced a checklist with 10 recommendations. Three recommendations focus on making it easy to quickly determine the relevance of the information and find the key messages. Five recommendations are about helping the reader understand the size of effects and how sure we are about those estimates. Two recommendations are about helping the reader put information about intervention effects in context and understand if and why the information is trustworthy.

CONCLUSIONS

These 10 recommendations summarise lessons we have learnt developing and evaluating ways of helping people to make well-informed decisions by making research evidence more understandable and useful for them. We welcome feedback for how to improve our advice.

Stakeholders' experiences with the evidence aid website to support 'real-time' use of research evidence to inform decision-making in crisis zones: a user testing study

Background

Humanitarian action in crisis zones is fraught with many challenges, including lack of timely and accessible research evidence to inform decision-making about humanitarian interventions. Evidence websites have the potential to address this challenge. Evidence Aid is the only evidence website designed for crisis zones that focuses on providing research evidence in the form of systematic reviews. The objective of this study is to explore stakeholders’ views of Evidence Aid, contributing further to our understanding of the use of research evidence in decision-making in crisis zones.

Methods

We designed a qualitative user-testing study to collect interview data from stakeholders about their impressions of Evidence Aid. Eligible stakeholders included those with and without previous experience of Evidence Aid. All participants were either currently working or have worked within the last year in a crisis zone. Participants were asked to perform the same user experience-related tasks and answer questions about this experience and their knowledge needs. Data were analysed using a deductive framework analysis approach drawing on Morville’s seven facets of the user experience — findability, usability, usefulness, desirability, accessibility, credibility and value.

Results

A total of 31 interviews were completed with senior decision-makers (n = 8), advisors (n = 7), field managers (n = 7), analysts/researchers (n = 5) and healthcare providers (n = 4). Participant self-reported knowledge needs varied depending on their role. Overall, participants did not identify any ‘major’ problems (highest order) and identified only two ‘big’ problems (second highest order) with using the Evidence Aid website, namely the lack of a search engine on the home page and that some full-text articles linked to/from the site require a payment. Participants identified seven specific suggestions about how to improve Evidence Aid, many of which can also be applied to other evidence websites.

Conclusions

Stakeholders in crisis zones found Evidence Aid to be useful, accessible and credible. However, they experienced some problems with the lack of a search engine on the home page and the requirement for payment for some full-text articles linked to/from the site.

Evaluation of an HIV-specific rapid response service for community-based organisations in Ontario, Canada

Background

To support AIDS service organisations and other community-based organisations’ use of research evidence to inform HIV-related programmes, services and policies, the Ontario HIV Treatment Network (OHTN) developed a Rapid Response Service. The final product of the rapid response process at the OHTN, which is more streamlined than that of traditional systematic reviews, consists of a detailed report answering questions regarding an HIV-specific issue and how the findings apply within the local context. In 2016, the OHTN conducted an evaluation to assess the effectiveness of its Rapid Response Service. This article reports on the development of this service as well as the results of the evaluation.

Methods

All rapid responses published between January 1, 2009, and September 30, 2016, by the OHTN (n = 102) were analysed using univariate analyses. Frequency distributions were determined for the following variables for each rapid response: populations observed, topics covered, requestor affiliations and number of downloads from the OHTN’s website. Requestors of rapid responses were also interviewed regarding perceived helpfulness and utility of the service and final products, and suggestions for changes to the service. Six-month follow-up interviews were conducted to determine how affiliated organisations used the evidence from the rapid response they requested.

Results

The 102 rapid responses published covered 14 different populations of interest. Topics covered included the HIV prevention, engagement and care cascade, determinants of health, syndemics, and comorbidities. Requestor affiliations consisted of AIDS service organisations, government agencies and policy-makers, non-HIV-focused community-based organisations, and hospitals, universities or health centres. Requestors perceived most aspects of the Rapid Response Service as very helpful and most frequently suggested that the rapid responses should provide recommendations. Follow-up interviews regarding the impact of rapid responses show that rapid responses have been used to assist organisations in numerous activities.

Conclusions

Organisations that have used the OHTN’s Rapid Response Service describe it as a valuable service useful for the development of programmes and policies. Improvements in capacity-building efforts may increase its utility. Describing the findings of this evaluation may serve as a reference for similar programmes to increase the use of research evidence among public health decision-makers.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0476-4) contains supplementary material, which is available to authorized users.

Making Smarter Decisions Faster: Systems Engineering to Improve the Global Public Health Response to HIV

PURPOSE OF REVIEW

This review offers an operational definition of systems engineering (SE) as applied to public health, reviews applications of SE in the field of HIV, and identifies opportunities and challenges of broader application of SE in global health.

RECENT FINDINGS

SE involves the deliberate sequencing of three steps: diagnosing a problem, evaluating options using modeling or optimization, and providing actionable recommendations. SE includes diverse tools (from process improvement to mathematical modeling) applied to decisions at various levels (from local staffing decisions to planning national-level roll-out of new interventions). Contextual factors are crucial to effective decision-making, but there are gaps in understanding global decision-making processes. Integrating SE into pre-service training and translating SE tools to be more accessible could increase utilization of SE approaches in global health. SE is a promising, but under-recognized approach to improve public health response to HIV globally.

Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries

BACKGROUND

Systematic reviews are an important source of evidence for public health decision-making, but length and technical jargon tend to hinder their use. In non-English speaking countries, inaccessibility of information in the native language often represents an additional barrier. In line with our vision to strengthen evidence-based public health in the German-speaking world, we developed a German language summary format for systematic reviews of public health interventions and undertook user-testing with public health decision-makers in Germany, Austria and Switzerland.

METHODS

We used several guiding principles and core elements identified from the literature to produce a prototype summary format and applied it to a Cochrane review on the impacts of changing portion and package sizes on selection and consumption of food, alcohol and tobacco. Following a pre-test in each of the three countries, we carried out 18 user tests with public health decision-makers in Germany, Austria and Switzerland using the 'think-aloud' method. We analysed participants' comments according to the facets credibility, usability, understandability, usefulness, desirability, findability, identification and accessibility. We also identified elements that hindered the facile and satisfying use of the summary format, and revised it based on participants' feedback.

RESULTS

The summary format was well-received; participants particularly appreciated receiving information in their own language. They generally found the summary format useful and a credible source of information, but also signalled several barriers to a positive user experience such as an information-dense structure and difficulties with understanding statistical terms. Many of the identified challenges were addressed through modifications of the summary format, in particular by allowing for flexible length, placing more emphasis on key messages and relevance for public health practice, expanding the interpretation aid for statistical findings, providing a glossary of technical terms, and only including graphical GRADE ratings. Some barriers to uptake, notably the participants' wish for actionable recommendations and contextual information, could not be addressed.

CONCLUSIONS

Participants welcomed the initiative, but user tests also revealed their problems with understanding and interpreting the findings summarised in our prototype format. The revised summary format will be used to communicate the results of Cochrane reviews of public health interventions.