Designing a rapid response program to support evidence-informed decision-making in the Americas region: using the best available evidence and case studies

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Knowledge management tools and mechanisms for evidence-informed decision-making in the WHO European Region: a scoping review

BACKGROUND

Knowledge management (KM) emerged as a strategy to promote evidence-informed decision-making. This scoping review aims to map existing KM tools and mechanisms used to promote evidence-informed health decision-making in the WHO European Region and identify knowledge gaps.

METHODS

Following the Joanna Briggs Institute (JBI) guidance for conducting scoping reviews, we searched Medline, PubMed, EMBASE, the Cochrane library, and Open Grey. We conducted a descriptive analysis of the general characteristics of the included papers and conducted narrative analysis of the included studies and categorized studies according to KM type and phase.

RESULTS

Out of 9541 citations identified, we included 141 studies. The KM tools mostly assessed are evidence networks, surveillance tools, observatories, data platforms and registries, with most examining KM tools in high-income countries of the WHO European region. Findings suggest that KM tools can identify health problems, inform health planning and resource allocation, increase the use of evidence by policymakers and stimulate policy discussion.

CONCLUSION

Policymakers and funding agencies are called to support capacity-building activities, and future studies to strengthen KM in the WHO European region particularly in Eastern Europe and Central Asia. An updated over-arching strategy to coordinate KM activities in the WHO European region will be useful in these efforts.

Strategies to approach the judicialization of health in Brazil: an evidence brief

Abstract This article seeks to identify and discuss evidence-informed options to address the judicialization of health. The Supporting Policy Relevant Reviews and Trials Tools were used to define the problem and the search strategy, which was carried out in the following databases: PubMed, Health Systems Evidence, Campbell, Cochrane Collaboration, Rx for Change Database, and PDQ-Evidence. Selection and assessment of methodological quality was performed by two independent reviewers. The results were presented in a narrative synthesis. This study selected 19 systematic reviews that pointed out four strategies to address the judicialization of health in Brazil: 1) Rapid response service, 2) Continuous education program, 3) Mediation service between the parties involved, and 4) Adoption of a computer-based, online decision-making support tool and patient-mediated interventions. This study therefore presented and characterized four options that can be considered to address the judicialization of health. The implementation of these options must ensure the participation of different actors, reflecting on different contexts and the impact on the health system. The availability of human and financial resources and the training of teams are critical points for the successful implementation of the options.

Strategies to approach the judicialization of health in Brazil: an evidence brief

This article seeks to identify and discuss evidence-informed options to address the judicialization of health. The Supporting Policy Relevant Reviews and Trials Tools were used to define the problem and the search strategy, which was carried out in the following databases: PubMed, Health Systems Evidence, Campbell, Cochrane Collaboration, Rx for Change Database, and PDQ-Evidence. Selection and assessment of methodological quality was performed by two independent reviewers. The results were presented in a narrative synthesis. This study selected 19 systematic reviews that pointed out four strategies to address the judicialization of health in Brazil: 1) Rapid response service, 2) Continuous education program, 3) Mediation service between the parties involved, and 4) Adoption of a computer-based, online decision-making support tool and patient-mediated interventions. This study therefore presented and characterized four options that can be considered to address the judicialization of health. The implementation of these options must ensure the participation of different actors, reflecting on different contexts and the impact on the health system. The availability of human and financial resources and the training of teams are critical points for the successful implementation of the options.

Paper 2: Performing rapid reviews

BACKGROUND

Health policy-makers must often make decisions in compressed time frames and with limited resources. Hence, rapid reviews have become a pragmatic alternative to comprehensive systematic reviews. However, it is important that rapid review methods remain rigorous to support good policy development and decisions. There is currently little evidence about which streamlined steps in a rapid review are less likely to introduce unacceptable levels of uncertainty while still producing a product that remains useful to policy-makers.

METHODS

This paper summarizes current research describing commonly used methods and practices that are used to conduct rapid reviews and presents key considerations and options to guide methodological choices for a rapid review.

RESULTS

The most important step for a rapid review is for an experienced research team to have early and ongoing engagement with the people who have requested the review. A clear research protocol, derived from a needs assessment conducted with the requester, serves to focus the review, defines the scope of the rapid review, and guides all subsequent steps. Common recommendations for rapid review methods include tailoring the literature search in terms of databases, dates, and languages. Researchers can consider using a staged search to locate high-quality systematic reviews and then subsequently published primary studies. The approaches used for study screening and selection, data extraction, and risk-of-bias assessment should be tailored to the topic, researcher experience, and available resources. Many rapid reviews use a single reviewer for study selection, risk-of-bias assessment, or data abstraction, sometimes with partial or full verification by a second reviewer. Rapid reviews usually use a descriptive synthesis method rather than quantitative meta-analysis. Use of brief report templates and standardized production methods helps to speed final report publication.

CONCLUSIONS

Researchers conducting rapid reviews need to make transparent methodological choices, informed by stakeholder input, to ensure that rapid reviews meet their intended purpose. Transparency is critical because it is unclear how or how much streamlined methods can bias the conclusions of reviews. There are not yet internationally accepted standards for conducting or reporting rapid reviews. Thus, this article proposes interim guidance for researchers who are increasingly employing these methods.

Advancing scholarship on anti-racism within developmental science: Reflections on the special section and recommendations for future research

To date, theoretical and conceptual scholarship on anti-racism has been advanced through leading contributions from several disciplines (e.g., sociology, education, psychology). Still, there remain fewer empirical studies on anti-racism constructs, and we know little about the development of anti-racism among diverse youth across key stages of development. In this special section of Child Development, we sought to address this gap by highlighting scholarship in developmental science that attends to the development of anti-racism in children across contexts (e.g., families, schools) and developmental stages (e.g., early childhood through emerging adulthood). In our introduction to the special section, we review the collective contributions of included studies and outline recommendations for future research in the development of anti-racism in youth.

A comprehensive monitoring and evaluation framework for evidence to policy networks

OBJECTIVE

To describe the development of a framework for monitoring and evaluating knowledge translation (KT) networks.

METHOD

The framework was developed using mixed methods over four phases, including i) a targeted literature review of KT networks, activities and indicators, ii) two scoping reviews to further enhance the set of indicators, iii) peer-reviews by international KT experts and an online expert consultation, and iv) piloting.

RESULTS

A comprehensive theory of change (ToC) and indicators, both for the Network Secretariat and its participating member countries, were identified to develop the monitoring and evaluation framework. The framework includes (i) a ToC, including three key indicator domains across the results chain (outputs, short term outcomes, intermediate outcomes), and (ii) indicators for the three key domains, that can be selected depending on the stage of network maturity, along with suggested data collection methods. The three key indicator domains are 1) KT capacity and skill building; 2) network (structure, governance and leadership); and 3) KT/evidence-informed policy value and culture.

CONCLUSION

The monitoring and evaluation framework that links KT activities with policy and health outcomes fills an important gap in optimizing KT procedures, generating lessons learned and increasing accountability of major multipartner KT networks.

Mask decontamination methods (model N95) for respiratory protection: a rapid review

BACKGROUND

N95 respiratory protection masks are used by healthcare professionals to prevent contamination from infectious microorganisms transmitted by droplets or aerosols.

METHODS

We conducted a rapid review of the literature analyzing the effectiveness of decontamination methods for mask reuse. The database searches were carried out up to September 2020. The systematic review was conducted in a way which simplified the stages of a complete systematic review, due to the worldwide necessity for reliable fast evidences on this matter.

RESULTS

A total of 563 articles were retrieved of which 48 laboratory-based studies were selected. Fifteen decontamination methods were included in the studies. A total of 19 laboratory studies used hydrogen peroxide, 21 studies used ultraviolet germicidal irradiation, 4 studies used ethylene oxide, 11 studies used dry heat, 9 studies used moist heat, 5 studies used ethanol, two studies used isopropanol solution, 11 studies used microwave oven, 10 studies used sodium hypochlorite, 7 studies used autoclave, 3 studies used an electric rice cooker, 1 study used cleaning wipes, 1 study used bar soap, 1 study used water, 1 study used multi-purpose high-level disinfection cabinet, and another 1 study used chlorine dioxide. Five methods that are promising are as follows: hydrogen peroxide vapor, ultraviolet irradiation, dry heat, wet heat/pasteurization, and microwave ovens.

CONCLUSIONS

We have presented the best available evidence on mask decontamination; nevertheless, its applicability is limited due to few studies on the topic and the lack of studies on real environments.

Research use at the Ministry of Health in Burkina Faso: the decision-makers' perspective

Background

Despite the increased emphasis placed on the use of evidence for policy development, relatively few initiatives have been developed to support evidence-informed decision-making, especially in West Africa. Moreover, studies examining the conditions under which policy-makers use research-based evidence are still scarce, but they show that their attitudes and opinions about research are one of the main determinants of such use. In February 2017, Burkina Faso’s Minister of Health planned to create a unit to promote evidence-informed decision-making within the ministry. Before the unit was set up, documenting the attitudes towards research at the highest levels of his Ministry appeared profitable to the unit’s planning.

Method

Individual interviews were conducted by the author with 14 actors positioned to consider evidence during decision-making from the Burkina Faso’s Minister of health cabinet. An interview grid was used to explore several themes such as attitudes towards research, obstacles and facilitators to research use, example of research use in decision-making and finally, ways to increase decision-makers’ participation in knowledge transfer activities. Interviews were partially transcribed and analysed by the author.

Results

The results show a mixed attitude towards research and relatively little indication of research use reported by respondents. Important obstacles were identified: evidence inaccessibility, lack of implementation guidelines, absence of clear communication strategy and studies’ lack of relevance for decision-making. Many suggestions were proposed such as raising awareness, improving access and research communication and prioritizing interactions with researchers. Respondents agree with the low participation of decision-makers in knowledge transfer activities: more leadership from the senior officials was suggested and greater awareness of the importance of their presence.

Conclusions

The conclusion presents avenues for reflection and action to increase the potential impact of the knowledge transfer unit planned within the Ministry of Health of Burkina Faso. This innovative initiative will be impactful if the obstacles identified in this study and policy-makers’ preferences and needs are taken into account during its development and implementation.

[Institutionalizing evidence-based policies in BrazilInstitucionalización de las políticas informadas por evidencia]

Objective

To evaluate capacities, organizational arrangements, and barriers to the implementation of Evidence Centers (NEvs) as part of Brazil's Evidence-Informed Policy Network (EVIPNet).

Method

A mixed methods descriptive-analytical, multiple-case exploratory study was performed. Coordinators of active NEvs answered a questionnaire in three parts: participant characteristics, assessment of the capacity to "acquire, assess, adapt, and apply" evidence (4A), and open questions addressing organizational arrangements and barriers to the implementation of NEvs.

Results

The study included 15 NEvs, mostly from the Midwest; 73.3% were based in universities, while 20% were installed in state/city health departments or in the Ministry of Health. All coordinators had completed graduate training and 80% reported 1 to 5 years' experience with evidence-based policies as well as proficiency in English. None of the participants worked exclusively as NEv coordinator. NEv teams included health care professionals, students (undergraduate/graduate), professors, and civil servants from health departments. The data revealed high capacity to "acquire" and "assess" evidence, and low capacity to "adapt" and "apply" evidence. On average, three activities or products were developed yearly by each NEv, especially knowledge translation initiatives (systematic reviews and deliberative dialogues) and training for health care professionals, managers and undergraduate/graduate students. Five barrier categories were described: 1) financing, 2) network integration, 3) institutionalization of demands, 4) adaptive capacity, and 5) research communication skills to recommend actions at the local level.

Conclusions

Trained human resources associated with academic and research institutions are available to support evidence-informed policies. However, the sustainability of NEvs depends on coordinated action to ensure the capacity to adapt and apply evidence.

Brazilian guidelines for the management of brain-dead potential organ donors. The task force of the AMIB, ABTO, BRICNet, and the General Coordination of the National Transplant System

OBJECTIVE

To contribute to updating the recommendations for brain-dead potential organ donor management.

METHOD

A group of 27 experts, including intensivists, transplant coordinators, transplant surgeons, and epidemiologists, joined a task force formed by the General Coordination Office of the National Transplant System/Brazilian Ministry of Health (CGSNT-MS), the Brazilian Association of Intensive Care Medicine (AMIB), the Brazilian Association of Organ Transplantation (ABTO), and the Brazilian Research in Intensive Care Network (BRICNet). The questions were developed within the scope of the 2011 Brazilian Guidelines for Management of Adult Potential Multiple-Organ Deceased Donors. The topics were divided into mechanical ventilation, hemodynamic support, endocrine-metabolic management, infection, body temperature, blood transfusion, and use of checklists. The outcomes considered for decision-making were cardiac arrest, number of organs recovered or transplanted per donor, and graft function/survival. Rapid systematic reviews were conducted, and the quality of evidence of the recommendations was assessed using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system. Two expert panels were held in November 2016 and February 2017 to classify the recommendations. A systematic review update was performed in June 2020, and the recommendations were reviewed through a Delphi process with the panelists between June and July 2020.

RESULTS

A total of 19 recommendations were drawn from the expert panel. Of these, 7 were classified as strong (lung-protective ventilation strategy, vasopressors and combining arginine vasopressin to control blood pressure, antidiuretic hormones to control polyuria, serum potassium and magnesium control, and antibiotic use), 11 as weak (alveolar recruitment maneuvers, low-dose dopamine, low-dose corticosteroids, thyroid hormones, glycemic and serum sodium control, nutritional support, body temperature control or hypothermia, red blood cell transfusion, and goal-directed protocols), and 1 was considered a good clinical practice (volemic expansion).

CONCLUSION

Despite the agreement among panel members on most recommendations, the grade of recommendation was mostly weak. The observed lack of robust evidence on the topic highlights the importance of the present guideline to improve the management of brain-dead potential organ donors.

Assessing how information is packaged in rapid reviews for policy-makers and other stakeholders: a cross-sectional study

Background

Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice.

Methods

We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere.

Results

Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read.

Conclusions

Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored.

Protocol availability

The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Evaluation of the performance and achievements of the WHO Evidence-informed Policy Network (EVIPNet) Europe

Background

The Evidence-informed Policy Network (EVIPNet) is one of the key mechanisms introduced by WHO to reduce the research-to-policy gap. EVIPNet Europe was launched in 2012. We evaluated the performance and achievements of EVIPNet Europe with the overall aims (1) to inform future developments and strategic planning of EVIPNet Europe and (2) to contribute to the evidence base for organisational knowledge translation activities by sharing the lessons learnt.

Methods

The evaluation covered the WHO Secretariat of EVIPNet Europe and its 21 member countries, from its inception to mid-2018. A mixed methods design was used to assess changes in three domains, including triangulation of quantitative and qualitative methods, based on the EVIPNet Europe Monitoring & Evaluation framework and theory of change. Data were collected between August and October 2018. Data collection comprised documentary review, social media analysis, online country evaluation, key informant interviews and validated tools. Two case studies were also developed.

Results

The evaluation showed promising results as well as lessons to guide the future development of EVIPNet in the WHO European Region and other regions of the world. EVIPNet Europe appears to be filling a niche in promoting the capacity of Network member countries for evidence-informed policy-making. There is evidence that EVIPNet Europe’s capacity-building programme of work is improving knowledge and skills at the individual level. There has been an increase in activity and outputs since its establishment and evidence has been used to inform new policies in some member countries. However, the speed at which member countries are developing or publishing products varies greatly and no formalised knowledge translation platforms have yet been created. Financial and human resources are limited and staff turnover is a cause for concern, both at the WHO Secretariat and country team levels.

Conclusions

Six years since the launch of EVIPNet Europe, the Network has grown quickly, is clearly valued and has had some successes. However, more work and support are needed if it is to achieve its vision of a Europe in which high-quality, context-sensitive evidence routinely informs health decision-making processes that ultimately serve to strengthen health outcomes across the Region.

Assessing the format and content of journal published and non-journal published rapid review reports: A comparative study

BACKGROUND

As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured.

OBJECTIVES

To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs.

METHODS

JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented.

RESULTS

We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups.

CONCLUSIONS

This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice.

STUDY REGISTRATION

The full protocol is available at: https://osf.io/29xvk/.

Strategies Implemented by Public Institutions to Approach the Judicialization of Health Care in Brazil: A Systematic Scoping Review

Background

The judicialization of health care is a social claim concerning the right to the access to health care. It usually occurs due to gaps in public policy or failures in its application. In Brazil, several public institutions have implemented strategies to approach this phenomenon. However, these strategies have not yet been systematized into functional categories.

Objective

To categorize and analyze the strategies implemented by public institutions in Brazil to approach the judicialization of health care.

Method

A systematic scoping review was developed following the method proposed by the Joanna Briggs Institute. The descriptor ‘judicialization of health’ was used to conduct the searches for studies in 18 electronic databases and other types of documents in the gray literature until March 2019. Documents containing the reports of strategies implemented in public institutions to approach the judicialization of health care in Brazil were included. Two independent reviewers assessed the eligibility of the documents and extracted the data. The strategies identified were categorized using definitions from the World Health Organization and existing Brazilian legislation.

Results

Seventy eight implemented strategies were identified and organized into nine categories: i. Technical support to the judiciary; ii. State health committees; iii. Organization of assistance; iv. Compliance with court orders, v. Computerized information systems; vi. Administrative proceeding; vii. Defense of the public authority; viii. Pharmacy and therapeutics committee; ix. Alternative dispute resolution. These categories are not mutually exclusive and often act in concert or complement each other’s activities. They represent services either existing or provided for in legal provisions by the public administration to meet various types of demands.

Conclusions

The categories proposed to approach the judicialization of health care represent some of the recommendations for qualifying public administration or are provided for in Brazilian legislation, or both. The existence of recommendations and legislation facilitate, but do not guarantee, the implementation of strategies by public institutions.

Realist review as a methodology for using evidence in health policies: an integrative review

OBJECTIVE

To identify and analyze the concepts of realist review and the recommended methodology for its development in the health area.

METHOD

An integrative review which sought theoretical-methodological qualitative or quantitative studies through 11 data sources which analyzed or proposed theories and/or realist review methodology.

RESULTS

Twenty-three (23) studies, mainly published in the United Kingdom, were included. The realist review aims at explaining what makes a complex policy, program, or intervention work, in which aspects, for whom, in what context, to what extent, and why, by constructing theory, decompressing the functioning mechanisms, complex policy contexts or programs which generate results. Such methodology relies on the involvement of stakeholders in the problem from the early stages of the review process.

CONCLUSION

The realist review has the potential to provide relevant and applicable subsidies for researchers, workers, policy makers and health decision makers.

Evaluation of an HIV-specific rapid response service for community-based organisations in Ontario, Canada

Background

To support AIDS service organisations and other community-based organisations’ use of research evidence to inform HIV-related programmes, services and policies, the Ontario HIV Treatment Network (OHTN) developed a Rapid Response Service. The final product of the rapid response process at the OHTN, which is more streamlined than that of traditional systematic reviews, consists of a detailed report answering questions regarding an HIV-specific issue and how the findings apply within the local context. In 2016, the OHTN conducted an evaluation to assess the effectiveness of its Rapid Response Service. This article reports on the development of this service as well as the results of the evaluation.

Methods

All rapid responses published between January 1, 2009, and September 30, 2016, by the OHTN (n = 102) were analysed using univariate analyses. Frequency distributions were determined for the following variables for each rapid response: populations observed, topics covered, requestor affiliations and number of downloads from the OHTN’s website. Requestors of rapid responses were also interviewed regarding perceived helpfulness and utility of the service and final products, and suggestions for changes to the service. Six-month follow-up interviews were conducted to determine how affiliated organisations used the evidence from the rapid response they requested.

Results

The 102 rapid responses published covered 14 different populations of interest. Topics covered included the HIV prevention, engagement and care cascade, determinants of health, syndemics, and comorbidities. Requestor affiliations consisted of AIDS service organisations, government agencies and policy-makers, non-HIV-focused community-based organisations, and hospitals, universities or health centres. Requestors perceived most aspects of the Rapid Response Service as very helpful and most frequently suggested that the rapid responses should provide recommendations. Follow-up interviews regarding the impact of rapid responses show that rapid responses have been used to assist organisations in numerous activities.

Conclusions

Organisations that have used the OHTN’s Rapid Response Service describe it as a valuable service useful for the development of programmes and policies. Improvements in capacity-building efforts may increase its utility. Describing the findings of this evaluation may serve as a reference for similar programmes to increase the use of research evidence among public health decision-makers.

Electronic supplementary material

The online version of this article (10.1186/s12961-019-0476-4) contains supplementary material, which is available to authorized users.

Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study

OBJECTIVE

To simulate possible changes in systematic review results if rapid review methods were used.

STUDY DESIGN AND SETTING

We recalculated meta-analyses for binary primary outcomes in Cochrane systematic reviews, simulating rapid review methods. We simulated searching only PubMed, excluding older articles (5, 7, 10, 15, and 20 years before the search date), excluding smaller trials (<50, <100, and <200 participants), and using the largest trial only. We examined percentage changes in pooled odds ratios (ORs) (classed as no important change [<5%], small [<20%], moderate [<30%], or large [≥30%]), statistical significance, and biases observed using rapid methods.

RESULTS

Two thousand five hundred and twelve systematic reviews (16,088 studies) were included. Rapid methods resulted in the loss of all data in 3.7-44.7% of meta-analyses. Searching only PubMed had the smallest risk of changed ORs (19% [477/2,512] were small changes or greater; 10% [260/2,512] were moderate or greater). Changes in ORs varied substantially with each rapid review method; 8.4-21.3% were small, 1.9-8.8% were moderate, and 4.7-34.1% were large. Changes in statistical significance occurred in 6.5-38.6% of meta-analyses. Changes from significant to nonsignificant were most common (2.1-13.7% meta-analyses). We found no evidence of bias with any rapid review method.

CONCLUSION

Searching PubMed only might be considered where a ∼10% risk of the primary outcome OR changing by >20% could be tolerated. This could be the case in scoping reviews, resource limitation, or where syntheses are needed urgently. Other situations, such as clinical guidelines and regulatory decisions, favor more comprehensive systematic review methods.

Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study

Objective

To simulate possible changes in systematic review results if rapid review methods were used.

Study Design and Setting

We recalculated meta-analyses for binary primary outcomes in Cochrane systematic reviews, simulating rapid review methods. We simulated searching only PubMed, excluding older articles (5, 7, 10, 15, and 20 years before the search date), excluding smaller trials (<50, <100, and <200 participants), and using the largest trial only. We examined percentage changes in pooled odds ratios (ORs) (classed as no important change [<5%], small [<20%], moderate [<30%], or large [≥30%]), statistical significance, and biases observed using rapid methods.

Results

Two thousand five hundred and twelve systematic reviews (16,088 studies) were included. Rapid methods resulted in the loss of all data in 3.7–44.7% of meta-analyses. Searching only PubMed had the smallest risk of changed ORs (19% [477/2,512] were small changes or greater; 10% [260/2,512] were moderate or greater). Changes in ORs varied substantially with each rapid review method; 8.4–21.3% were small, 1.9–8.8% were moderate, and 4.7–34.1% were large. Changes in statistical significance occurred in 6.5–38.6% of meta-analyses. Changes from significant to nonsignificant were most common (2.1–13.7% meta-analyses). We found no evidence of bias with any rapid review method.

Conclusion

Searching PubMed only might be considered where a ∼10% risk of the primary outcome OR changing by >20% could be tolerated. This could be the case in scoping reviews, resource limitation, or where syntheses are needed urgently. Other situations, such as clinical guidelines and regulatory decisions, favor more comprehensive systematic review methods.

Rapid response in health technology assessment: a Delphi study for a Brazilian guideline

Background

Rapid response in health technology assessment is a synthesis of the best available evidence prepared in a timely manner to meet specific demands. We build a consensus among Brazilian specialists in health technology assessment to propose guidelines for the development of rapid response.

Methods

Based on a systematic review that proposed eight methodological steps to conduct rapid response, we applied a modified Delphi technique (without open questions in the first round) to reach consensus among Brazilian experts in health technology assessment. Twenty participants were invited to judge the feasibility of each methodological step in a five-point Likert scale. Consensus was reached if the step had 70% positive approval or interquartile range ≤ 1.

Results

The achievement of consensus was reached in the second round. Between the first and the second round, we scrutinized all points reported by the experts. The Delphi panel reached consensus of eight steps: definition of the structured question of rapid response (with a restricted scope); definition of the eligibility criteria for study types (preferably systematic reviews); search strategy (language and data limits) and sources of information (minimum two); selection of studies (independently by two responders); critical appraisal of the included studies and the risk of bias for the outcomes of interest; data extraction from the included articles; summary of evidence; and preparation of the report.

Conclusions

The guidelines for rapid response in health technology assessment may help governments to make better decisions in a short period of time (35 days). The adoption of methodological processes should improve both the quality and consistency of health technology assessments of rapid decisions in the Brazilian setting.

Five shared decision-making tools in 5 months: use of rapid reviews to develop decision boxes for seniors living with dementia and their caregivers

BACKGROUND

Decision support tools build upon comprehensive and timely syntheses of literature. Rapid reviews may allow supporting their development by omitting certain components of traditional systematic reviews. We thus aimed to describe a rapid review approach underlying the development of decision support tools, i.e., five decision boxes (DB) for shared decision-making between seniors living with dementia, their caregivers, and healthcare providers.

METHOD

We included studies based on PICO questions (Participant, Intervention, Comparison, Outcome) describing each of the five specific decision. We gave priority to higher quality evidence (e.g., systematic reviews). For each DB, we first identified secondary sources of literature, namely, clinical summaries, clinical practice guidelines, and systematic reviews. After an initial extraction, we searched for primary studies in academic databases and grey literature to fill gaps in evidence. We extracted study designs, sample sizes, populations, and probabilities of benefits/harms of the health options. A single reviewer conducted the literature search and study selection. The data extracted by one reviewer was verified by a second experienced reviewer. Two reviewers assessed the quality of the evidence. We converted all probabilities into absolute risks for ease of understanding. Two to five experts validated the content of each DB. We conducted descriptive statistical analyses on the review processes and resources required.

RESULTS

The approach allowed screening of a limited number of references (range: 104 to 406/review). For each review, we included 15 to 26 studies, 2 to 10 health options, 11 to 62 health outcomes and we conducted 9 to 47 quality assessments. A team of ten reviewers with varying levels of expertise was supported at specific steps by an information specialist, a biostatistician, and a graphic designer. The time required to complete a rapid review varied from 7 to 31 weeks per review (mean ± SD, 19 ± 10 weeks). Data extraction required the most time (8 ± 6.8 weeks). The average estimated cost of a rapid review was C$11,646 (SD = C$10,914).

CONCLUSIONS

This approach enabled the development of clinical tools more rapidly than with a traditional systematic review. Future studies should evaluate the applicability of this approach to other teams/tools.

What are the best methodologies for rapid reviews of the research evidence for evidence-informed decision making in health policy and practice: a rapid review

Background

Rapid reviews have the potential to overcome a key barrier to the use of research evidence in decision making, namely that of the lack of timely and relevant research. This rapid review of systematic reviews and primary studies sought to answer the question: What are the best methodologies to enable a rapid review of research evidence for evidence-informed decision making in health policy and practice?

Methods

This rapid review utilised systematic review methods and was conducted according to a pre-defined protocol including clear inclusion criteria (PROSPERO registration: CRD42015015998). A comprehensive search strategy was used, including published and grey literature, written in English, French, Portuguese or Spanish, from 2004 onwards. Eleven databases and two websites were searched. Two review authors independently applied the eligibility criteria. Data extraction was done by one reviewer and checked by a second. The methodological quality of included studies was assessed independently by two reviewers. A narrative summary of the results is presented.

Results

Five systematic reviews and one randomised controlled trial (RCT) that investigated methodologies for rapid reviews met the inclusion criteria. None of the systematic reviews were of sufficient quality to allow firm conclusions to be made. Thus, the findings need to be treated with caution. There is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting rapid reviews. While a wide range of ‘shortcuts’ are used to make rapid reviews faster than a full systematic review, the included studies found little empirical evidence of their impact on the conclusions of either rapid or systematic reviews. There is some evidence from the included RCT (that had a low risk of bias) that rapid reviews may improve clarity and accessibility of research evidence for decision makers.

Conclusions

Greater care needs to be taken in improving the transparency of the methods used in rapid review products. There is no evidence available to suggest that rapid reviews should not be done or that they are misleading in any way. We offer an improved definition of rapid reviews to guide future research as well as clearer guidance for policy and practice.

Electronic supplementary material

The online version of this article (doi:10.1186/s12961-016-0155-7) contains supplementary material, which is available to authorized users.