The effectiveness of evidence summaries on health policymakers and health system managers use of evidence from systematic reviews: a systematic review

Effects of a nurse-led eHealth programme on functional outcomes and quality of life of patients with stroke: a pooled analysis of randomized controlled trials

Background

Stroke remains a leading cause of disability worldwide. Nurse-led eHealth programs have emerged as a potentially effective strategy to improve functional outcomes and quality of life in stroke survivors. However, the variability of study designs and outcomes measured across trials necessitates a pooled analysis to comprehensively assess the efficacy of these interventions. This protocol outlines the methodology for a pooled analysis that aims to synthesize evidence from randomized controlled trials (RCTs) evaluating nurse-led eHealth interventions for stroke patients.

Methods and analysis

This pooled analysis will be conducted according to the PRISMA guidelines. We will include RCTs that evaluate nurse-led eHealth programs and report on functional outcomes or quality of life in stroke patients. Comprehensive searches of electronic databases including Pubmed, EMBASE, the Cochrane Library, CINAHL, and PsycINFO will be conducted with a predefined search strategy. Study selection will involve screening titles and abstracts, followed by full-text review using explicit inclusion and exclusion criteria. Data extraction will be undertaken independently by two reviewers. The risk of bias will be assessed through the Cochrane Risk of Bias tool. Additionally, the quality of evidence for each outcome will be evaluated using the GRADE approach. Meta-analyses will be performed using random-effects models, and heterogeneity will be quantified using the I2 statistic. Subgroup and sensitivity analyses will explore potential sources of heterogeneity.

Discussion and conclusions

This pooled analysis is poised to provide a nuanced understanding of the effectiveness of nurse-led eHealth programs in stroke rehabilitation, leveraging a thorough methodological framework and GRADE tool to ensure robustness and reliability of evidence. The investigation anticipates diverse improvements in patient outcomes, underscoring the potential of personalized, accessible eHealth interventions to enhance patient engagement and treatment adherence. Despite the challenges posed by the heterogeneity of interventions and rapid technological advancements, the findings stand to influence clinical pathways by integrating eHealth into standard care, if substantiated by the evidence. Our study's depth and methodological rigor possess the potential to initiate changes in healthcare policy, advocating for the adoption of eHealth and subsequent investigations into its cost-efficiency. Ultimately, we aim to contribute rich, evidence-based insights into the burgeoning field of digital health, offering a foundational assessment of its applications in stroke care. Our data is expected to have a lasting impact, not only guiding immediate clinical decisions but also shaping the trajectory of future healthcare strategies in stroke recovery.

Systematic review registration

Identifier (CRD42024520100: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=520100).

Co-design workshops to develop evidence synthesis summary formats for use by clinical guideline development groups

BACKGROUND

Evidence synthesis is used by decision-makers in various ways, such as developing evidence-based recommendations for clinical guidelines. Clinical guideline development groups (GDGs) typically discuss evidence synthesis findings in a multidisciplinary group, including patients, healthcare providers, policymakers, etc. A recent mixed methods systematic review (MMSR) identified no gold standard format for optimally presenting evidence synthesis findings to these groups. However, it provided 94 recommendations to help produce more effective summary formats for general evidence syntheses (e.g., systematic reviews). To refine the MMSR recommendations to create more actionable guidance for summary producers, we aimed to explore these 94 recommendations with participants involved in evidence synthesis and guideline development.

METHODS

We conducted a descriptive qualitative study using online focus group workshops in February and March 2023. These groups used a participatory co-design approach with interactive voting activities to identify preferences for a summary format's essential content and style. We created a topic guide focused on recommendations from the MMSR with mixed methods support, ≥ 3 supporting studies, and those prioritized by an expert advisory group via a pragmatic prioritization exercise using the MoSCoW method (Must, Should, Could, and Will not haves). Eligible participants must be/have been involved in GDGs and/or evidence synthesis. Groups were recorded and transcribed. Two independent researchers analyzed transcripts using directed content analysis with 94 pre-defined codes from the MMSR.

RESULTS

Thirty individuals participated in six focus groups. We coded 79 of the 94 pre-defined codes. Participants suggested a "less is more" structured approach that minimizes methodological steps and statistical data, promoting accessibility to all audiences by judicious use of links to further information in the full report. They emphasized concise, consistently presented formats that highlight key messages, flag readers to indicators of trust in the producers (i.e., logos, websites, and conflict of interest statements), and highlight the certainty of evidence (without extenuating details).

CONCLUSIONS

This study identified guidance based on the preferences of guideline developers and evidence synthesis producers about the format of evidence synthesis summaries to support decision-making. The next steps involve developing and user-testing prototype formats through one-on-one semi-structured interviews to optimize evidence synthesis summaries and support decision-making.

The effectiveness of interventions to disseminate the results of non-commercial randomised clinical trials to healthcare professionals: a systematic review

BACKGROUND

It is unclear how to disseminate the results of randomised controlled trials effectively to health professionals and policymakers to improve treatment, care or prevention through changing policy and practice. This systematic review examined the effectiveness of different methods of dissemination of clinical research results to professional audiences.

METHODS

We systematically reviewed the published and grey literature from 2000 to 2022 for studies assessing different approaches for disseminating clinical study results to professional audiences (health professionals, policymakers and guideline developers). Two reviewers assessed potentially relevant full texts for inclusion. We grouped studies by intervention type, synthesising findings using effect direction plots. Outcomes were grouped into out-takes (e.g. awareness, knowledge, understanding), outcomes (e.g. attitude changes) and impact (changes in policy/practice). The quality of evidence was assessed using GRADE.

RESULTS

Our search identified 13,264 unique records, of which 416 full texts were assessed for eligibility. Of 60 studies that were identified as eligible for inclusion, 20 evaluated the effectiveness of interventions to disseminate clinical research results (13 RCTs, 2 observational studies, 3 pre- and post-intervention surveys and 2 cross-sectional surveys). Studies were grouped by intervention: 7 studies that involved face-to-face meetings between the target audience and trained educators were classified as 'outreach interventions'; 5 studies that provided a summary format for systematic review findings (e.g. summary of findings tables) were grouped together. There was high certainty evidence of a small beneficial impact of outreach interventions on health and moderate certainty evidence of impact on practice (mostly prescribing). There was no evidence of impact on policy and very low certainty around benefits on outcomes and out-takes. We found no consistent benefits of summary formats for systematic review results on outcomes or out-takes (moderate quality evidence). Other interventions with less evidence are reported in the Additional Materials.

CONCLUSIONS

Outreach interventions to disseminate clinical research results can lead to changes in practice and improvements in health. However, these interventions can be resource-intensive. Investment is vital to identify and implement effective and cost-effective ways to disseminate results, so that the potential benefits of trials to patients can be realised.

TRIAL REGISTRATION

International Prospective Register of Systematic Reviews (PROSPERO), CRD42019137364.

Strategies for communicating scientific evidence on healthcare to managers and the population: a scoping review

BACKGROUND

Health evidence needs to be communicated and disseminated in a manner that is clearly understood by decision-makers. As an inherent component of health knowledge translation, communicating results of scientific studies, effects of interventions and health risk estimates, in addition to understanding key concepts of clinical epidemiology and interpreting evidence, represent a set of essential instruments to reduce the gap between science and practice. The advancement of digital and social media has reshaped the concept of health communication, introducing new, direct and powerful communication platforms and gateways between researchers and the public. The objective of this scoping review was to identify strategies for communicating scientific evidence in healthcare to managers and/or population.

METHODS

We searched Cochrane Library, Embase®, MEDLINE® and other six electronic databases, in addition to grey literature, relevant websites from related organizations for studies, documents or reports published from 2000, addressing any strategy for communicating scientific evidence on healthcare to managers and/or population.

RESULTS

Our search identified 24 598 unique records, of which 80 met the inclusion criteria and addressed 78 strategies. Most strategies focused on risk and benefit communication in health, were presented by textual format and had been implemented and somehow evaluated. Among the strategies evaluated and appearing to yield some benefit are (i) risk/benefit communication: natural frequencies instead of percentages, absolute risk instead relative risk and number needed to treat, numerical instead nominal communication, mortality instead survival; negative or loss content appear to be more effective than positive or gain content; (ii) evidence synthesis: plain languages summaries to communicate the results of Cochrane reviews to the community were perceived as more reliable, easier to find and understand, and better to support decisions than the original summaries; (iii) teaching/learning: the Informed Health Choices resources seem to be effective for improving critical thinking skills.

CONCLUSION

Our findings contribute to both the knowledge translation process by identifying communication strategies with potential for immediate implementation and to future research by recognizing the need to evaluate the clinical and social impact of other strategies to support evidence-informed policies. Trial registration protocol is prospectively available in MedArxiv (doi.org/10.1101/2021.11.04.21265922).

Decision-makers' experiences with rapid evidence summaries to support real-time evidence informed decision-making in crises: a mixed methods study

BACKGROUND

There is a clear need for research evidence to drive policymaking and emergency responses so that lives are saved and resources are not wasted. The need for evidence support for health and humanitarian crisis is even more pertinent because of the time and practical constraints that decision-makers in these settings face. To improve the use of research evidence in policy and practice, it is important to provide evidence resources tailored to the target audience. This study aims to gain real-world insights from decision-makers about how they use evidence summaries to inform real-time decision-making in crisis-settings, and to use our findings to improve the format of evidence summaries.

METHODS

This study used an explanatory sequential mixed method study design. First, we used a survey to identify the views and experiences of those who were directly involved in crisis response in different contexts, and who may or may not have used evidence summaries. Second, we used the insights generated from the survey to help inform qualitative interviews with decision-makers in crisis-settings to derive an in-depth understanding of how they use evidence summaries and their desired format for evidence summaries.

RESULTS

We interviewed 26 decision-makers working in health and humanitarian emergencies. The study identified challenges decision-makers face when trying to find and use research evidence in crises, including insufficient time and increased burden of responsibilities during crises, limited access to reliable internet connection, large volume of data not translated into user friendly summaries, and little information available on preparedness and response measures. Decision-makers preferred the following components in evidence summaries: title, target audience, presentation of key findings in an actionable checklist or infographic format, implementation considerations, assessment of the quality of evidence presented, citation and hyperlink to the full review, funding sources, language of full review, and other sources of information on the topic. Our study developed an evidence summary template with accompanying training material to inform real-time decision-making in crisis-settings.

CONCLUSIONS

Our study provided a deeper understanding of the preferences of decision-makers working in health and humanitarian emergencies about the format of evidence summaries to enable real-time evidence informed decision-making.

Dissemination of Evidence by Cochrane Public Health Europe in German-Speaking Countries: An Online Stakeholder Survey

Objectives: To investigate the reach and impact of "Infomails", email summaries of Cochrane reviews in German, regularly disseminated by Cochrane Public Health Europe (CPHE) to stakeholders in Austria, Germany and Switzerland. Methods: We analysed email campaign reports from 15 Infomails delivered until November 2020. Furthermore, we invited all previous Infomail recipients to participate in an online survey on the impact and perceptions regarding our Infomails in November 2020. We analysed the results using descriptive statistics. Results: The Infomails' open rate ranged from 10.9% to 39.3% (median 26.0%), and the median click rate on the embedded links was 28.0% (range 8.6-53.8%), highest for nutrition and prevention topics. Out of 1259 recipients, 267 (21.2%) completed our survey. Infomails were most used in discussions, writing reports or statements, for policy or strategy development or programme or guideline development. Persons who remembered receiving Infomails rated them as useful, comprehensible or informative. Conclusion: Infomails summarising recent Cochrane reviews were considered useful for the daily work of public health stakeholders in German-speaking countries. Regular targeted messaging may increase the perceived usefulness.

Using GRADE Evidence to Decision frameworks to support the process of health policy-making: an example application regarding taxation of sugar-sweetened beverages

BACKGROUND

Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks are well-known tools that enable guideline panels to structure the process of developing recommendations and making decisions in healthcare and public health. To date, they have not regularly been used for health policy-making. This article aims to illustrate the application of the GRADE EtD frameworks in the process of nutrition-related policy-making for a European country.

METHODS

Based on methodological guidance by the GRADE Working Group and the findings of our recently published scoping review, we illustrate the process of moving from evidence to recommendations, by applying the EtD frameworks to a fictitious example. Sugar-sweetened beverage (SSB) taxation based on energy density was chosen as an example application.

RESULTS

A fictitious guideline panel was convened by a national nutrition association to develop a population-level recommendation on SSB taxation aiming to reduce the burden of overweight and obesity. Exemplary evidence was summarized for each EtD criterion and conclusions were drawn based on all judgements made in relation to each criterion. As a result of the high priority to reduce the burden of obesity and because of the moderate desirable effects on health outcomes, but considering scarce or varying research evidence for other EtD criteria, the panel made a conditional recommendation for SSB taxation. Decision-makers may opt for conducting a pilot study prior to implementing the policy on a national level.

CONCLUSIONS

GRADE EtD frameworks can be used by guideline panels to make the process of developing recommendations in the field of health policy more systematic, transparent and comprehensible.

The effectiveness and acceptability of evidence synthesis summary formats for clinical guideline development groups: a mixed-methods systematic review

INTRODUCTION

Clinical guideline development often involves a rigorous synthesis of evidence involving multidisciplinary stakeholders with different priorities and knowledge of evidence synthesis; this makes communicating findings complex. Summary formats are typically used to communicate the results of evidence syntheses; however, there is little consensus on which formats are most effective and acceptable for different stakeholders.

METHODS

This mixed-methods systematic review (MMSR) aimed to evaluate the effectiveness and acceptability (e.g. preferences and attitudes and preferences towards) of evidence synthesis summary formats for GDG members. We followed the PRISMA 2020 guideline and Joanna Briggs Institute Manual for Evidence Synthesis for MMSRs. We searched six databases (inception to April 20, 2021) for randomised controlled trials (RCTs), RCTs with a qualitative component, and qualitative studies. Screening, data extraction, and quality appraisal were performed in duplicate. Qualitative findings were synthesised using meta-aggregation, and quantitative findings are described narratively.

RESULTS

We identified 17,240 citations and screened 54 full-text articles, resulting in 22 eligible articles (20 unique studies): 4 articles reported the results of 5 RCTs, one of which also had a qualitative component. The other 18 articles discussed the results of 16 qualitative studies. Therefore, we had 5 trials and 17 qualitative studies to extract data from. Studies were geographically heterogeneous and included a variety of stakeholders and summary formats. All 5 RCTs assessed knowledge or understanding with 3 reporting improvement with newer formats. The qualitative analysis identified 6 categories of recommendations: 'presenting information', 'tailoring information' for end users, 'trust in producers and summary', 'knowledge required' to understand findings, 'quality of evidence', and properly 'contextualising information'. Across these categories, the synthesis resulted in 126 recommendations for practice. Nine recommendations were supported by both quantitative and qualitative evidence and 116 by only qualitative. A majority focused on how to present information (n = 64) and tailor content for different end users (n = 24).

CONCLUSIONS

This MMSR provides guidance on how to improve evidence summary structure and layout. This can be used by synthesis producers to better communicate to GDGs. Study findings will inform the co-creation of evidence summary format prototypes based on GDG member's needs. Trial registration The protocol for this project was previously published, and the project was preregistered on Open Science Framework (Clyne and Sharp, Evidence synthesis and translation of findings for national clinical guideline development: addressing the needs and preferences of guideline development groups, 2021; Sharp and Clyne, Evidence synthesis summary formats for decision-makers and Clinical Guideline Development Groups: A mixed-methods systematic review protocol, 2021).

Partisan differences in the effects of economic evidence and local data on legislator engagement with dissemination materials about behavioral health: a dissemination trial

BACKGROUND

State legislators make policy decisions that influence children's exposure to adverse childhood experiences (ACEs), such as child maltreatment, and their effects on behavioral health. Effective dissemination of scientific research can increase the likelihood that legislators' decisions are aligned with evidence to prevent ACEs and their consequences, and effective dissemination requires legislators to engage with dissemination materials. Informed by the elaboration likelihood model of persuasive communication and Brownson's Model of Dissemination Research, we tested the hypothesis that inclusion of economic evidence and local data would increase legislator engagement with dissemination materials about evidence-supported policies related to ACEs and behavioral health.

METHODS

A three-arm randomized dissemination trial was conducted. A university researcher e-mailed dissemination materials which contained evidence about ACEs and behavioral health problems to state legislators (two e-mails sent 2 weeks apart, 12,662 e-mails delivered to 6509 legislators). The e-mail subject lines, text, and policy brief content were manipulated across the study arms. The intervention condition received state-tailored data about rates of ACEs and state-tailored economic evidence about the costs of ACEs for public systems, the enhanced control condition received state-tailored data and not economic evidence, and the control condition received national data and not economic evidence. Outcomes were rates of e-mail views, policy brief link clicks, requests for researcher consultation, and mentions of child maltreatment terms in legislators' social media posts.

RESULTS

For the first e-mail, the e-mail view rate was 42.6% higher in the intervention than in the enhanced control condition (22.8% vs. 14.8%) and 20.8% higher than in the control condition (22.8% vs. 18.5%) (both p < .0001). Similar results were observed for the second e-mail. These differences remained significant after adjustment for demographic differences across study conditions in individual-level models, but not multilevel models. There was a significant interaction between the experimental condition and political party (p < .0001) in which the intervention increased e-mail view rates among Democrats but not Republicans. The intervention had no effect on policy brief link clicks or requests for consultation and a mixed effect on social media posts.

CONCLUSIONS

Inclusion of state-tailored economic evidence in dissemination materials can increase engagement with research evidence among Democrat, but not Republican, legislators. Dissemination strategies tailored for legislators' political party affiliation may be needed.

Evidence synthesis summary formats for clinical guideline development group members: a mixed-methods systematic review protocol

Introduction: Evidence syntheses, often in the form of systematic reviews, are essential for clinical guideline development and informing changes to health policies. However, clinical guideline development groups (CGDG) are multidisciplinary, and participants such as policymakers, healthcare professionals and patient representatives can face obstacles when trying to understand and use evidence synthesis findings. Summary formats to communicate the results of evidence syntheses have become increasingly common, but it is currently unclear which format is most effective for different stakeholders. This mixed-methods systematic review (MMSR) evaluates the effectiveness and acceptability of different evidence synthesis summary formats for CGDG members. Methods: This protocol follows guidance from the Joanna Briggs Institute on MMSRs and is reported according to the Preferred Reporting Items for Systematic Reviews (PRISMA)-P guideline. A comprehensive search of six databases will be performed with no language restrictions. Primary outcomes are those relating to the effectiveness and preferences for and attitudes towards the different summary formats. We will include qualitative research and randomised controlled trials. Two reviewers will perform title, abstract, and full-text screening. Independent double-extraction of study characteristics and critical appraisal items will be undertaken using a standardised form. We will use a convergent segregated approach to analyse quantitative and qualitative data separately; results will then be integrated. Discussion: The results of this systematic review will provide an overview of the effectiveness and acceptability of different summary formats for evidence synthesis findings. These findings can be helpful for those in or communicating to guideline development groups. The results can also inform the development and pilot-testing of summary formats for evidence summaries.

Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews

Background

The use of research evidence as an input for health decision-making is a need for most health systems. There are a number of approaches for promoting evidence use at different levels of the health system, but knowledge of their effectiveness is still scarce. The objective of this overview was to evaluate the effectiveness of knowledge communication and dissemination interventions, strategies or approaches targeting policy-makers and health managers.

Methods

This overview of systematic reviews used systematic review methods and was conducted according to a predefined and published protocol. A comprehensive electronic search of 13 databases and a manual search in four websites were conducted. Both published and unpublished reviews in English, Spanish or Portuguese were included. A narrative synthesis was undertaken, and effectiveness statements were developed, informed by the evidence identified.

Results

We included 27 systematic reviews. Three studies included only a communication strategy, while eight only included dissemination strategies, and the remaining 16 included both. None of the selected reviews provided “sufficient evidence” for any of the strategies, while four provided some evidence for three communication and four dissemination strategies. Regarding communication strategies, the use of tailored and targeted messages seemed to successfully lead to changes in the decision-making practices of the target audience. Regarding dissemination strategies, interventions that aimed at improving only the reach of evidence did not have an impact on its use in decisions, while interventions aimed at enhancing users’ ability to use and apply evidence had a positive effect on decision-making processes. Multifaceted dissemination strategies also demonstrated the potential for changing knowledge about evidence but not its implementation in decision-making.

Conclusions

There is limited evidence regarding the effectiveness of interventions targeting health managers and policy-makers, as well as the mechanisms required for achieving impact. More studies are needed that are informed by theoretical frameworks or specific tools and using robust methods, standardized outcome measures and clear descriptions of the interventions. We found that passive communication increased access to evidence but had no effect on uptake. Some evidence indicated that the use of targeted messages, knowledge-brokering and user training was effective in promoting evidence use by managers and policy-makers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00780-4.

Methods to support evidence-informed decision-making in the midst of COVID-19: creation and evolution of a rapid review service from the National Collaborating Centre for Methods and Tools

BACKGROUND

The COVID-19 public health crisis has produced an immense and quickly evolving body of evidence. This research speed and volume, along with variability in quality, could overwhelm public health decision-makers striving to make timely decisions based on the best available evidence. In response to this challenge, the National Collaborating Centre for Methods and Tools developed a Rapid Evidence Service, building on internationally accepted rapid review methodologies, to address priority COVID-19 public health questions.

RESULTS

Each week, the Rapid Evidence Service team receives requests from public health decision-makers, prioritizes questions received, and frames the prioritized topics into searchable questions. We develop and conduct a comprehensive search strategy and critically appraise all relevant evidence using validated tools. We synthesize the findings into a final report that includes key messages, with a rating of the certainty of the evidence using GRADE, as well as an overview of evidence and remaining knowledge gaps. Rapid reviews are typically completed and disseminated within two weeks. From May 2020 to July 21, 2021, we have answered more than 31 distinct questions and completed 32 updates as new evidence emerged. Reviews receive an average of 213 downloads per week, with some reaching over 7700. To date reviews have been accessed and cited around the world, and a more fulsome evaluation of impact on decision-making is planned.

CONCLUSIONS

The development, evolution, and lessons learned from our process, presented here, provides a real-world example of how review-level evidence can be made available - rapidly and rigorously, and in response to decision-makers' needs - during an unprecedented public health crisis.

Evidence synthesis summary formats for clinical guideline development group members: a mixed-methods systematic review protocol

Introduction: Evidence syntheses, often in the form of systematic reviews, are essential for clinical guideline development and informing changes to health policies. However, clinical guideline development groups (CGDG) are multidisciplinary, and participants such as policymakers, healthcare professionals and patient representatives can face obstacles when trying to understand and use evidence synthesis findings. Summary formats to communicate the results of evidence syntheses have become increasingly common, but it is currently unclear which format is most effective for different stakeholders. This mixed-methods systematic review (MMSR) evaluates the effectiveness and acceptability of different evidence synthesis summary formats for CGDG members. Methods: This protocol follows guidance from the Joanna Briggs Institute on MMSRs and is reported according to the Preferred Reporting Items for Systematic Reviews (PRISMA)-P guideline. A comprehensive search of six databases will be performed with no language restrictions. Primary outcomes are those relating to the effectiveness and preferences for and attitudes towards the different summary formats. We will include qualitative research and randomised controlled trials. Two reviewers will perform title, abstract, and full-text screening. Independent double-extraction of study characteristics and critical appraisal items will be undertaken using a standardised form. We will use a convergent segregated approach to analyse quantitative and qualitative data separately; results will then be integrated. Discussion: The results of this systematic review will provide an overview of the effectiveness and acceptability of different summary formats for evidence synthesis findings. These findings can be helpful for those in or communicating to guideline development groups. The results can also inform the development and pilot-testing of summary formats for evidence summaries.

Policy options to increase motivation for improving evidence-informed health policy-making in Iran

BACKGROUND

Current incentive programmes are not sufficient to motivate researchers and policy-makers to use research evidence in policy-making. We conducted a mixed-methods design to identify context-based policy options for strengthening motivations among health researchers and policy-makers to support evidence-informed health policy-making (EIHP) in Iran.

METHODS

This study was conducted in 2019 in two phases. In the first phase, we conducted a scoping review to extract interventions implemented or proposed to strengthen motivations to support EIHP. Additionally, we employed a comparative case study design for reviewing the performance evaluation (PE) processes in Iran and other selected countries to determine the current individual and organizational incentives to encourage EIHP. In the second phase, we developed two policy briefs and then convened two policy dialogues, with 12 and 8 key informants, respectively, where the briefs were discussed. Data were analysed using manifest content analysis in order to propose contextualized policy options.

RESULTS

The policy options identified to motivate health researchers and policy-makers to support EIHP in Iran were: revising the criteria of academic PE; designing appropriate incentive programmes for nonacademic researchers; developing an indicator for the evaluation of research impact on policy-making or health outcomes; revising the current policies of scientific journals; revising existing funding mechanisms; presenting the knowledge translation plan when submitting a research proposal, as a mandatory condition; encouraging and supporting mechanisms for increasing interactions between policy-makers and researchers; and revising some administrative processes (e.g. managers and staff PEs; selection, appointment, and changing managers and reward mechanisms).

CONCLUSIONS

The current individual or organizational incentives are mainly focused on publications, rather than encouraging researchers and policy-makers to support EIHP. Relying more on incentives that consider the other impacts of research (e.g. impacts on health system and policy, or health outcomes) is recommended. These incentives may encourage individuals and organizations to be more involved in conducting research evidence, resulting in promoting EIHP.

TRIAL REGISTRATION

NA.

Communication vs evidence: What hinders the outreach of science during an infodemic? A narrative review

Background

The COVID-19 pandemic is placing significant pressure on national and international health organizations and the measures taken to combat it are having many impacts beyond health. At the same time, misleading communication practices and what has been called an “infodemic” by the World Health Organization have been hampering the uptake of coronavirus-related scientific information. Moreover, public awareness about the dangers of the infodemic remains poor, and misinformation may lead to hazardous behaviours. We therefore analysed factors potentially undermining communication of scientific evidence and proposed strategies to counteract this phenomenon.

Methods

We sought official academic and institutional publications of any type, published in English and analyzed their approaches to communication used during the pandemic.

Results

The factors that might undermine appropriate communication include but are not limited to (a) the exponential increase of COVID-19-related publications, often including biases in the peer-review and editorial process; (b) the role of traditional media; (c) politicization of the virus; and (d) the impact of social media. We argue that evidence synthesis and knowledge translation are useful tools to communicate accurate scientific evidence to decision-makers.

Conclusion

Clear and concise messages in this form can help decision-makers to interpret data correctly, take consequent actions, and avoid being compromised by low-quality or even misleading information.

[Insights for healthy settings: a database to support the translation of findings from systematic reviews into practice]

The database "Insights for healthy settings" was developed by the GKV-Bündnis für Gesundheit, a joint initiative of all health insurance funds for developing and implementing setting-based health promotion and prevention measures. It aims to support health insurance funds and other practice partners in planning and implementing evidence-based health promotion and prevention measures in different settings. The database complements existing intervention databases by providing insights based on findings from systematic reviews. This article aims to present the database, describe possible applications, and discuss options for further development.The database contains findings on the effectiveness and strategies for implementation of setting-based health promotion and prevention measures. In addition to structured summaries of systematic reviews, it includes summaries of scoping reviews and rapid reviews. Different search options (e.g., free text search, subject headings, and search filters) can be used to find database entries (currently n = 13). Quality-assured database entries are created by means of a standardized form and based on the four-eyes principle. To allow for easy access, key findings are presented briefly, and technical terms are clarified. Experts from health insurance funds are continuously involved in the development process of the database.As part of the database's further development, new reviews of the GKV-Bündnis für Gesundheit as well as from other sources will be added. Additionally, quality assessments of included reviews will be depicted and supporting activities developed to further encourage translation of scientific knowledge into practice.

Use of a quantitative data report in a hypothetical decision scenario for health policymaking: a computer-assisted laboratory study

BACKGROUND

Quantitative data reports are widely produced to inform health policy decisions. Policymakers are expected to critically assess provided information in order to incorporate the best available evidence into the decision-making process. Many other factors are known to influence this process, but little is known about how quantitative data reports are actually read. We explored the reading behavior of (future) health policy decision-makers, using innovative methods.

METHODS

We conducted a computer-assisted laboratory study, involving starting and advanced students in medicine and health sciences, and professionals as participants. They read a quantitative data report to inform a decision on the use of resources for long-term care in dementia in a hypothetical decision scenario. Data were collected through eye-tracking, questionnaires, and a brief interview. Eye-tracking data were used to generate 'heatmaps' and five measures of reading behavior. The questionnaires provided participants' perceptions of understandability and helpfulness as well as individual characteristics. Interviews documented reasons for attention to specific report sections. The quantitative analysis was largely descriptive, complemented by Pearson correlations. Interviews were analyzed by qualitative content analysis.

RESULTS

In total, 46 individuals participated [students (85%), professionals (15%)]. Eye-tracking observations showed that the participants spent equal time and attention for most parts of the presented report, but were less focused when reading the methods section. The qualitative content analysis identified 29 reasons for attention to a report section related to four topics. Eye-tracking measures were largely unrelated to participants' perceptions of understandability and helpfulness of the report.

CONCLUSIONS

Eye-tracking data added information on reading behaviors that were not captured by questionnaires or interviews with health decision-makers.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Assessing how information is packaged in rapid reviews for policy-makers and other stakeholders: a cross-sectional study

Background

Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice.

Methods

We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere.

Results

Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read.

Conclusions

Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored.

Protocol availability

The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Assessing the format and content of journal published and non-journal published rapid review reports: A comparative study

BACKGROUND

As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured.

OBJECTIVES

To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs.

METHODS

JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented.

RESULTS

We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups.

CONCLUSIONS

This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice.

STUDY REGISTRATION

The full protocol is available at: https://osf.io/29xvk/.

Sound Therapy for Tinnitus: Options for Audiologists

Purpose

Sound therapy to reduce the emotional and functional effects of tinnitus has been used by audiologists since the 1970s when Jack Vernon introduced the masking method to provide a sense of relief from tinnitus distress. Our group renamed masking sound as “soothing” sound and distinguished it from “interesting” and “background” sound, each of which has a different purpose for tinnitus sound therapy. Other methods of sound therapy have the potential to reduce the “sensation” of tinnitus, including notched noise, matched noise, desynchronization, and residual inhibition. The purpose of this article is to provide an overview of the different sound therapy approaches to serve as a resource for audiologists who often provide sound therapy to their patients with tinnitus.

Conclusion

Although, according to systematic reviews, sound therapy does not have strong evidence for treatment of tinnitus, it is nonetheless well evidenced both through abundant research and clinical utilization mostly by audiologists. It is unknown if any one form of sound therapy is superior to any other.

Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies

BACKGROUND

Nutrition and physical activity policies have the potential to influence lifestyle patterns and reduce the burden of non-communicable diseases. In the world of health-related guidelines, GRADE (Grading of Recommendations, Assessment, Development and Evaluation) is the most widely used approach for assessing the certainty of evidence and determining the strength of recommendations. Thus, it is relevant to explore its usefulness also in the process of nutrition and physical activity policymaking and evaluation. The purpose of this scoping review was (i) to generate an exemplary overview of documents using the GRADE approach in the process of nutrition and physical activity policymaking and evaluation, (ii) to find out how the GRADE approach has been applied, and (iii) to explore which facilitators of and barriers to the use of GRADE have been described on the basis of the identified documents. The overarching aim of this work is to work towards improving the process of evidence-informed policymaking in the areas of dietary behavior, physical activity, and sedentary behavior.

METHODS

A scoping review was conducted according to current reporting standards. MEDLINE via Ovid, the Cochrane Library, and Web of Science were systematically searched up until 4 July 2019. Documents describing a body of evidence which was assessed for the development or evaluation of a policy, including documents labeled as "guidelines," or systematic reviews used to inform policymaking were included.

RESULTS

Thirty-six documents were included. Overall, 313 GRADE certainty of evidence ratings were identified in systematic reviews and guidelines; the strength of recommendations/policies was assessed in four documents, and six documents mentioned facilitators or barriers for the use of GRADE. The major reported barrier was the initial low starting level of a body of evidence from non-randomized studies when assessing the certainty of evidence.

CONCLUSION

This scoping review found that the GRADE approach has been used for policy evaluations, in the evaluation of the effectiveness of policy-relevant interventions (policymaking), as well as in the development of guidelines intended to guide policymaking. Several areas for future research were identified to explore the use of GRADE in health policymaking and evaluation.

State mental health agency officials' preferences for and sources of behavioral health research

State mental health agencies (SMHAs) are integral to public behavioral health service systems. As such, senior-level officials within SMHAs are important targets for advocacy and dissemination of behavioral health research findings. Evidence-informed decision making in SMHAs can potentially be enhanced by developing summaries of behavioral health research (e.g., policy briefs) that reflect SMHA officials' information preferences, but knowledge about these preferences is lacking. An exploratory study was conducted with the aims of characterizing senior-level SMHA officials' preferences for behavioral health research and describing where they turn for this research when making policy decisions. A cross-sectional, web-based survey of senior-level SMHA officials (1 per state) was conducted in March-May 2017 (n = 43, response rate = 84%). The features of behavioral health research that SMHA officials identified as "very important" most frequently were research being relevant to state residents (93.0%), providing data on cost-effectiveness (86.0%) and budget impact (81.4%), and being presented concisely (81.0%). The primary sources that SMHA officials turned to for behavioral research when making policy decisions were professional organizations (79.1%), SMHA agency staff (60.5%), and university researchers (55.8%). Compared with state legislators' responses to the same survey questions, results suggest that senior-level SMHA officials and legislators have similar preferences for behavioral health research but turn to different sources for this research. Advocates and researchers who seek to promote evidence-informed decision making in SMHAs should consider developing policy briefs that are concise, provide state-level prevalence data about behavioral conditions, and contain economic evaluation data, and they should disseminate these materials to multiple sources. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Evidence summary resources may influence clinical decision making: A case-based scenario evaluation of an evidence summary tool

PURPOSE

Evidence summary resources are popular with clinicians but it is unknown whether they can influence clinical decision making. We evaluated whether an extremely condensed and explicit evidence summary tool could influence clinical decision making.

MATERIALS AND METHODS

An evidence summary tool was developed using a formal mapping exercise and graphic design principles. An invitation to participate was sent to subscribers of a critical care e-mail discussion list. Participants received a study package (evidence summary tool précising prone positioning in severe ARDS; case-based scenario describing a patient with severe ARDS plus evaluation questionnaire). Influence on clinical decisions was captured regarding six competing interventions, with Belief in benefit measured before and after reading the summary tool.

RESULTS

Among 93 participants, 87% were male with a mean age of 49.6(SD9.3) years. Mean ICU experience was 20.0(SD9.9) years. The evidence summary tool significantly influenced clinical decision making: belief in benefit of prone positioning increased (P < .001), belief in benefit of higher PEEP decreased (P = .002) and belief in benefit in ECMO decreased (P = .07).

CONCLUSIONS

Using a before-after evaluation, we demonstrated an extremely condensed and explicit information format can influence clinical decision making. Evidence summary tools may be a useful adjunct to support closing evidence-practice gaps.

Making Smarter Decisions Faster: Systems Engineering to Improve the Global Public Health Response to HIV

PURPOSE OF REVIEW

This review offers an operational definition of systems engineering (SE) as applied to public health, reviews applications of SE in the field of HIV, and identifies opportunities and challenges of broader application of SE in global health.

RECENT FINDINGS

SE involves the deliberate sequencing of three steps: diagnosing a problem, evaluating options using modeling or optimization, and providing actionable recommendations. SE includes diverse tools (from process improvement to mathematical modeling) applied to decisions at various levels (from local staffing decisions to planning national-level roll-out of new interventions). Contextual factors are crucial to effective decision-making, but there are gaps in understanding global decision-making processes. Integrating SE into pre-service training and translating SE tools to be more accessible could increase utilization of SE approaches in global health. SE is a promising, but under-recognized approach to improve public health response to HIV globally.

Examining the readiness of best evidence in medical education guides for integration into educational practice: A meta-synthesis

BACKGROUND

To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria-quality, accessibility and relevance-we risk translating weak evidence into practice and/or providing information that is not useful to educators.

METHODS

A librarian searched Web of Science for knowledge syntheses, specifically Best Evidence in Medical Education (BEME) Guides. This meta-synthesis focuses on BEME Guides because of their explicit goal to inform educational practice and policy. Two authors extracted data from all Guides, guided by the 25-item STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES).

RESULTS

Forty-two Guides published in Medical Teacher between 1999 and 2017 were analyzed. No Guide met all STORIES criteria, but all included structured summaries and most described their literature search (n = 39) and study inclusion/exclusion (n = 40) procedures. Eleven Guides reported the presence of theory and/or educational principles, and eight consulted with external subject matter experts. Accessibility to each Guide's full-text and supplemental materials was variable.

DISCUSSION

For a subset of HPE knowledge syntheses, BEME Guides, this meta-synthesis identifies factors that support readiness and indicates potential areas of improvement, such as consistent access to Guides and inclusion of external subject matter experts on the review team. This analysis is useful for understanding the current readiness of HPE knowledge syntheses and informing future reviews to evolve so they can catalyze translation of evidence into educational practice.

The Cochrane Rehabilitation eBook: a knowledge translation tool to transfer evidence to different rehabilitation audiences

Systematic reviews are a powerful method for summarizing and synthesizing evidence and create a methodological opportunity for preparing data integration tables to enable review-level synthesis of the evidence. One of the main tasks of Cochrane Rehabilitation is to make it possible to improve the application of evidence-based clinical practice by all rehabilitation professionals, and to facilitate policy-makers in decision-making according to the best evidence. In this context arises the need to diffuse the high-quality information of Cochrane Systematic Reviews (CSR), through synthetic and easy-to-use messages for the different rehabilitation audiences. Cochrane Rehabilitation has recently launched the production of an eBook including all the evidence coming from the CSR tagged by its Review Committee as of rehabilitation interest with the final aim to enhance the application of evidence in clinical practice, medical education, health system policies, and in the community. This will be a powerful tool for the dissemination of evidence-based practice, education and knowledge, as well as to support political decisions for both effective organization and resource allocation in rehabilitation field.

[Access to information for evidence-based decision-making from the perspective of Municipal Health Secretaries in the State of Paraná, Brazil, 2014]

The study aimed to identify information sources for decision-making and barriers to access in the use of scientific journals by Municipal Health Secretaries in the State of Paraná, Brazil. Study participants included Secretaries of Health from 181 municipalities in the year 2014. Interviews focused on the frequency of consultation of resources for obtaining evidence, including from scientific journals. 40.9% of the participants were 18 to 39 years of age (mean age was 43±10.2 years; median 42 years). More than two-thirds (69.1%) had held the office of health secretary for less than five years. The majority were males (56.4%), and only one in three had a graduate degree in public health. The main sources of information were "non-scientific periodicals" (76.2%), "print/online journals" (71.4%), and "Ministry of Health guidelines" (71.3%). The main barriers to use of scientific journals were "lack of time for reading" (72.9%), "periodical's high cost" (69.1%), and "difficulty in identifying the best periodicals" (63.5%). The main information sources were non-scientific periodicals and Ministry of Health guidelines. The use of scientific journals is hindered by lack of time and limited familiarity with scientific language. Access to scientific information should be facilitated by the use of more accessible sources and training for municipal administrators in the use of open-access periodicals. Such measures can foster knowledge of current evidence among municipal health secretaries.

Facilitating evidence uptake: development and user testing of a systematic review summary format to inform public health decision-making in German-speaking countries

BACKGROUND

Systematic reviews are an important source of evidence for public health decision-making, but length and technical jargon tend to hinder their use. In non-English speaking countries, inaccessibility of information in the native language often represents an additional barrier. In line with our vision to strengthen evidence-based public health in the German-speaking world, we developed a German language summary format for systematic reviews of public health interventions and undertook user-testing with public health decision-makers in Germany, Austria and Switzerland.

METHODS

We used several guiding principles and core elements identified from the literature to produce a prototype summary format and applied it to a Cochrane review on the impacts of changing portion and package sizes on selection and consumption of food, alcohol and tobacco. Following a pre-test in each of the three countries, we carried out 18 user tests with public health decision-makers in Germany, Austria and Switzerland using the 'think-aloud' method. We analysed participants' comments according to the facets credibility, usability, understandability, usefulness, desirability, findability, identification and accessibility. We also identified elements that hindered the facile and satisfying use of the summary format, and revised it based on participants' feedback.

RESULTS

The summary format was well-received; participants particularly appreciated receiving information in their own language. They generally found the summary format useful and a credible source of information, but also signalled several barriers to a positive user experience such as an information-dense structure and difficulties with understanding statistical terms. Many of the identified challenges were addressed through modifications of the summary format, in particular by allowing for flexible length, placing more emphasis on key messages and relevance for public health practice, expanding the interpretation aid for statistical findings, providing a glossary of technical terms, and only including graphical GRADE ratings. Some barriers to uptake, notably the participants' wish for actionable recommendations and contextual information, could not be addressed.

CONCLUSIONS

Participants welcomed the initiative, but user tests also revealed their problems with understanding and interpreting the findings summarised in our prototype format. The revised summary format will be used to communicate the results of Cochrane reviews of public health interventions.

Antipsychotic drug exposure and risk of fracture: a systematic review and meta-analysis of observational studies

To investigate the extent to which exposure to first-generation and second-generation antipsychotics (APs) is associated with an increased risk of fractures, with a particular focus on hip fractures, and to ascertain the risk associated with exposure to individual drugs. We included observational studies that reported data on fractures in individuals exposed to APs compared with unexposed individuals or individuals with previous exposure. We extracted information on study design, source of data, population characteristics, outcomes of interest, matching and confounding factors, and used a modified version of the Newcastle-Ottawa Scale to judge study risk of bias. We pooled adjusted estimates of relative effects to generate pooled odds ratios (ORs) and their 95% confidence interval (CI) using a random-effects model. We rated the quality of evidence using the GRADE approach. Of 36 observational studies, 29 proved to have a low risk of bias and seven were found to have a high risk of bias. The risk of hip fracture (OR: 1.57, 95% CI: 1.42-1.74, low quality of evidence) and of any fracture (OR: 1.17, 95% CI: 1.04-1.31, very low quality of evidence) increased with exposure to APs, with similar increases in risk in the first generation and second generation. The risk was similar among different diagnostic categories. The few studies that provided data were insufficient to allow inferences on individual drugs. AP exposure in unselected populations was associated with a 57% increase in the risk of hip fractures and a 17% increase in the risk of any fractures. Between-study heterogeneity limits the confidence in this estimate.

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study

BACKGROUND

Systematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use.

METHODS

A three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted with HCMs and PMs in four Canadian provinces to determine perceptions of a traditional systematic review format. In phase 2, systematic review format prototypes were created by HCMs and PMs via Conceptboard©. In phase 3, prototypes underwent usability testing by HCMs and PMs.

RESULTS

Two hundred two participants (80 HCMs, 122 PMs) completed the phase 1 survey. Respondents reported that inadequate format (Mdn = 4; IQR = 4; range = 1-7) and content (Mdn = 4; IQR = 3; range = 1-7) influenced their use of systematic reviews. Most respondents (76%; n = 136/180) reported they would be more likely to use systematic reviews if the format was modified. Findings from 11 interviews (5 HCMs, 6 PMs) revealed that participants preferred systematic reviews of effects that were easy to access and read and provided more information on intervention effectiveness and less information on review methodology. The mean System Usability Scale (SUS) score was 55.7 (standard deviation [SD] 17.2) for the traditional format; a SUS score < 68 is below average usability. In phase 2, 14 HCMs and 20 PMs co-created prototypes, one for HCMs and one for PMs. HCMs preferred a traditional information order (i.e., methods, study flow diagram, forest plots) whereas PMs preferred an alternative order (i.e., background and key messages on one page; methods and limitations on another). In phase 3, the prototypes underwent usability testing with 5 HCMs and 7 PMs, 11 out of 12 participants co-created the prototypes (mean SUS score 86 [SD 9.3]).

CONCLUSIONS

HCMs and PMs co-created prototypes for systematic review of effects formats based on their needs. The prototypes will be compared to a traditional format in a randomized trial.

The GRADE Evidence to Decision (EtD) framework for health system and public health decisions

OBJECTIVE

To describe a framework for people making and using evidence-informed health system and public health recommendations and decisions.

BACKGROUND

We developed the GRADE Evidence to Decision (EtD) framework for health system and public health decisions as part of the DECIDE project, in which we simultaneously developed frameworks for these and other types of healthcare decisions, including clinical recommendations, coverage decisions and decisions about diagnostic tests.

DEVELOPING THE FRAMEWORK

Building on GRADE EtD tables, we used an iterative approach, including brainstorming, consultation of the literature and with stakeholders, and an international survey of policy-makers. We applied the framework to diverse examples, conducted workshops and user testing with health system and public health guideline developers and policy-makers, and observed and tested its use in real-life guideline panels.

FINDINGS

All the GRADE EtD frameworks share the same basic structure, including sections for formulating the question, making an assessment and drawing conclusions. Criteria listed in the assessment section of the health system and public health framework cover the important factors for making these types of decisions; in addition to the effects and economic impact of an option, the priority of the problem, the impact of the option on equity, and its acceptability and feasibility are important considerations that can inform both whether and how to implement an option. Because health system and public health interventions are often complex, detailed implementation considerations should be made when making a decision. The certainty of the evidence is often low or very low, but decision-makers must still act. Monitoring and evaluation are therefore often important considerations for these types of decisions. We illustrate the different components of the EtD framework for health system and public health decisions by presenting their application in a framework adapted from a real-life guideline.

DISCUSSION

This framework provides a structured and transparent approach to support policy-making informed by the best available research evidence, while making the basis for decisions accessible to those whom they will affect. The health system and public health EtD framework can also be used to facilitate dissemination of recommendations and enable decision-makers to adopt, and adapt, recommendations or decisions.

Setting priorities for knowledge translation of Cochrane reviews for health equity: Evidence for Equity

BACKGROUND

A focus on equity in health can be seen in many global development goals and reports, research and international declarations. With the development of a relevant framework and methods, the Campbell and Cochrane Equity Methods Group has encouraged the application of an 'equity lens' to systematic reviews, and many organizations publish reviews intended to address health equity. The purpose of the Evidence for Equity (E4E) project was to conduct a priority-setting exercise and apply an equity lens by developing a knowledge translation product comprising summaries of systematic reviews from the Cochrane Library. E4E translates evidence from systematic reviews into 'friendly front end' summaries for policy makers.

METHODS

The following topic areas with high burdens of disease globally, were selected for the pilot: diabetes/obesity, HIV/AIDS, malaria, nutrition, and mental health/depression. For each topic area, a "stakeholder panel" was assembled that included policymakers and researchers. A systematic search of Cochrane reviews was conducted for each area to identify equity-relevant interventions with a meaningful impact. Panel chairs developed a rating sheet which was used by all panels to rank the importance of these interventions by: 1) Ease of Implementation; 2) Health System Requirements; 3)Universality/Generalizability/Share of Burden; and 4) Impact on Inequities/Effect on equity. The ratings of panel members were averaged for each intervention and criterion, and interventions were ordered according to the average overall ratings.

RESULTS

Stakeholder panels identified the top 10 interventions from their respective topic areas. The evidence on these interventions is being summarized with an equity focus and the results posted online, at http://methods.cochrane.org/equity/e4e-series .

CONCLUSIONS

This method provides an explicit approach to setting priorities by systematic review groups and funders for providing decision makers with evidence for the most important equity-relevant interventions.