Maximizing the impact of systematic reviews in health care decision making: a systematic scoping review of knowledge-translation resources

Unlocking data: Decision-maker perspectives on cross-sectoral data sharing and linkage as part of a whole-systems approach to public health policy and practice

Background

Secondary data from different policy sectors can provide unique insights into the social, environmental, economic and political determinants of health. This is especially pertinent in the context of whole-systems approaches to healthy public policy, which typically combine cross-sectoral collaboration with the application of theoretical insights from systems science. However, the sharing and linkage of data between different sectors are still relatively rare. Previous research has documented the perspectives of researchers and members of the public on data sharing, especially healthcare data, but has not engaged with relevant policy and practice decision-makers.

Aim

We sought to work collaboratively with decision-makers relevant to healthy public policy and practice in Scotland to identify practical ways that cross-sectoral data sharing and linkage could be used to best effect to improve health and reduce health inequalities.

Methods

We facilitated three sequential stakeholder workshops with 20 participants from local and central government, public health teams, Health and Social Care Partnerships, the third sector, organisations which support data-intensive research and public representatives from across Scotland. Workshops were informed by two scoping reviews (carried out in June 2021) and three case studies of existing cross-sectoral linkage projects. Workshop activities included brainstorming of factors that would help participants make better decisions in their current role; reflective questions on lessons learnt from the case studies; and identifying and prioritising recommendations for change. Findings were synthesised using thematic analysis.

Setting and scope

Scotland; public and third sector data.

Results

Based on the workshops, and supported by the reviews and case studies, we created a visual representation of the use of evidence, and secondary data in particular, in decision-making for healthy public policy and practice. This covered three key overarching themes: differing understandings of evidence; diverse functions of evidence; and factors affecting use (such as technical, political and institutional, workforce and governance). Building on this, workshop participants identified six guiding principles for cross-sectoral data sharing and linkage: it should be pragmatic; participatory; ambitious; fair; iterative; with holistic and proportionate governance. Participants proposed 21 practical actions to this end, including: a strategic approach to identifying and sharing key data sets; streamlining governance processes (e.g. through standardised data sharing agreements; central data repositories; and a focus on reusable data resources) and building workforce capacity. To make these possible, participants identified a need for strong political and organisational leadership as well as a transparent and inclusive public conversation.

Limitations

Participation from some stakeholders was limited by workload pressures associated with the COVID-19 pandemic. No consensus was reached on the impact, effort, and/or timing of some recommendations. Findings were closely informed by the Scottish context but are nonetheless likely to be relevant to other jurisdictions.

Conclusions

There is broad consensus among key stakeholders that linked cross-sectoral data can be used far more extensively for public health decision-making than it is at present. No single change will lead to improved use of such data: a range of technical, organisational and political constraints must be addressed.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme as award number NIHR133585.

Identification of Intervention Characteristics Within Diabetes Prevention Programs Using the Template for Intervention Description and Replication: A Scoping Review

OBJECTIVES

Diabetes prevention programs (DPPs) targeting dietary and physical activity behaviour change have been shown to decrease the incidence of type 2 diabetes; however, a more thorough reporting of intervention characteristics is needed to expedite the translation of such programs into different communities. In this scoping review, we aim to synthesize how DPPs are being reported and implemented.

METHODS

A scoping review using Arkey and O'Malley methods was conducted in accordance with the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analysis. MEDLINE, CINAHL, PsycINFO, EMBASE, and SPORTDiscus were searched for studies relating to diabetes prevention and diet/exercise interventions. Only studies delivering a diet/exercise intervention for adults identified as "at risk" for developing type 2 diabetes were included. The Template for Intervention Description and Replication (TIDieR) was used to guide data extraction, and each DPP was scored on a scale from 0 to 2 for how thoroughly it reported each of the items (0 = did not report, 2 = reported in full; total score out of 26).

RESULTS

Of the 25,110 publications screened, 351 (based on 220 programs) met the inclusion criteria and were included for data extraction. No studies comprehensively reported on all TIDieR domains (mean TIDieR score: 15.7 of 26; range 7 to 25). Reporting was particularly poor among domains related to "modifications," "tailoring," and "how well (planned/actual)." "How well (planned)" assesses the intended delivery of an intervention, detailing the initial strategies and components as per the original design, whereas "how well (actual)" evaluates the extent to which the intervention was executed as planned during the study, including any deviations or modifications made in practice.

CONCLUSIONS

Although there is evidence to suggest that DPPs are efficacious, a more thorough reporting of program content and delivery is needed to improve the ability for effective programs to be implemented or translated into different communities.

The Power of Rapid Reviews for Bridging the Knowledge-to-Action Gap in Evidence-Based Virtual Health Care

Despite the surge in popularity of virtual health care services as a means of delivering health care through technology, the integration of research evidence into practice remains a challenge. Rapid reviews, a type of time-efficient evidence synthesis, offer a potential solution to bridge the gap between knowledge and action. This paper aims to highlight the experiences of the Fraser Health Authority's Virtual Health team in conducting rapid reviews. This paper discusses the experiences of the Virtual Health team in conducting 15 rapid reviews over the course of 1.5 years and the benefit of involving diverse stakeholders including researchers, project and clinical leads, and students for the creation of user-friendly knowledge products to summarize results. The Virtual Health team found rapid reviews to be a valuable tool for evidence-informed decision-making in virtual health care. Involving stakeholders and focusing on implementation considerations are crucial for maximizing the impact of rapid reviews. Health care decision makers are encouraged to consider implementing rapid review processes to improve the translation of research evidence into practice, ultimately enhancing patient outcomes and promoting a culture of evidence-informed care.

Co-design workshops to develop evidence synthesis summary formats for use by clinical guideline development groups

BACKGROUND

Evidence synthesis is used by decision-makers in various ways, such as developing evidence-based recommendations for clinical guidelines. Clinical guideline development groups (GDGs) typically discuss evidence synthesis findings in a multidisciplinary group, including patients, healthcare providers, policymakers, etc. A recent mixed methods systematic review (MMSR) identified no gold standard format for optimally presenting evidence synthesis findings to these groups. However, it provided 94 recommendations to help produce more effective summary formats for general evidence syntheses (e.g., systematic reviews). To refine the MMSR recommendations to create more actionable guidance for summary producers, we aimed to explore these 94 recommendations with participants involved in evidence synthesis and guideline development.

METHODS

We conducted a descriptive qualitative study using online focus group workshops in February and March 2023. These groups used a participatory co-design approach with interactive voting activities to identify preferences for a summary format's essential content and style. We created a topic guide focused on recommendations from the MMSR with mixed methods support, ≥ 3 supporting studies, and those prioritized by an expert advisory group via a pragmatic prioritization exercise using the MoSCoW method (Must, Should, Could, and Will not haves). Eligible participants must be/have been involved in GDGs and/or evidence synthesis. Groups were recorded and transcribed. Two independent researchers analyzed transcripts using directed content analysis with 94 pre-defined codes from the MMSR.

RESULTS

Thirty individuals participated in six focus groups. We coded 79 of the 94 pre-defined codes. Participants suggested a "less is more" structured approach that minimizes methodological steps and statistical data, promoting accessibility to all audiences by judicious use of links to further information in the full report. They emphasized concise, consistently presented formats that highlight key messages, flag readers to indicators of trust in the producers (i.e., logos, websites, and conflict of interest statements), and highlight the certainty of evidence (without extenuating details).

CONCLUSIONS

This study identified guidance based on the preferences of guideline developers and evidence synthesis producers about the format of evidence synthesis summaries to support decision-making. The next steps involve developing and user-testing prototype formats through one-on-one semi-structured interviews to optimize evidence synthesis summaries and support decision-making.

Identifying characteristics of resistance-based therapeutic exercise interventions for Achilles tendinopathy: A scoping review

OBJECTIVE

This scoping review describes resistance-based therapeutic exercise intervention characteristics for Achilles tendinopathy (AT) treatment (e.g., therapeutic dose, underlying mechanisms targeted by exercise) and assesses participant reporting characteristics.

METHODS

Seven electronic databases were searched; studies delivering a resistance exercise-focused treatment for individuals with AT were included. The Template for Intervention Description and Replication (TIDieR) and the ICON 2019 'Recommended standards for reporting participant characteristics in tendinopathy research' checklists framed data extraction, and study quality was assessed using the Mixed Methods Appraisal Tool 2018 version.

RESULTS

68 publications (describing 59 studies and 72 exercise programs) were included. Results demonstrate that therapeutic exercise interventions for AT are well reported according to the TIDieR checklist, and participant characteristics are well reported according to the ICON checklist. Various underlying therapeutic mechanisms were proposed, with the most common being increasing tendon strength, increasing calf muscle strength, and enhancing collagen synthesis.

CONCLUSIONS

While evidence suggests that resistance-based therapeutic exercise interventions are effective in treating AT, more reporting on program fidelity, adherence, and compliance is needed. By summarizing currently published AT exercise programs and reporting key intervention characteristics in a single location, this review can assist clinicians in developing individualized resistance training programs for people with AT.

Rapid Reviews to Support Practice: A Guide for Professional Organization Practice Networks

Background. Occupational Therapists, among other healthcare decision makers, often need to make decisions within limited timeframes and cannot wait for the completion of large rigorous systematic reviews and meta-analyses. Rapid reviews are one method to increase the integration of research evidence into clinical decision making. Rapid reviews streamline the systematic review process to allow for the timely synthesis of evidence; however, there does not exist a single agreed upon guide for the methodology and reporting of rapid reviews. Purpose. This paper proposes a rapid review methodology that is customized to a professional organization practice which can feasibly be used by practice networks such as those of the Canadian Association for Occupational Therapy to conduct reviews. Implications. Practice networks provide a sustainable mechanism to integrate research evidence and foster communication amongst practitioners. This guide for conducting and reporting rapid reviews can be used across Occupational Therapy practice networks and similar groups to support the consistent and timely synthesis of evidence necessary to improve evidence-informed clinical decision making.

Improving academic and public health impact of Cochrane public health reviews: what can we learn from bibliographic metrics and author dissemination strategies? A cross-sectional study

BACKGROUND

To facilitate the development of impactful research dissemination strategies, this study aimed to: (i) survey authors of trials included in a sample of Cochrane reviews to describe strategies to disseminate trial findings, and examine their association with academic and policy impacts and (ii) audit academic and policy impact of CPH reviews.

METHODS

Authors of 104 trials within identified Cochrane reviews completed survey items assessing the dissemination strategies. Field weighted citation (FWCI) data extracted from bibliographic databases served as a measure of academic impact of trials and CPH reviews. Policy and practice impacts of trials were assessed during the survey of trial authors using items based on the Payback Framework, and for CPH reviews using 'policy mention' data collected via Altmetric Explorer.

RESULTS

Among the included trials, univariate (but not multivariable) regression models revealed significant associations between the use of dissemination strategies (i.e. posts on social media; workshops with end-users; media-releases) and policy or practice impacts. No significant associations were reported between dissemination strategies and trial FWCI. The mean FWCI of CPH reviews suggest that they are cited 220% more than other reviews in their field.

CONCLUSIONS

Comprehensive dissemination strategies are likely required to maximize the potential the potential impacts of public health research.

Strategies for communicating scientific evidence on healthcare to managers and the population: a scoping review

BACKGROUND

Health evidence needs to be communicated and disseminated in a manner that is clearly understood by decision-makers. As an inherent component of health knowledge translation, communicating results of scientific studies, effects of interventions and health risk estimates, in addition to understanding key concepts of clinical epidemiology and interpreting evidence, represent a set of essential instruments to reduce the gap between science and practice. The advancement of digital and social media has reshaped the concept of health communication, introducing new, direct and powerful communication platforms and gateways between researchers and the public. The objective of this scoping review was to identify strategies for communicating scientific evidence in healthcare to managers and/or population.

METHODS

We searched Cochrane Library, Embase®, MEDLINE® and other six electronic databases, in addition to grey literature, relevant websites from related organizations for studies, documents or reports published from 2000, addressing any strategy for communicating scientific evidence on healthcare to managers and/or population.

RESULTS

Our search identified 24 598 unique records, of which 80 met the inclusion criteria and addressed 78 strategies. Most strategies focused on risk and benefit communication in health, were presented by textual format and had been implemented and somehow evaluated. Among the strategies evaluated and appearing to yield some benefit are (i) risk/benefit communication: natural frequencies instead of percentages, absolute risk instead relative risk and number needed to treat, numerical instead nominal communication, mortality instead survival; negative or loss content appear to be more effective than positive or gain content; (ii) evidence synthesis: plain languages summaries to communicate the results of Cochrane reviews to the community were perceived as more reliable, easier to find and understand, and better to support decisions than the original summaries; (iii) teaching/learning: the Informed Health Choices resources seem to be effective for improving critical thinking skills.

CONCLUSION

Our findings contribute to both the knowledge translation process by identifying communication strategies with potential for immediate implementation and to future research by recognizing the need to evaluate the clinical and social impact of other strategies to support evidence-informed policies. Trial registration protocol is prospectively available in MedArxiv (doi.org/10.1101/2021.11.04.21265922).

An appraisal of the methodology and quality of evidence of systematic reviews on the efficacy of prone positional ventilation in adult patients with acute respiratory distress syndrome: an umbrella review

The objective of the study was to evaluate all available systematic reviews on the use of prone positional ventilation in adult patients with acute respiratory distress syndrome (ARDS). An umbrella review on the efficacy of prone positional ventilation in adult patients ventilation in adult patients with acute respiratory distress syndrome was conducted. We performed a systematic search in the database of Medline (Pubmed), Scopus, Cochrane Library, Web of Science, and Epistemonikos. The ROBIS tools and GRADE methodology were used to assess the risk of bias and certainty of evidence. We estimated the necessary number of patients to be treated to have benefit. For the synthesis of the result, we selected the review with the lowest risk of bias. Sixteen systematic reviews including 64 randomized clinical trials and evaluating the effect of prone positional ventilation, with or without other ventilation strategies were included. Aoyama 2019 observed prone positioning, without complementary ventilation strategies, leading to a reduction in the 28-day mortality only when compared to high-frequency oscillatory ventilation (RR 0.61; 95% CI 0.39-0.95) and lung-protective ventilation in the supine position (RR 0.69; 95% CI 0.48-0.98), with an ARR of 9.32% and 14.94%, an NNTB of 5.89 and 8.04, and a low and moderate certainty of evidence, respectively. Most reviews had severe methodological flaws that led to results with very low certainty of evidence. The review with the lowest risk of bias presented results in favor of prone positional ventilation compared with high-frequency oscillatory ventilation and lung-protective ventilation. There is a need to update the available reviews to obtain more accurate results.

Publication bias in meta-analyses of the therapeutic efficacy of remdesivir interventions for patients with COVID-19

Purpose

No comprehensive statistical assessment of publication bias has been conducted in remdesivir-based intervention research for COVID-19 patients. This study aims to examine all meta-analyses of the efficacy of remdesivir interventions in COVID-19 patients and perform a statistical assessment of publication bias.

Design/methodology/approach

This is an analytic study conducted to assess the impact of publication bias on the results of meta-analyses of remdesivir-based interventions in patients infected with COVID-19. All English full-text meta-analyses published in peer-reviewed journals in 2019–2021 were included. A computerized search of PubMed and Web of Science electronic databases was performed on December 24, 2021. The trim-and-fill method calculated the number of missing studies and the adjusted cumulative effect sizes.

Findings

The final analysis comprised 21 studies with 88 outcomes. The investigation revealed missing studies in 46 outcomes (52%). Seventy-six missing studies were replaced in the outcomes using the trim-and-fill procedure. The adjusted recalculated effect sizes of the 27 outcomes increased by an average of 0.04. In comparison, the adjusted effect size of 18 outcomes fell by an average of 0.036. Only 14 out of 46 outcomes with publication bias were subjected to a gray literature search (30%). To discover related research, no gray literature search was conducted in most outcomes with publication bias (n = 32; 70%). In conclusion, the reported effect estimates regarding the effect of remdesivir in COVID-19 patients are only slightly affected by publication bias and can be considered authentic. Health-care decision-makers in COVID-19 should consider current research results when making clinical decisions.

Research limitations/implications

Most health decisions are based on the effect sizes revealed in meta-analyses. When deciding on remdesivir-based treatment for COVID-19 patients, therefore, the outcomes of this investigation may be of paramount importance to health policymakers, leading to better treatment strategies.

Practical implications

According to the results, no major publication bias and missing studies were detected on average. Therefore, the calculated effect sizes of remdesivir-based interventions on meta-analyses can be used as authentic and unbiased benchmarks by health-care decision-makers in treating patients with COVID-19.

Originality/value

This is the first study to examine the effect of publication bias and gray literature searches on the results of meta-analyses of treatment with COVID-19 (remdesivir).

User experiences of structured stakeholder engagement to consider transferability: The TRANSFER approach

Background

Systematic reviews are increasingly used to inform decision-making in health, education, social care and environmental protection. However, decision makers still experience barriers to using reviews, including not knowing how findings might translate to their own contexts, and lack of collaboration with systematic review authors. The TRANSFER approach is a novel method that aims to support review authors to systematically and transparently collaborate with stakeholders to consider context and the transferability of review findings from the beginning of the review process. Such collaboration is intended to improve the usefulness and relevance of review findings for decision makers.

Objectives

We aim to explore the user experience of the TRANSFER approach conversation guide, and in doing so gain a better understanding of the role and perceived value of stakeholder engagement in systematic reviews for informed decision-making.

Methods

We conducted four user tests of groups using the guide, organized around simulated meetings between review authors and stakeholders. Review authors led the meeting using the TRANSFER approach conversation guide. We audio-recorded and observed the meetings, collected feedback forms and conducted semi-structured interviews with review authors following the meeting. We analysed the data using framework analysis to examine the user experience of the TRANSFER approach conversation guide and of stakeholder engagement more generally.

Results

Seventeen participants in four user groups participated in the user tests. Most participants were generally positive toward the structured approach using the conversation guide, and felt it would be useful in systematic review projects. We observed examples of misunderstanding of the terminology included in the guide, and received multiple suggestions for how to make the conversation guide more user friendly. We observed numerous challenges related to the hypothetical nature of a user test, including lack of familiarity with the review question/topic among participants and lack of preparation for the meeting.

Conclusions

Review authors and stakeholders are positive toward using a structured approach to guide collaboration within the context of a systematic review. The TRANSFER conversation guide helps participants to discuss the review question and context in a structured way. Such structured collaboration could help to improve the usefulness and relevance of systematic reviews for decision making by improving the review question, inclusion criteria and consideration of transferability of review findings. The conversation guide needs to be modified to improve user experience. Further research is needed to explore stakeholder collaboration and the use of the TRANSFER conversation guide in systematic review processes.

Rapid evidence synthesis to enable innovation and adoption in health and social care

BACKGROUND

The rapid identification and adoption of effective innovations in healthcare is a known challenge. The strongest evidence base for innovations can be provided by evidence synthesis, but this is frequently a lengthy process and even rapid versions of this can be time-consuming and complex. In the UK, the Accelerated Access Review and Academic Health Science Network (AHSN) have provided the impetus to develop a consistently rapid process to support the identification and adoption of high-value innovations in the English NHS.

METHODS

The Greater Manchester Applied Research Collaboration (ARC-GM) developed a framework for a rapid evidence synthesis (RES) approach, which is highly integrated within the innovation process of the Greater Manchester AHSN and the associated healthcare and research ecosystem. The RES uses evidence synthesis approaches and draws on the GRADE Evidence to Decision framework to provide rapid assessments of the existing evidence and its relevance to specific decision problems. We implemented this in a real-time context of decision-making around adoption of innovative health technologies.

RESULTS

Key stakeholders in the Greater Manchester decision-making process for healthcare innovations have found that our approach is both timely and flexible; it is valued for its combination of rigour and speed. Our RES approach rapidly and systematically identifies, appraises and contextualises relevant evidence, which can then be transparently incorporated into decisions about the wider adoption of innovations. The RES also identifies limitations in existing evidence for innovations and this can inform subsequent evaluations. There is substantial interest from other ARCs and AHSNs in implementing a similar process. We are currently exploring methods to make completed RES publicly available. We are also exploring methods to evaluate the impact of using RES as more implementation decisions are made.

CONCLUSIONS

The RES framework we have implemented combines transparency and consistency with flexibility and rapidity. It therefore maximises utility in a real-time decision-making context for healthcare innovations.

The effectiveness and acceptability of evidence synthesis summary formats for clinical guideline development groups: a mixed-methods systematic review

INTRODUCTION

Clinical guideline development often involves a rigorous synthesis of evidence involving multidisciplinary stakeholders with different priorities and knowledge of evidence synthesis; this makes communicating findings complex. Summary formats are typically used to communicate the results of evidence syntheses; however, there is little consensus on which formats are most effective and acceptable for different stakeholders.

METHODS

This mixed-methods systematic review (MMSR) aimed to evaluate the effectiveness and acceptability (e.g. preferences and attitudes and preferences towards) of evidence synthesis summary formats for GDG members. We followed the PRISMA 2020 guideline and Joanna Briggs Institute Manual for Evidence Synthesis for MMSRs. We searched six databases (inception to April 20, 2021) for randomised controlled trials (RCTs), RCTs with a qualitative component, and qualitative studies. Screening, data extraction, and quality appraisal were performed in duplicate. Qualitative findings were synthesised using meta-aggregation, and quantitative findings are described narratively.

RESULTS

We identified 17,240 citations and screened 54 full-text articles, resulting in 22 eligible articles (20 unique studies): 4 articles reported the results of 5 RCTs, one of which also had a qualitative component. The other 18 articles discussed the results of 16 qualitative studies. Therefore, we had 5 trials and 17 qualitative studies to extract data from. Studies were geographically heterogeneous and included a variety of stakeholders and summary formats. All 5 RCTs assessed knowledge or understanding with 3 reporting improvement with newer formats. The qualitative analysis identified 6 categories of recommendations: 'presenting information', 'tailoring information' for end users, 'trust in producers and summary', 'knowledge required' to understand findings, 'quality of evidence', and properly 'contextualising information'. Across these categories, the synthesis resulted in 126 recommendations for practice. Nine recommendations were supported by both quantitative and qualitative evidence and 116 by only qualitative. A majority focused on how to present information (n = 64) and tailor content for different end users (n = 24).

CONCLUSIONS

This MMSR provides guidance on how to improve evidence summary structure and layout. This can be used by synthesis producers to better communicate to GDGs. Study findings will inform the co-creation of evidence summary format prototypes based on GDG member's needs. Trial registration The protocol for this project was previously published, and the project was preregistered on Open Science Framework (Clyne and Sharp, Evidence synthesis and translation of findings for national clinical guideline development: addressing the needs and preferences of guideline development groups, 2021; Sharp and Clyne, Evidence synthesis summary formats for decision-makers and Clinical Guideline Development Groups: A mixed-methods systematic review protocol, 2021).

Dentists' awareness and knowledge of evidence- based dentistry principles, methods and practices: a systematic review

Introduction As an attempt to provide supporting evidence for the formulation of future educational strategies on knowledge translation, this systematic review assessed and synthesised the available evidence related to the dentists' awareness, perceived and actual knowledge of evidence-based dentistry (EBD) principles, methods and practices.Methods Primary studies that considered dentists' reports collected from interviews, questionnaires, or conversation sessions were selected. Studies enrolling students, dental hygienists, or other health professionals were not included. Reviews, editorials, letters, study protocols, articles presenting knowledge translation strategies and initiatives, examples of EBD approaches to specific clinical questions, and guidelines focused on EBD implementation were also excluded. Cochrane, Embase, PubMed, Scopus and Web of Science databases were searched. Grey literature was partially covered by the Google Scholar search and the reference lists of the pre-selected studies. The study search was concluded in February 2021. Descriptive data of the selected studies were synthesised, and the risk of bias was assessed according to the National Institutes of Health Quality Assessment Tool for observational cohort and cross-sectional studies.Results Twenty-one articles were included. High percentages of dentists were aware of EBD. Variable proportions of professionals declared to have some understanding of EBD, although few presented actual knowledge of principles, methods and practices.Discussion Methodologically, most studies presented limitations regarding sample representativity, participation rates, detailing of the outcome measures, and validation of the assessment tools. Additionally, extensive overall ranges of responses were often observed across the studies, possibly as a result of heterogeneity across samples and assessment tools. The authors thus suggest developing valid questionnaires including all dimensions (awareness, perceived knowledge and actual knowledge) within an assessment tool. This would contribute to establishing knowledge translation strategies to overcome specific gaps in EBD knowledge.

Paramedic management of back pain: a scoping review

BACKGROUND

Research examining paramedic care of back pain is limited.

OBJECTIVE

To describe ambulance service use and usual paramedic care for back pain, the effectiveness and safety of paramedic care of back pain, and the characteristics of people with back pain who seek care from paramedics.

METHODS

We included published peer-reviewed studies of people with back pain who received any type of paramedic care on-scene and/or during transport to hospital. We searched MEDLINE, EMBASE, CINAHL, Web of Science and SciELO from inception to July 2022. Two authors independently screened and selected the studies, performed data extraction, and assessed the methodological quality using the PEDro, AMSTAR 2 and Hawker tools. This review followed the JBI methodological guidance for scoping reviews and PRISMA extension for scoping reviews.

RESULTS

From 1987 articles we included 26 articles (25 unique studies) consisting of 22 observational studies, three randomised controlled trials and one review. Back pain is frequently in the top 3 reasons for calls to an ambulance service with more than two thirds of cases receiving ambulance dispatch. It takes ~ 8 min from time of call to an ambulance being dispatched and 16% of calls for back pain receive transport to hospital. Pharmacological management of back pain includes benzodiazepines, NSAIDs, opioids, nitrous oxide, and paracetamol. Non-pharmacological care is poorly reported and includes referral to alternate health service, counselling and behavioural interventions and self-care advice. Only three trials have evaluated effectiveness of paramedic treatments (TENS, active warming, and administration of opioids) and no studies provided safety or costing data.

CONCLUSION

Paramedics are frequently responding to people with back pain. Use of pain medicines is common but varies according to the type of back pain and setting, while non-pharmacological care is poorly reported. There is a lack of research evaluating the effectiveness and safety of paramedic care for back pain.

Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews

Background

The use of research evidence as an input for health decision-making is a need for most health systems. There are a number of approaches for promoting evidence use at different levels of the health system, but knowledge of their effectiveness is still scarce. The objective of this overview was to evaluate the effectiveness of knowledge communication and dissemination interventions, strategies or approaches targeting policy-makers and health managers.

Methods

This overview of systematic reviews used systematic review methods and was conducted according to a predefined and published protocol. A comprehensive electronic search of 13 databases and a manual search in four websites were conducted. Both published and unpublished reviews in English, Spanish or Portuguese were included. A narrative synthesis was undertaken, and effectiveness statements were developed, informed by the evidence identified.

Results

We included 27 systematic reviews. Three studies included only a communication strategy, while eight only included dissemination strategies, and the remaining 16 included both. None of the selected reviews provided “sufficient evidence” for any of the strategies, while four provided some evidence for three communication and four dissemination strategies. Regarding communication strategies, the use of tailored and targeted messages seemed to successfully lead to changes in the decision-making practices of the target audience. Regarding dissemination strategies, interventions that aimed at improving only the reach of evidence did not have an impact on its use in decisions, while interventions aimed at enhancing users’ ability to use and apply evidence had a positive effect on decision-making processes. Multifaceted dissemination strategies also demonstrated the potential for changing knowledge about evidence but not its implementation in decision-making.

Conclusions

There is limited evidence regarding the effectiveness of interventions targeting health managers and policy-makers, as well as the mechanisms required for achieving impact. More studies are needed that are informed by theoretical frameworks or specific tools and using robust methods, standardized outcome measures and clear descriptions of the interventions. We found that passive communication increased access to evidence but had no effect on uptake. Some evidence indicated that the use of targeted messages, knowledge-brokering and user training was effective in promoting evidence use by managers and policy-makers.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-021-00780-4.

Surgery is no more effective than conservative treatment for Femoroacetabular impingement syndrome: Systematic review and meta-analysis of randomized controlled trials

OBJECTIVE

To summarize the effects of surgical treatment compared to conservative treatment in femoroacetabular impingement syndrome in the short, medium, and long term.

STUDY DESIGN

Systematic review.

METHODS

The following databases were searched on 14/09/2020: MEDLINE, EMBASE, CENTRAL, Web of Science, and PEDro. There were no date or language limits. The methodological quality assessment was performed using the PEDro scale and the quality of the evidence followed the GRADE recommendation. The outcomes pain, disability, and adverse effects were extracted.

RESULTS

Of 6264 initial studies, three met the full-text inclusion criteria. All studies were of good methodological quality. Follow up ranged from six months to two years, with 650 participants in total. The meta-analyses found no difference in disability between surgical versus conservative treatment, with a mean difference (MD) between groups of 3.91 points (95% CI -2.19 to 10.01) at six months, MD of 5.53 points (95% CI -3.11 to 14.16) at 12 months and 3.8 points (95% CI -6.0 to 13.6) at 24 months. The quality of the evidence (GRADE) varied from moderate to low across all comparisons.

CONCLUSION

There is moderate-quality evidence that surgical treatment is not superior to conservative treatment for femoroacetabular impingement syndrome in the short term, and there is low-quality evidence that it is not superior in the medium term.

LEVEL OF EVIDENCE

Therapy, level 1a.

REGISTRATION NUMBER

PROSPERO CRD42019134118.

Methodological components and quality of evidence summaries: a scoping review protocol

OBJECTIVE

The objective of this review is to identify and map the available evidence related to evidence summary methodologies and indicators of quality.

INTRODUCTION

It can be challenging for clinicians and policy makers to keep up-to-date with current evidence and best practice. An evidence summary is a way to provide health care decision makers with the most recent, highest quality evidence available on a particular topic in an easily digestible format to facilitate evidence-based clinical decisions. However, objectively evaluating the methodological quality of these types of evidence reviews is challenging.

INCLUSION CRITERIA

Articles, papers, books, dissertations, reports and websites will be included if they evaluate, or describe the development or appraisal of, an evidence summary methodology.

METHODS

A three-step search strategy will be used to find both published and unpublished literature. The following databases will be searched: US National Library of Medicine Database (PubMed) Cumulative Index to Nursing and Allied Health Literature (CINAHL), Scopus, ProQuest Dissertations and Theses, and Embase. The gray literature search will include relevant government and university websites, the Health Evidence Network website, the World Health Organization (WHO) Health Evidence Network website, the McMaster Health Systems Evidence website, and relevant websites included in the Canadian Agency for Drugs and Technologies in Health (CADTH) Grey Matters Handbook. Sources published in English will be considered, with no date limitation.

Occupational health and safety hazards faced by healthcare professionals in Taiwan: A systematic review of risk factors and control strategies

Background

Healthcare professionals in Taiwan are exposed to a myriad of occupational health and safety hazards, including physical, biological, chemical, ergonomic, and psychosocial hazards. Healthcare professionals working in hospitals and healthcare facilities are more likely to be subjected to these hazards than their counterparts working in other areas.

Objectives

This review aims to assess current research literature regarding this situation with a view to informing policy makers and practitioners about the risks of exposure and offer evidence-based recommendations on how to eliminate or reduce such risks.

Methods

Using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses review strategy, we conducted a systematic review of studies related to occupational health and safety conducted between January 2000 and January 2019 using MEDLINE (Ovid), PubMed, PMC, TOXLINE, CINAHL, PLOS One, and Access Pharmacy databases.

Results

The review detected 490 studies addressing the issue of occupational health and safety hazards; of these, 30 articles were included in this systematic review. These articles reported a variety of exposures faced by healthcare professionals. This review also revealed a number of strategies that can be adopted to control, eliminate, or reduce hazards to healthcare professionals in Taiwan.

Conclusion

Hospitals and healthcare facilities have many unique occupational health and safety hazards that can potentially affect the health and performance of healthcare professionals. The impact of such hazards on healthcare professionals poses a serious public health issue in Taiwan; therefore, controlling, eliminating, or reducing exposure can contribute to a stronger healthcare workforce with great potential to improve patient care and the healthcare system in Taiwan. Eliminating or reducing hazards can best be achieved through engineering measures, administrative policy, and the use of personal protective equipment.

Implications

This review has research, policy, and practice implications and provides future students and researchers with information on systematic review methodologies based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses strategy. It also identifies occupational health and safety risks and provides insights and strategies to address them.

Barriers involved in the application of evidence-based dentistry principles: A systematic review

BACKGROUND

The authors' objective in this systematic review was to investigate the barriers involved in the application of evidence-based dentistry principles, as reported by dentists. The authors registered the protocol in the PROSPERO database.

TYPES OF STUDIES REVIEWED

Eligible studies included qualitative and quantitative approaches, constituting information about barriers, collected through interviews, questionnaires, or conversation sessions. The authors searched databases and reference lists of preselected studies. After the selection process, the authors evaluated the included studies for potential risk of bias and collected either qualitative or quantitative data.

RESULTS

After the selection process, the authors included 35 studies, of which 16 were reported in this article. The authors synthesized and classified the barriers in 4 categories: self-related, evidence-related, context-related, and patient-related barriers. Shortage of time and financial constraints were the barriers most frequently studied. However, the quantification of these barriers, as well as others, was not possible because of the variability of the results and methodological issues of the included studies.

CONCLUSIONS AND PRACTICAL IMPLICATIONS

The authors suggest the development of valid questionnaires and their use in representative samples to quantify the effects of specific barriers. The authors encourage practitioners to participate in educational programs focused on training in evidence-based dentistry abilities, in addition to seeking accessible and synthesized formats of reliable scientific knowledge.

Supporting the use of research evidence in decision-making in crisis zones in low- and middle-income countries: a critical interpretive synthesis

BACKGROUND

Decision-makers in crisis zones are faced with the challenge of having to make health-related decisions under limited time and resource constraints and in light of the many factors that can influence their decisions, of which research evidence is just one. To address a key gap in the research literature about how best to support the use of research evidence in such situations, we conducted a critical interpretive synthesis approach to develop a conceptual framework that outlines the strategies that leverage the facilitators and address the barriers to evidence use in crisis zones.

METHODS

We systematically reviewed both empirical and non-empirical literature and used an interpretive analytic approach to synthesise the results and develop the conceptual framework. We used a 'compass' question to create a detailed search strategy and conducted electronic searches in CINAHL, EMBASE, MEDLINE, SSCI and Web of Science. A second reviewer was assigned to a representative sample of articles. We purposively sampled additional papers to fill in conceptual gaps.

RESULTS

We identified 21 eligible papers to be analysed and purposively sampled an additional 6 to fill conceptual gaps. The synthesis resulted in a conceptual framework that focuses on evidence use in crisis zones examined through the lens of four systems - political, health, international humanitarian aid and health research. Within each of the four systems, the framework identifies the most actionable strategies that leverage the facilitators and address the barriers to evidence use.

CONCLUSIONS

This study presents a new conceptual framework that outlines strategies that leverage the facilitators and address the barriers to evidence use in crisis zones within different systems. This study expands on the literature pertaining to evidence-informed decision-making.

The use of evidence in decision making by hospital managers in Lebanon: A cross-sectional study

BACKGROUND

Knowledge synthesis products have emerged as support agents for decision making in clinical practice and policy. However, their use for evidence-informed decision making remains limited in health care management especially in low- and middle-income countries. This study assesses the use of evidence by middle and senior managers in Lebanese hospitals.

METHODS

This multihospital cross-sectional study used a self-administered web survey of middle and senior managers. Hospitals were purposively selected, and data were analyzed using descriptive statistics and thematic analysis.

RESULTS

Hospital participation rate was 25%, while adjusted managers' response rate was 44.8%. Prevalence of using evidence was 70%, while prevalence of evidence-seeking behavior was 90%. Evidence was mainly used in design of policies, protocols, and procedures; nursing issues; or procurement decisions. Facilitators for evidence-informed decision making included upper management support and organizational culture, whereas limited resources such as funding, time, and training hindered use of evidence.

CONCLUSIONS

Findings indicate that utilization of evidence was comparable with that of high-income countries. Training and continuous education were crucial for advancing evidence-informed decision making among hospital managers. However, neither the quality nor the sources of evidence used for decision making were assessed in this study. Future studies should assess the quality and sources of evidence utilized in decision making.

A systematic map and in-depth review of European telehealth interventions efficacy for chronic obstructive pulmonary disease

BACKGROUND

Evidence to support the implementation of telehealth (TH) interventions in the management of chronic obstructive pulmonary disease (COPD) varies throughout Europe. Despite more than ten years of TH research in COPD management, it is still not possible to define which TH interventions are beneficial to which patient group. Therefore, informing policymakers on TH implementation is complicated. We aimed to examine the provision and efficacy of TH for COPD management to guide future decision-making.

METHODS

A mapping study of twelve systematic reviews of TH interventions for COPD management was conducted. This was followed by an in-depth review of fourteen clinical trials performed in Europe extracted from the systematic reviews. Efficacy outcomes for COPD management were synthesized.

RESULTS

The mapping study revealed that systematic reviews with a meta-analysis often report positive clinical outcomes. Despite this, we identified a lack of pragmatic trial design affecting the synthesis of reported outcomes. The in-depth review visualized outcomes for three TH categories, which revealed a plethora of heterogeneous outcomes. Suggestions for reporting within these three outcomes are synthesized as targets for future empirical research reporting.

CONCLUSION

The present study indicates the need for more standardized and updated systematic reviews. Policymakers should advocate for improved TH trial designs, focusing on the entire intervention's adoption process evaluation. One of the policymakers' priorities should be the harmonization of the outcome sets, which would be considered suitable for deciding about subsequent reimbursement. We propose possible outcome sets in three TH categories which could be used for discussion with stakeholders.

Beginning with high value care in mind: A scoping review and toolkit to support the content, delivery, measurement, and sustainment of high value care

OBJECTIVE

To create a shared vision for the content, delivery, measurement, and sustainment of patient-centered high value care.

METHODS

We performed a scoping review and translated findings into toolkit for system leaders. For our scoping review, we searched Medline, 2005-November 2015, for literature on patient-centered care (PCC) and its relationship to a high value care change model. We supplemented searches with key author, Google Scholar, and key website searches. One author reviewed all titles, abstracts, and articles for inclusion; another reviewed a random 20%. To develop our toolkit, we translated evidence into simple, actionable briefs on key topics and added resources. We then iteratively circulated briefs and the overall toolkit to potential users, making updates as needed.

RESULTS

In our scoping review, we found multiple interventions and measures to support the components of PCC and our change model. We found little on the overall effects of PCC or how PCC creates value. Potential users reported our toolkit was simple, understandable, thorough, timely, and likely to be globally useful.

CONCLUSIONS

Considerable evidence supports patient-centered high value care and a toolkit garnered enthusiasm.

PRACTICE IMPLICATIONS

The toolkit is ready for use, but needs comparison to other approaches.

Bringing research to the bedside: Knowledge translation in the mental health care of burns patients

Advances in surgical techniques and wound management have improved outcomes for burn patients; however, the psychological impacts on burn survivors have had less attention. There is a higher rate of mental health disorders amongst burns victims, with those with pre-existing mental health conditions likely to have worse outcomes. To implement effective burns care and rehabilitation, knowledge and understanding of mental health issues is required. This position paper discusses the extent to which clinicians currently translate knowledge around mental health and burns into practice to identify enables and inhibitors. Successful knowledge translation requires dissemination and accessibility of information with the capacity and readiness for change. Clinicians and researchers need to identify how translating research to practice can meet the needs of burn survivors. There is a gap in the utilization of evidence concerning mental health and the needs of burns survivors, and we need to understand what we know as compared to what we do. Clinicians are well placed to determine how and why knowledge does not necessarily translate to practice and how they can better accommodate the needs of burn survivors.

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

BACKGROUND

Scoping reviews are a relatively new approach to evidence synthesis and currently there exists little guidance regarding the decision to choose between a systematic review or scoping review approach when synthesising evidence. The purpose of this article is to clearly describe the differences in indications between scoping reviews and systematic reviews and to provide guidance for when a scoping review is (and is not) appropriate.

RESULTS

Researchers may conduct scoping reviews instead of systematic reviews where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct. While useful in their own right, scoping reviews may also be helpful precursors to systematic reviews and can be used to confirm the relevance of inclusion criteria and potential questions.

CONCLUSIONS

Scoping reviews are a useful tool in the ever increasing arsenal of evidence synthesis approaches. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy. Our hope is that with clear guidance available regarding whether to conduct a scoping review or a systematic review, there will be less scoping reviews being performed for inappropriate indications better served by a systematic review, and vice-versa.

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

BACKGROUND

Scoping reviews are a relatively new approach to evidence synthesis and currently there exists little guidance regarding the decision to choose between a systematic review or scoping review approach when synthesising evidence. The purpose of this article is to clearly describe the differences in indications between scoping reviews and systematic reviews and to provide guidance for when a scoping review is (and is not) appropriate.

RESULTS

Researchers may conduct scoping reviews instead of systematic reviews where the purpose of the review is to identify knowledge gaps, scope a body of literature, clarify concepts or to investigate research conduct. While useful in their own right, scoping reviews may also be helpful precursors to systematic reviews and can be used to confirm the relevance of inclusion criteria and potential questions.

CONCLUSIONS

Scoping reviews are a useful tool in the ever increasing arsenal of evidence synthesis approaches. Although conducted for different purposes compared to systematic reviews, scoping reviews still require rigorous and transparent methods in their conduct to ensure that the results are trustworthy. Our hope is that with clear guidance available regarding whether to conduct a scoping review or a systematic review, there will be less scoping reviews being performed for inappropriate indications better served by a systematic review, and vice-versa.

Knowledge translation for realist reviews: a participatory approach for a review on scaling up complex interventions

Background

Knowledge syntheses that use a realist methodology are gaining popularity. Yet, there are few reports in the literature that describe how results are summarised, shared and used. This paper aims to inform knowledge translation (KT) for realist reviews by describing the process of developing a KT strategy for a review on pathways for scaling up complex public health interventions.

Methods

The participatory approach used for the realist review was also used to develop the KT strategy. The approach included three main steps, namely (1) an international meeting focused on interpreting preliminary findings from the realist review and seeking input on KT activities; (2) a targeted literature review on KT for realist reviews; and (3) consultations with primary knowledge users of the review.

Results

The international meeting identified a general preference among knowledge users for findings from the review that are action oriented. A need was also identified for understanding how to tailor findings for specific knowledge user groups in relation to their needs. The literature review identified four papers that included brief descriptions of planned or actual KT activities for specific research studies; however, information was minimal on what KT activities or products work for whom, under what conditions and why. The consultations revealed that KT for realist reviews should consider the following: (1) activities closely aligned with the preferences of specific knowledge user groups; (2) key findings that are sensitive to factors within the knowledge user’s context; and (3) actionable statements that can advance KT goals, activities or products. The KT strategy derived from the three activities includes a planning framework and tailored KT activities that address preferences of knowledge users for findings that are action oriented and context relevant.

Conclusions

This paper provides an example of a KT strategy for realist reviews that blends theoretical and practical insights. Evaluation of the strategy’s implementation will provide useful insights on its effectiveness and potential for broader application.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0374-1) contains supplementary material, which is available to authorized users.

Do evidence summaries increase health policy-makers' use of evidence from systematic reviews? A systematic review

This review summarizes the evidence from six randomized controlled trials that judged the effectiveness of systematic review summaries on policymakers' decision making, or the most effective ways to present evidence summaries to increase policymakers' use of the evidence. This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one. The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary. None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?" Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.

Plain language summary

Policy briefs make systematic reviews easier to understand but little evidence of impact on use of study findings: It is likely that evidence summaries are easier to understand than complete systematic reviews. Whether these summaries increase the use of evidence from systematic reviews in policymaking is not clear.What is this review about?: Systematic reviews are long and technical documents that may be hard for policymakers to use when making decisions. Evidence summaries are short documents that describe research findings in systematic reviews. These summaries may simplify the use of systematic reviews.Other names for evidence reviews are policy briefs, evidence briefs, summaries of findings, or plain language summaries. The goal of this review was to learn whether evidence summaries help policymakers use evidence from systematic reviews. This review also aimed to identify the best ways to present the evidence summary to increase the use of evidence.What are the main findings of this review?: This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one.The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary.None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?"Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read.. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.What do the findings of this review mean?: Our review suggests that evidence summaries help policymakers to better understand the findings presented in systematic reviews. In short, evidence summaries should be developed to make it easier for policymakers to understand the evidence presented in systematic reviews. However, right now there is very little evidence on the best way to present systematic review evidence to policymakers.How up to date is this review?: The authors of this review searched for studies through June 2016.

Executive summary/Abstract

Background: Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. Strategies to promote the use of evidence to decision makers are required, and evidence summaries have been suggested as a facilitator. Evidence summaries include policy briefs, briefing papers, briefing notes, evidence briefs, abstracts, summary of findings tables, and plain language summaries. There are many organizations developing and disseminating systematic review evidence summaries for different populations or subsets of decision makers. However, evidence on the usefulness and effectiveness of systematic review summaries is lacking. We present an overview of the available evidence on systematic review evidence summaries.Objectives: This systematic review aimed to 1) assess the effectiveness of evidence summaries on policy-makers' use of the evidence and 2) identify the most effective summary components for increasing policy-makers' use of the evidence.Search methods: We searched several online databases (Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Global Health Library, Popline, Africa-wide, Public Affairs Information Services, Worldwide Political Science Abstracts, Web of Science, and DfiD), websites of research groups and organizations which produce evidence summaries, and reference lists of included summaries and related systematic reviews. These databases were searched in March-April, 2016.Selection criteria: Eligible studies included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) studies, and interrupted time series (ITS) studies. We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of "evidence summary", "policy brief", or other product derived from systematic reviews that presented evidence in a summarized form. These interventions could be compared to active comparators (e.g. other summary formats) or no intervention.The primary outcomes were: 1) use of systematic review summaries decision-making (e.g. self-reported use of the evidence in policy-making, decision-making) and 2) policymaker understanding, knowledge, and/or beliefs (e.g. changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g. format) of the summaries.Results: Our database search combined with our grey literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text and we included 6 studies (reported in 7 papers, 1661 participants) as well as protocols from 2 ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (4 studies) and perceived usefulness or usability (3 studies). Summary of Findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for Summary of Findings tables, certain elements, such as reporting study event rates and absolute differences were preferred as well as avoiding the use of footnotes. No studies assessed adverse effects. The risks of bias in these studies were mainly assessed as unclear or low however, two studies were assessed as high risk of bias for incomplete outcome data due to very high rates of attrition.Authors' conclusions: Evidence summaries may be easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Sex/gender reporting and analysis in Campbell and Cochrane systematic reviews: a cross-sectional methods study

BACKGROUND

The importance of sex and gender considerations in research is being increasingly recognized. Evidence indicates that sex and gender can influence intervention effectiveness. We assessed the extent to which sex/gender is reported and analyzed in Campbell and Cochrane systematic reviews.

METHODS

We screened all the systematic reviews in the Campbell Library (n = 137) and a sample of systematic reviews from 2016 to 2017 in the Cochrane Library (n = 674). We documented the frequency of sex/gender terms used in each section of the reviews.

RESULTS

We excluded 5 Cochrane reviews because they were withdrawn or published and updated within the same time period as well as 4 Campbell reviews and 114 Cochrane reviews which only included studies focused on a single sex. Our analysis includes 133 Campbell reviews and 555 Cochrane reviews. We assessed reporting of sex/gender considerations for each section of the systematic review (Abstract, Background, Methods, Results, Discussion). In the methods section, 83% of Cochrane reviews (95% CI 80-86%) and 51% of Campbell reviews (95% CI 42-59%) reported on sex/gender. In the results section, less than 30% of reviews reported on sex/gender. Of these, 37% (95% CI 29-45%) of Campbell and 75% (95% CI 68-82%) of Cochrane reviews provided a descriptive report of sex/gender and 63% (95% CI 55-71%) of Campbell reviews and 25% (95% CI 18-32%) of Cochrane reviews reported analytic approaches for exploring sex/gender, such as subgroup analyses, exploring heterogeneity, or presenting disaggregated data by sex/gender.

CONCLUSION

Our study indicates that sex/gender reporting in Campbell and Cochrane reviews is inadequate.

How is the use of research evidence in health policy perceived? A comparison between the reporting of researchers and policy-makers

Background

The use of health policy and systems research (HPSR) to inform health policy-making is an international challenge. Incorporating HPSR into decision-making primarily involves two groups, namely researchers (knowledge producers) and policy-makers (knowledge users). The purpose of this study was to compare the perceptions of Israeli health systems and policy researchers and health services policy-makers regarding the role of HPSR, factors influencing its uses and potential facilitators and barriers to HPSR, and implementation of knowledge transfer and exchange (KTE) activities.

Methods

A cross-sectional survey was administered to researchers and policy-makers in Israel. The survey consisted of seven closed questions. Descriptive analyses were carried out for closed-ended questions and comparative analysis were conducted between groups using the χ² test.

Results

A total of 37 researchers and 32 policy-makers responded to the survey. While some views were in alignment, others showed differences. More policy-makers than researchers perceived that the use of HPSR in policy was hindered by practical implementation constraints, whereas more researchers felt that its use was hindered by a lack of coordination between knowledge producers and users. A larger percentage of policy-makers, as compared to researchers, reported that facilitators to the KTE process are in place and a larger percentage of researchers perceived barriers within the KTE environment. A larger percentage of policy-makers perceived KTE activities were in place as compared to researchers. Results also showed large differences in the perceptions of the two groups regarding policy formulation and which organisations they perceived as exerting strong influence on policy-making.

Conclusions

This research demonstrated that there are differences in the perceptions of knowledge producers and users about the process of KTE. Future work should focus on minimising the challenges highlighted here and implementing new KTE activities. These activities could include making the researchers aware of the most effective manner in which to package their results, providing training to policy-makers and assuring that policy-makers have technical access to appropriate databases to search for HPSR. These results underscore the need for the groups to communicate and clarify to each other what they can offer and what they require.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0345-6) contains supplementary material, which is available to authorized users.

Sustainability in Health care by Allocating Resources Effectively (SHARE) 8: developing, implementing and evaluating an evidence dissemination service in a local healthcare setting

BACKGROUND

This is the eighth in a series of papers reporting Sustainability in Health care by Allocating Resources Effectively (SHARE) in a local healthcare setting. The SHARE Program was a systematic, integrated, evidence-based program for disinvestment within a large Australian health service. One of the aims was to explore methods to deliver existing high quality synthesised evidence directly to decision-makers to drive decision-making proactively. An Evidence Dissemination Service (EDS) was proposed. While this was conceived as a method to identify disinvestment opportunities, it became clear that it could also be a way to review all practices for consistency with current evidence. This paper reports the development, implementation and evaluation of two models of an in-house EDS.

METHODS

Frameworks for development of complex interventions, implementation of evidence-based change, and evaluation and explication of processes and outcomes were adapted and/or applied. Mixed methods including a literature review, surveys, interviews, workshops, audits, document analysis and action research were used to capture barriers, enablers and local needs; identify effective strategies; develop and refine proposals; ascertain feedback and measure outcomes.

RESULTS

Methods to identify, capture, classify, store, repackage, disseminate and facilitate use of synthesised research evidence were investigated. In Model 1, emails containing links to multiple publications were sent to all self-selected participants who were asked to determine whether they were the relevant decision-maker for any of the topics presented, whether change was required, and to take the relevant action. This voluntary framework did not achieve the aim of ensuring practice was consistent with current evidence. In Model 2, the need for change was established prior to dissemination, then a summary of the evidence was sent to the decision-maker responsible for practice in the relevant area who was required to take appropriate action and report the outcome. This mandatory governance framework was successful. The factors influencing decisions, processes and outcomes were identified.

CONCLUSION

An in-house EDS holds promise as a method of identifying disinvestment opportunities and/or reviewing local practice for consistency with current evidence. The resource-intensive nature of delivery of the EDS is a potential barrier. The findings from this study will inform further exploration.

The effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare: a systematic review

BACKGROUND

It is widely acknowledged that health policy and management decisions rarely reflect research evidence. Therefore, it is important to determine how to improve evidence-informed decision-making. The primary aim of this systematic review was to evaluate the effectiveness of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare. The secondary aim of the review was to describe factors perceived to be associated with effective strategies and the inter-relationship between these factors.

METHODS

An electronic search was developed to identify studies published between January 01, 2000, and February 02, 2016. This was supplemented by checking the reference list of included articles, systematic reviews, and hand-searching publication lists from prominent authors. Two reviewers independently screened studies for inclusion, assessed methodological quality, and extracted data.

RESULTS

After duplicate removal, the search strategy identified 3830 titles. Following title and abstract screening, 96 full-text articles were reviewed, of which 19 studies (21 articles) met all inclusion criteria. Three studies were included in the narrative synthesis, finding policy briefs including expert opinion might affect intended actions, and intentions persisting to actions for public health policy in developing nations. Workshops, ongoing technical assistance, and distribution of instructional digital materials may improve knowledge and skills around evidence-informed decision-making in US public health departments. Tailored, targeted messages were more effective in increasing public health policies and programs in Canadian public health departments compared to messages and a knowledge broker. Sixteen studies (18 articles) were included in the thematic synthesis, leading to a conceptualisation of inter-relating factors perceived to be associated with effective research implementation strategies. A unidirectional, hierarchal flow was described from (1) establishing an imperative for practice change, (2) building trust between implementation stakeholders and (3) developing a shared vision, to (4) actioning change mechanisms. This was underpinned by the (5) employment of effective communication strategies and (6) provision of resources to support change.

CONCLUSIONS

Evidence is developing to support the use of research implementation strategies for promoting evidence-informed policy and management decisions in healthcare. The design of future implementation strategies should be based on the inter-relating factors perceived to be associated with effective strategies.

TRIAL REGISTRATION

This systematic review was registered with Prospero (record number: 42016032947).

Does access to a demand-led evidence briefing service improve uptake and use of research evidence by health service commissioners? A controlled before and after study

Background

The Health and Social Care Act mandated research use as a core consideration of health service commissioning arrangements in England. We undertook a controlled before and after study to evaluate whether access to a demand-led evidence briefing service improved the use of research evidence by commissioners compared with less intensive and less targeted alternatives.

Methods

Nine Clinical Commissioning Groups (CCGs) in the North of England received one of three interventions: (A) access to an evidence briefing service; (B) contact plus an unsolicited push of non-tailored evidence; or (C) unsolicited push of non-tailored evidence. Data for the primary outcome measure were collected at baseline and 12 months using a survey instrument devised to assess an organisations’ ability to acquire, assess, adapt and apply research evidence to support decision-making. Documentary and observational evidence of the use of the outputs of the service were sought.

Results

Over the course of the study, the service addressed 24 topics raised by participating CCGs. At 12 months, the evidence briefing service was not associated with increases in CCG capacity to acquire, assess, adapt and apply research evidence to support decision-making, individual intentions to use research findings or perceptions of CCG relationships with researchers. Regardless of intervention received, participating CCGs indicated that they remained inconsistent in their research-seeking behaviours and in their capacity to acquire research. The informal nature of decision-making processes meant that there was little traceability of the use of evidence. Low baseline and follow-up response rates and missing data limit the reliability of the findings.

Conclusions

Access to a demand-led evidence briefing service did not improve the uptake and use of research evidence by NHS commissioners compared with less intensive and less targeted alternatives. Commissioners appear well intentioned but ad hoc users of research. Further research is required on the effects of interventions and strategies to build individual and organisational capacity to use research.

Effects of a demand-led evidence briefing service on the uptake and use of research evidence by commissioners of health services: a controlled before-and-after study

Background

The Health and Social Care Act 2012 (Great Britain.Health and Social Care Act 2012. London: The Stationery Office; 2012) has mandated research use as a core consideration of health service commissioning arrangements. We evaluated whether or not access to a demand-led evidence briefing service improved the use of research evidence by commissioners, compared with less intensive and less targeted alternatives.

Design

Controlled before-and-after study.

Setting

Clinical Commissioning Groups (CCGs) in the north of England.

Main outcome measures

Change at 12 months from baseline of a CCG’s ability to acquire, assess, adapt and apply research evidence to support decision-making. Secondary outcomes measured individual clinical leads’ and managers’ intentions to use research evidence in decision-making.

Methods

Nine CCGs received one of three interventions: (1) access to an evidence briefing service; (2) contact plus an unsolicited push of non-tailored evidence; or (3) an unsolicited push of non-tailored evidence. Data for the primary outcome measure were collected at baseline and 12 months post intervention, using a survey instrument devised to assess an organisation’s ability to acquire, assess, adapt and apply research evidence to support decision-making. In addition, documentary and observational evidence of the use of the outputs of the service was sought and interviews with CCG participants were undertaken.

Results

Most of the requests were conceptual; they were not directly linked to discrete decisions or actions but were intended to provide knowledge about possible options for future actions. Symbolic use to justify existing decisions and actions were less frequent and included a decision to close a walk-in centre and to lend weight to a major initiative to promote self-care already under way. The opportunity to impact directly on decision-making processes was limited to work to establish disinvestment policies. In terms of impact overall, the evidence briefing service was not associated with increases in CCGs’ capacity to acquire, assess, adapt and apply research evidence to support decision-making, individual intentions to use research findings or perceptions of CCGs’ relationships with researchers. Regardless of the intervention received, at baseline participating CCGs indicated that they felt that they were inconsistent in their research-seeking behaviours and their capacity to acquire research remained so at follow-up. The informal nature of decision-making processes meant that there was little or no traceability of the use of evidence.

Limitations

Low baseline and follow-up response rates (of 68% and 44%, respectively) and missing data limit the reliability of these findings.

Conclusions

Access to a demand-led evidence briefing service did not improve the uptake and use of research evidence by NHS commissioners compared with less intensive and less targeted alternatives. Commissioners appear to be well intentioned but ad hoc users of research.

Future work

Further research is required on the effects of interventions and strategies to build individual and organisational capacity to use research. Resource-intensive approaches to providing evidence may best be employed to support instrumental decision-making. Comparative evaluation of the impact of less intensive but targeted strategies on the uptake and use of research by commissioners is warranted.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

PUBLIC AND PATIENT INVOLVEMENT IN HEALTH TECHNOLOGY ASSESSMENT: A FRAMEWORK FOR ACTION

OBJECTIVE

As health technology assessment (HTA) organizations in Canada and around the world seek to involve the public and patients in their activities, frameworks to guide decisions about whom to involve, through which mechanisms, and at what stages of the HTA process have been lacking. The aim of this study was to describe the development and outputs of a comprehensive framework for involving the public and patients in a government agency's HTA process.

METHODS

The framework was informed by a synthesis of international practice and published literature, a dialogue with local, national and international stakeholders, and the deliberations of a government agency's public engagement subcommittee in Ontario, Canada.

RESULTS

The practice and literature synthesis failed to identify a single, optimal approach to involving the public and patients in HTA. Choice of methods should be considered in the context of each HTA stage, goals for incorporating societal and/or patient perspectives into the process, and relevant societal and/or patient values at stake. The resulting framework is structured around four actionable elements: (i) guiding principles and goals for public and patient involvement (PPI) in HTA, (ii) the establishment of a common language to support PPI efforts, (iii) a flexible array of PPI approaches, and (iv) on-going evaluation of PPI to inform adjustments over time.

CONCLUSIONS

A public and patient involvement framework has been developed for implementation in a government agency's HTA process. Core elements of this framework may apply to other organizations responsible for HTA and health system quality improvement.

Designing a rapid response program to support evidence-informed decision-making in the Americas region: using the best available evidence and case studies

Background

The objective of this work was to inform the design of a rapid response program to support evidence-informed decision-making in health policy and practice for the Americas region. Specifically, we focus on the following: (1) What are the best methodological approaches for rapid reviews of the research evidence? (2) What other strategies are needed to facilitate evidence-informed decision-making in health policy and practice? and (3) How best to operationalize a rapid response program?

Methods

The evidence used to inform the design of a rapid response program included (i) two rapid reviews of methodological approaches for rapid reviews of the research evidence and strategies to facilitate evidence-informed decision-making, (ii) supplementary literature in relation to the “shortcuts” that could be considered to reduce the time needed to complete rapid reviews, (iii) four case studies, and (iv) supplementary literature to identify additional operational issues for the design of the program.

Results

There is no agreed definition of rapid reviews in the literature and no agreed methodology for conducting them. Better reporting of rapid review methods is needed. The literature found in relation to shortcuts will be helpful in choosing shortcuts that maximize timeliness while minimizing the impact on quality. Evidence for other strategies that can be used concurrently to facilitate the uptake of research evidence, including evidence drawn from rapid reviews, is presented. Operational issues that need to be considered in designing a rapid response program include the implications of a “user-pays” model, the importance of recruiting staff with the right mix of skills and qualifications, and ensuring that the impact of the model on research use in decision-making is formally evaluated.

Conclusions

When designing a new rapid response program, greater attention needs to be given to specifying the rapid review methods and reporting these in sufficient detail to allow a quality assessment. It will also be important to engage in other strategies to facilitate the uptake of the rapid reviews and to evaluate the chosen model in order to make refinements and add to the evidence base for evidence-informed decision-making.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0472-9) contains supplementary material, which is available to authorized users.

Systematic reviews on child welfare services: identifying and disseminating the evidence

RATIONALE, AIMS AND OBJECTIVES

Evidence-based practice is at an early stage of uptake within child welfare services. To facilitate well-informed decisions, we disseminated evidence from systematic reviews (SR) to local child welfare stakeholders in Norway through plain language summaries on a website (http://www.r-bup.no).

METHOD

We developed and implemented our dissemination strategy through seven steps: (1) systematic literature search; (2) selection of relevant SRs; (3) assembly of an advisory board; (4) selection of child welfare SRs relevant to Norway; (5) prioritization of the included SRs; (6) development of a plain language summary (PLS) after feedback from the advisory board; and (7) implementation of website.

RESULTS

A total of 9266 potentially relevant records were screened and 120 SRs were included. The advisory board was assembled from local policymakers, practitioners, researchers, carers and consumers. The advisory board members independently ranked the 120 SRs according to relevance and prioritized 20 SRs that were written up into the PLS. The format of the PLS was tested and agreed with the board members. A website was developed and the PLSs were published starting September 2014.

CONCLUSION

We think that the PLSs will be valuable resources to practitioners and it will be easily accessible to caregivers and consumers. This knowledge will inform research priorities and practice in Norway, leading the way to the use of evidence-based decisions in local child welfare services.

Evidence based policy making and the 'art' of commissioning - how English healthcare commissioners access and use information and academic research in 'real life' decision-making: an empirical qualitative study

BACKGROUND

Policymakers such as English healthcare commissioners are encouraged to adopt 'evidence-based policy-making', with 'evidence' defined by researchers as academic research. To learn how academic research can influence policy, researchers need to know more about commissioning, commissioners' information seeking behaviour and the role of research in their decisions.

METHODS

In case studies of four commissioning organisations, we interviewed 52 people including clinical and managerial commissioners, observed 14 commissioning meetings and collected documentation e.g. meeting minutes and reports. Using constant comparison, data were coded, summarised and analysed to facilitate cross case comparison.

RESULTS

The 'art of commissioning' entails juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case. Policymakers sought information to identify options, navigate ways through, justify decisions and convince others to approve and/or follow the suggested course. 'Evidence-based policy-making' usually meant pragmatic selection of 'evidence' such as best practice guidance, clinicians' and users' views of services and innovations from elsewhere. Inconclusive or negative research was unhelpful in developing policymaking plans and did not inform disinvestment decisions. Information was exchanged through conversations and stories, which were fast, flexible and suited the rapidly changing world of policymaking. Local data often trumped national or research-based evidence. Local evaluations were more useful than academic research.

DISCUSSION

Commissioners are highly pragmatic and will only use information that helps them create a compelling case for action.Therefore, researchers need to start producing more useful information.

CONCLUSIONS

To influence policymakers' decisions, researchers need to 1) learn more about local policymakers' priorities 2) develop relationships of mutual benefit 3) use verbal instead of writtencommunication 4) work with intermediaries such as public health consultants and 5) co-produce local evaluations.

Mobilising knowledge to improve UK health care: learning from other countries and other sectors – a multimethod mapping study

Background

The past two decades have seen rich conceptual development and a wide variety of practical initiatives around research use or ‘knowledge mobilisation’, but so far there has been little systematic effort to map, conceptualise and learn from these initiatives, or to investigate the degree to which they are underpinned by contemporary thinking as set out in the literature. This gap is particularly apparent when looking at knowledge mobilisation at the ‘macro’ level, that is the strategies and activities of major research funders, major research producers and key research ‘intermediaries’.

Aims and objectives

The study had three key objectives with associated research questions: to map the knowledge mobilisation landscape in health care (in the UK and internationally) and in social care and education within the UK; to understand the models, theories and frameworks that underpin the approaches to knowledge mobilisation; and to learn from the success or otherwise of the strategies and approaches in use.

Methods

The study was multimethod and multiphased, with considerable interactivity between the different strands. Data were collected through a review of 71 published reviews on knowledge mobilisation; website review of the knowledge mobilisation activities of 186 agencies; in-depth interviews (n = 52) with key individuals in agencies; a web survey (response rate 57%;n = 106); and two stakeholder workshops (at months 6 and 16).

Findings

We identified a wide range of models, theories and frameworks used to describe knowledge mobilisation and created a conceptual map that highlights six domains of thinking and debate in the literature. The interview and survey data showed three broad, overlapping roles undertaken by agencies: developing and sharing research-based products; emphasising brokering; and focusing on implementation. The knowledge mobilisation approaches in use had been shaped by many factors but there was only limited use of the models, theories and frameworks from the literature. Participants saw formal evaluation of knowledge mobilisation activities as important but highly challenging. Rich formative experience was described but formal evaluation was relatively rare. Few agencies involved service users or members of the public in knowledge mobilisation activities. Working inductively from the study data we derived eight key archetypes or ‘bundles of knowledge mobilisation activities’ that could be used by agencies to explore their knowledge mobilisation activities, future strategies and stakeholder perspectives.

Conclusions

Knowledge mobilisation could be enhanced by providing support to enable cross-sector and interagency learning, reflection on the conceptual basis of approaches and increased evaluation of knowledge mobilisation activities. Further research is needed to evaluate approaches to assessing research use and impact, on systems approaches to knowledge mobilisation, on sustaining and scaling-up approaches, and on applying a wider range of literatures to knowledge mobilisation. Further research would also be useful on the knowledge mobilisation archetypes and how they can work in complementary ways.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Effects of a demand-led evidence briefing service on the uptake and use of research evidence by commissioners of health services: protocol for a controlled before and after study

BACKGROUND

Clinical Commissioning Groups (CCGs) are mandated to use research evidence effectively to ensure optimum use of resources by the National Health Service (NHS), both in accelerating innovation and in stopping the use of less effective practices and models of service delivery. We intend to evaluate whether access to a demand-led evidence service improves uptake and use of research evidence by NHS commissioners compared with less intensive and less targeted alternatives.

METHODS/DESIGN

This is a controlled before and after study involving CCGs in the North of England. Participating CCGs will receive one of three interventions to support the use of research evidence in their decision-making: 1) consulting plus responsive push of tailored evidence; 2) consulting plus an unsolicited push of non-tailored evidence; or 3) standard service unsolicited push of non-tailored evidence. Our primary outcome will be changed at 12 months from baseline of a CCGs ability to acquire, assess, adapt and apply research evidence to support decision-making. Secondary outcomes will measure individual clinical leads and managers' intentions to use research evidence in decision making. Documentary evidence of the use of the outputs of the service will be sought. A process evaluation will evaluate the nature and success of the interactions both within the sites and between commissioners and researchers delivering the service.

DISCUSSION

The proposed research will generate new knowledge of direct relevance and value to the NHS. The findings will help to clarify which elements of the service are of value in promoting the use of research evidence. Those involved in NHS commissioning will be able to use the results to inform how best to build the infrastructure they need to acquire, assess, adapt and apply research evidence to support decision-making and to fulfil their statutory duties under the Health and Social Care Act.

What guidance are researchers given on how to present network meta-analyses to end-users such as policymakers and clinicians? A systematic review

INTRODUCTION

Network meta-analyses (NMAs) are complex methodological approaches that may be challenging for non-technical end-users, such as policymakers and clinicians, to understand. Consideration should be given to identifying optimal approaches to presenting NMAs that help clarify analyses. It is unclear what guidance researchers currently have on how to present and tailor NMAs to different end-users.

METHODS

A systematic review of NMA guidelines was conducted to identify guidance on how to present NMAs. Electronic databases and supplementary sources were searched for NMA guidelines. Presentation format details related to sample formats, target audiences, data sources, analysis methods and results were extracted and frequencies tabulated. Guideline quality was assessed following criteria developed for clinical practice guidelines.

RESULTS

Seven guidelines were included. Current guidelines focus on how to conduct NMAs but provide limited guidance to researchers on how to best present analyses to different end-users. None of the guidelines provided reporting templates. Few guidelines provided advice on tailoring presentations to different end-users, such as policymakers. Available guidance on presentation formats focused on evidence networks, characteristics of individual trials, comparisons between direct and indirect estimates and assumptions of heterogeneity and/or inconsistency. Some guidelines also provided examples of figures and tables that could be used to present information.

CONCLUSIONS

Limited guidance exists for researchers on how best to present NMAs in an accessible format, especially for non-technical end-users such as policymakers and clinicians. NMA guidelines may require further integration with end-users' needs, when NMAs are used to support healthcare policy and practice decisions. Developing presentation formats that enhance understanding and accessibility of NMAs could also enhance the transparency and legitimacy of decisions informed by NMAs.

Foundation of evidence-based decision making for health care managers-part II: meta-analysis and applying the evidence

This article is a continuation of a 2-part series that seeks to build the skills of health care managers and supervisors in critically reading and interpreting systematic reviews and meta-analyses. The first article, part I, defined the types of systematic reviews and outlined the process of conducting them. This article, part II, focuses on meta-analysis, a type of systematic review. This article explains the additional procedures associated with meta-analyses and describes the potential shortcomings of both systematic reviews and meta-analyses. Finally, this article provides a way for health care managers and supervisors to appraise systematic reviews and meta-analyses and to determine whether the evidence applies to their problems and settings.

Towards a better understanding of the nomenclature used in information-packaging efforts to support evidence-informed policymaking in low- and middle-income countries

Background

The growing recognition of the importance of concisely communicating research evidence and other policy-relevant information to policymakers has underpinned the development of several information-packaging efforts over the past decade. This has led to a wide variability in the types of documents produced, which is at best confusing and at worst discouraging for those they intend to reach. This paper has two main objectives: to develop a better understanding of the range of documents and document names used by the organizations preparing them; and to assess whether there are any consistencies in the characteristics of sampled documents across the names employed to label (in the title) or describe (in the document or website) them.

Methods

We undertook a documentary analysis of web-published document series that are prepared by a variety of organizations with the primary intention of providing information to health systems policymakers and stakeholders, and addressing questions related to health policy and health systems with a focus on low- and middle-income countries. No time limit was set.

Results

In total, 109 individual documents from 24 series produced by 16 different organizations were included. The name ‘policy brief/briefing’ was the most frequently used (39%) to label or describe a document, and was used in all eight broad content areas that we identified, even though they did not have obviously common traits among them. In terms of document characteristics, most documents (90%) used skimmable formats that are easy to read, with understandable, jargon-free, language (80%). Availability of information on the methods (47%) or the quality of the presented evidence (27%) was less common. One-third (32%) chose the topic based on an explicit process to assess the demand for information from policy makers and even fewer (19%) engaged with policymakers to discuss the content of these documents such as through merit review.

Conclusions

This study highlights the need for organizations embarking on future information-packaging efforts to be more thoughtful when deciding how to name these documents and the need for greater transparency in describing their content, purpose and intended audience.

An overview and methodological assessment of systematic reviews and meta-analyses of enhanced recovery programmes in colorectal surgery

OBJECTIVES

To identify and critically assess the extent to which systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery differ in their methodology and reported estimates of effect.

DESIGN

Review of published systematic reviews. We searched the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effects (DARE) and Health Technology Assessment (HTA) Database from 1990 to March 2013. Systematic reviews of enhanced recovery programmes for patients undergoing colorectal surgery were eligible for inclusion.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome was length of hospital stay. We assessed changes in pooled estimates of treatment effect over time and how these might have been influenced by decisions taken by researchers as well as by the availability of new trials. The quality of systematic reviews was assessed using the Centre for Reviews and Dissemination (CRD) DARE critical appraisal process.

RESULTS

10 systematic reviews were included. Systematic reviews of randomised controlled trials have consistently shown a reduction in length of hospital stay with enhanced recovery compared with traditional care. The estimated effect tended to increase from 2006 to 2010 as more trials were published but has not altered significantly in the most recent review, despite the inclusion of several unique trials. The best estimate appears to be an average reduction of around 2.5 days in primary postoperative length of stay. Differences between reviews reflected differences in interpretation of inclusion criteria, searching and analytical methods or software.

CONCLUSIONS

Systematic reviews of enhanced recovery programmes show a high level of research waste, with multiple reviews covering identical or very similar groups of trials. Where multiple reviews exist on a topic, interpretation may require careful attention to apparently minor differences between reviews. Researchers can help readers by acknowledging existing reviews and through clear reporting of key decisions, especially on inclusion/exclusion and on statistical pooling.

How contexts and issues influence the use of policy-relevant research syntheses: a critical interpretive synthesis

CONTEXT

Evidence briefs have emerged as a promising approach to synthesizing the best available research evidence for health system policymakers and stakeholders. An evidence brief may draw on systematic reviews and many other types of policy-relevant information, including local data and studies, to describe a problem, options for addressing it, and key implementation considerations. We conducted a systematic review to examine the ways in which context- and issue-related factors influence the perceived usefulness of evidence briefs among their intended users.

METHODS

We used a critical interpretive synthesis approach to review both empirical and nonempirical literature and to develop a model that explains how context and issues influence policymakers' and stakeholders' views of the utility of evidence briefs prepared for priority policy issues. We used a "compass" question to create a detailed search strategy and conducted electronic searches in CINAHL, EMBASE, HealthSTAR, IPSA, MEDLINE, OAIster (gray literature), ProQuest A&I Theses, ProQuest (Sociological Abstracts, Applied Social Sciences Index and Abstracts, Worldwide Political Science Abstracts, International Bibliography of Social Sciences, PAIS, Political Science), PsychInfo, Web of Science, and WilsonWeb (Social Science Abstracts). Finally, we used a grounded and interpretive analytic approach to synthesize the results.

FINDINGS

Of the 4,461 papers retrieved, 3,908 were excluded and 553 were assessed for "relevance," with 137 included in the initial sample of papers to be analyzed and an additional 23 purposively sampled to fill conceptual gaps. Several themes emerged: (1) many established types of "evidence" are viewed as useful content in an evidence brief, along with several promising formatting features; (2) contextual factors, particularly the institutions, interests, and values of a given context, can influence views of evidence briefs; (3) whether an issue is polarizing and whether it is salient (or not) and familiar (or not) to actors in the policy arena can influence views of evidence briefs prepared for that issue; (4) influential factors can emerge in several ways (as context driven, issue driven, or a result of issue-context resonance); (5) these factors work through two primary pathways, affecting either the users or the producers of briefs; and (6) these factors influence views of evidence briefs through a variety of mechanisms.

CONCLUSIONS

Those persons funding and preparing evidence briefs need to consider a variety of context- and issue-related factors when deciding how to make them most useful in policymaking.

A KT intervention including the evidence alert system to improve clinician's evidence-based practice behavior--a cluster randomized controlled trial

Background

It is difficult to foster research utilization among allied health professionals (AHPs). Tailored, multifaceted knowledge translation (KT) strategies are now recommended but are resource intensive to implement. Employers need effective KT solutions but little is known about; the impact and viability of multifaceted KT strategies using an online KT tool, their effectiveness with AHPs and their effect on evidence-based practice (EBP) decision-making behavior. The study aim was to measure the effectiveness of a multifaceted KT intervention including a customized KT tool, to change EBP behavior, knowledge, and attitudes of AHPs.

Methods

This is an evaluator-blinded, cluster randomized controlled trial conducted in an Australian community-based cerebral palsy service. 135 AHPs (physiotherapists, occupational therapists, speech pathologists, psychologists and social workers) from four regions were cluster randomized (n = 4), to either the KT intervention group (n = 73 AHPs) or the control group (n = 62 AHPs), using computer-generated random numbers, concealed in opaque envelopes, by an independent officer. The KT intervention included three-day skills training workshop and multifaceted workplace supports to redress barriers (paid EBP time, mentoring, system changes and access to an online research synthesis tool). Primary outcome (self- and peer-rated EBP behavior) was measured using the Goal Attainment Scale (individual level). Secondary outcomes (knowledge and attitudes) were measured using exams and the Evidence Based Practice Attitude Scale.

Results

The intervention group’s primary outcome scores improved relative to the control group, however when clustering was taken into account, the findings were non-significant: self-rated EBP behavior [effect size 4.97 (95% CI -10.47, 20.41) (p = 0.52)]; peer-rated EBP behavior [effect size 5.86 (95% CI -17.77, 29.50) (p = 0.62)]. Statistically significant improvements in EBP knowledge were detected [effect size 2.97 (95% CI 1.97, 3.97 (p < 0.0001)]. Change in EBP attitudes was not statistically significant.

Conclusions

Improvement in EBP behavior was not statistically significant after adjusting for cluster effect, however similar improvements from peer-ratings suggest behaviorally meaningful gains. The large variability in behavior observed between clusters suggests barrier assessments and subsequent KT interventions may need to target subgroups within an organization.

Trial registration

Registered on the Australian New Zealand Clinical Trials Registry (ACTRN12611000529943).

Balancing exploration and exploitation in transferring research into practice: a comparison of five knowledge translation entity archetypes

BACKGROUND

Translating knowledge from research into clinical practice has emerged as a practice of increasing importance. This has led to the creation of new organizational entities designed to bridge knowledge between research and practice. Within the UK, the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) have been introduced to ensure that emphasis is placed in ensuring research is more effectively translated and implemented in clinical practice. Knowledge translation (KT) can be accomplished in various ways and is affected by the structures, activities, and coordination practices of organizations. We draw on concepts in the innovation literature--namely exploration, exploitation, and ambidexterity--to examine these structures and activities as well as the ensuing tensions between research and implementation.

METHODS

Using a qualitative research approach, the study was based on 106 semi-structured, in-depth interviews with the directors, theme leads and managers, key professionals involved in research and implementation in nine CLAHRCs. Data was also collected from intensive focus group workshops.

RESULTS

In this article we develop five archetypes for organizing KT. The results show how the various CLAHRC entities work through partnerships to create explorative research and deliver exploitative implementation. The different archetypes highlight a range of structures that can achieve ambidextrous balance as they organize activity and coordinate practice on a continuum of exploration and exploitation.

CONCLUSION

This work suggests that KT entities aim to reach their goals through a balance between exploration and exploitation in the support of generating new research and ensuring knowledge implementation. We highlight different organizational archetypes that support various ways to maintain ambidexterity, where both exploration and exploitation are supported in an attempt to narrow the knowledge gaps. The KT entity archetypes offer insights on strategies in structuring collaboration to facilitate an effective balance of exploration and exploitation learning in the KT process.

Assessing availability of scientific journals, databases, and health library services in Canadian health ministries: a cross-sectional study

BACKGROUND

Evidence-informed health policymaking logically depends on timely access to research evidence. To our knowledge, despite the substantial political and societal pressure to enhance the use of the best available research evidence in public health policy and program decision making, there is no study addressing availability of peer-reviewed research in Canadian health ministries.

OBJECTIVES

To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic databases, and (3) health library services in the fourteen Canadian health ministries.

METHODS

From May to October 2011, we conducted a cross-sectional survey among librarians employed by Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria were: relevance for health policy based on scope note information about subject categories and journal popularity based on impact factors.

RESULTS

We found that the majority of Canadian health ministries did not have subscription access to key journals and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial ministries' average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary databases. Many organisations provided access to the library resources described through library partnerships or consortia. No professionally led health library environment was found in four out of fourteen Canadian health ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut Department of Health and Social Services and Yukon Department of Health and Social Services).

CONCLUSIONS

There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. This inequity could present a problem, as each province and territory is responsible for formulating and implementing evidence-informed health policies and services for the benefit of its population.