Interventions encouraging the use of systematic reviews in clinical decision-making: a systematic review

Implementation strategies for knowledge products in primary healthcare: a systematic review of systematic reviews (Preprint)

Background

The underuse or overuse of knowledge products leads to waste in health care, and primary care is no exception.

Objective

This study aimed to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes that are measured.

Methods

We performed a systematic review (SR) of SRs using the Cochrane systematic approach to include eligible SRs. The inclusion criteria were any primary care contexts, health care professionals and patients, any Effective Practice and Organization of Care implementation strategies of specified knowledge products, any comparators, and any implementation outcomes based on the Proctor framework. We searched the MEDLINE, EMBASE, CINAHL, Ovid PsycINFO, Web of Science, and Cochrane Library databases from their inception to October 2019 without any restrictions. We searched the references of the included SRs. Pairs of reviewers independently performed selection, data extraction, and methodological quality assessment by using A Measurement Tool to Assess Systematic Reviews 2. Data extraction was informed by the Effective Practice and Organization of Care taxonomy for implementation strategies and the Proctor framework for implementation outcomes. We performed a descriptive analysis and summarized the results by using a narrative synthesis.

Results

Of the 11,101 records identified, 81 (0.73%) SRs were included. Of these 81, a total of 47 (58%) SRs involved health care professionals alone. Moreover, 15 SRs had a high or moderate methodological quality. Most of them addressed 1 type of knowledge product (56/81, 69%), common clinical practice guidelines (26/56, 46%) or management, and behavioral or pharmacological health interventions (24/56, 43%). Mixed strategies were used for implementation (67/81, 83%), predominantly education-based (meetings in 60/81, 74%; materials distribution in 59/81, 73%; and academic detailing in 45/81, 56%), reminder (53/81, 36%), and audit and feedback (40/81, 49%) strategies. Education meetings (P=.13) and academic detailing (P=.11) seemed to be used more when the population was composed of health care professionals alone. Improvements in the adoption of knowledge products were the most commonly measured outcome (72/81, 89%). The evidence level was reported in 12% (10/81) of SRs on 62 outcomes (including 48 improvements in adoption), of which 16 (26%) outcomes were of moderate or high level.

Conclusions

Clinical practice guidelines and management and behavioral or pharmacological health interventions are the most commonly implemented knowledge products and are implemented through the mixed use of educational, reminder, and audit and feedback strategies. There is a need for a strong methodology for the SR of randomized controlled trials to explore their effectiveness and the entire cascade of implementation outcomes.

Green-synthesized metal nanoparticles for mosquito control: A systematic review about their toxicity on non-target organisms

Studies capturing the high efficiency of green-synthesized metal nanoparticles (NPs) in targeting mosquito vectors of the world's main infectious diseases suggest the NPs' possible utilization as bio-insecticides. However, it is necessary to confirm that these potential bio-insecticides are not harmful to non-target organisms that are often sympatric and natural enemies of the vectors of these diseases. In this systematic review, we comprehensively analyse the content of 56 publications focused on the potentially deleterious effects of NPs on these non-target organisms. Current research on biosynthesised NPs, characterization, and impact on mosquito vectors and non-target larvivorous organisms is reviewed and critically discussed. Finally, we pinpoint some major challenges that merit future investigation. Plants (87.5%) were mainly used for synthesizing NPs in the studies. NPs were found to be spherical or mainly spherical in shape with a large distribution size. In most of the included studies, NPs showed interesting mosquitocidal activity (LC₅₀ < 50 ppm). Some plant families (e.g., Meliaceae, Poaceae, Lamiaceae) have produced NPs with a particularly high larvicidal and pupicidal activity (LC₅₀ < 10 ppm). Regarding non-target organisms, most of the studies concluded that NPs were safe to them, with boosted predatory activity in NP-treated milieu. In contrast, some studies reported NP-elicited adverse effects (i.e., genotoxic, nuclear, and enzymatic effects) on these non-target organisms. This review outlines the promising mosquitocidal effects of biosynthesized NPs, recognizing that NPs' potential usage is currently limited by the harm NPs are thought pose to non-target organism. It is of utmost importance to investigate green NPs to determine whether laboratory findings have applications in the real world.

Publication and related bias in quantitative health services and delivery research: a multimethod study

Background

Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research.

Objectives

To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias.

Methods

The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8).

Results

We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5.

Conclusions

This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required.

Study registration

This study is registered as PROSPERO CRD42016052333 and CRD42016052366.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 33. See the NIHR Journals Library website for further project information.

Evidence summary resources may influence clinical decision making: A case-based scenario evaluation of an evidence summary tool

PURPOSE

Evidence summary resources are popular with clinicians but it is unknown whether they can influence clinical decision making. We evaluated whether an extremely condensed and explicit evidence summary tool could influence clinical decision making.

MATERIALS AND METHODS

An evidence summary tool was developed using a formal mapping exercise and graphic design principles. An invitation to participate was sent to subscribers of a critical care e-mail discussion list. Participants received a study package (evidence summary tool précising prone positioning in severe ARDS; case-based scenario describing a patient with severe ARDS plus evaluation questionnaire). Influence on clinical decisions was captured regarding six competing interventions, with Belief in benefit measured before and after reading the summary tool.

RESULTS

Among 93 participants, 87% were male with a mean age of 49.6(SD9.3) years. Mean ICU experience was 20.0(SD9.9) years. The evidence summary tool significantly influenced clinical decision making: belief in benefit of prone positioning increased (P < .001), belief in benefit of higher PEEP decreased (P = .002) and belief in benefit in ECMO decreased (P = .07).

CONCLUSIONS

Using a before-after evaluation, we demonstrated an extremely condensed and explicit information format can influence clinical decision making. Evidence summary tools may be a useful adjunct to support closing evidence-practice gaps.

Implementation science in resource-poor countries and communities

Background

Implementation science in resource-poor countries and communities is arguably more important than implementation science in resource-rich settings, because resource poverty requires novel solutions to ensure that research results are translated into routine practice and benefit the largest possible number of people.

Methods

We reviewed the role of resources in the extant implementation science frameworks and literature. We analyzed opportunities for implementation science in resource-poor countries and communities, as well as threats to the realization of these opportunities.

Results

Many of the frameworks that provide theoretical guidance for implementation science view resources as contextual factors that are important to (i) predict the feasibility of implementation of research results in routine practice, (ii) explain implementation success and failure, (iii) adapt novel evidence-based practices to local constraints, and (iv) design the implementation process to account for local constraints. Implementation science for resource-poor settings shifts this view from “resources as context” to “resources as primary research object.” We find a growing body of implementation research aiming to discover and test novel approaches to generate resources for the delivery of evidence-based practice in routine care, including approaches to create higher-skilled health workers—through tele-education and telemedicine, freeing up higher-skilled health workers—through task-shifting and new technologies and models of care, and increasing laboratory capacity through new technologies and the availability of medicines through supply chain innovations. In contrast, only few studies have investigated approaches to change the behavior and utilization of healthcare resources in resource-poor settings. We identify three specific opportunities for implementation science in resource-poor settings. First, intervention and methods innovations thrive under constraints. Second, reverse innovation transferring novel approaches from resource-poor to research-rich settings will gain in importance. Third, policy makers in resource-poor countries tend to be open for close collaboration with scientists in implementation research projects aimed at informing national and local policy.

Conclusions

Implementation science in resource-poor countries and communities offers important opportunities for future discoveries and reverse innovation. To harness this potential, funders need to strongly support research projects in resource-poor settings, as well as the training of the next generation of implementation scientists working on new ways to create healthcare resources where they lack most and to ensure that those resources are utilized to deliver care that is based on the latest research results.

Do evidence summaries increase health policy-makers' use of evidence from systematic reviews? A systematic review

This review summarizes the evidence from six randomized controlled trials that judged the effectiveness of systematic review summaries on policymakers' decision making, or the most effective ways to present evidence summaries to increase policymakers' use of the evidence. This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one. The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary. None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?" Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.

Plain language summary

Policy briefs make systematic reviews easier to understand but little evidence of impact on use of study findings: It is likely that evidence summaries are easier to understand than complete systematic reviews. Whether these summaries increase the use of evidence from systematic reviews in policymaking is not clear.What is this review about?: Systematic reviews are long and technical documents that may be hard for policymakers to use when making decisions. Evidence summaries are short documents that describe research findings in systematic reviews. These summaries may simplify the use of systematic reviews.Other names for evidence reviews are policy briefs, evidence briefs, summaries of findings, or plain language summaries. The goal of this review was to learn whether evidence summaries help policymakers use evidence from systematic reviews. This review also aimed to identify the best ways to present the evidence summary to increase the use of evidence.What are the main findings of this review?: This review included six randomized controlled studies. A randomized controlled study is one in which the participants are divided randomly (by chance) into separate groups to compare different treatments or other interventions. This method of dividing people into groups means that the groups will be similar and that the effects of the treatments they receive will be compared more fairly. At the time the study is done, it is not known which treatment is the better one.The researchers who did these studies invited people from Europe, North America, South America, Africa, and Asia to take part in them. Two studies looked at "policy briefs," one study looked at an "evidence summary," two looked at a "summary of findings table," and one compared a "summary of findings table" to an evidence summary.None of these studies looked at how policymakers directly used evidence from systematic reviews in their decision making, but two studies found that there was little to no difference in how they used the summaries. The studies relied on reports from decision makers. These studies included questions such as, "Is this summary easy to understand?"Some of the studies looked at users' knowledge, understanding, beliefs, or how credible (trustworthy) they believed the summaries to be. There was little to no difference in the studies that looked at these outcomes. Study participants rated the graded entry format higher for usability than the full systematic review. The graded entry format allows the reader to select how much information they want to read.. The study participants felt that all evidence summary formats were easier to understand than full systematic reviews.What do the findings of this review mean?: Our review suggests that evidence summaries help policymakers to better understand the findings presented in systematic reviews. In short, evidence summaries should be developed to make it easier for policymakers to understand the evidence presented in systematic reviews. However, right now there is very little evidence on the best way to present systematic review evidence to policymakers.How up to date is this review?: The authors of this review searched for studies through June 2016.

Executive summary/Abstract

Background: Systematic reviews are important for decision makers. They offer many potential benefits but are often written in technical language, are too long, and do not contain contextual details which makes them hard to use for decision-making. Strategies to promote the use of evidence to decision makers are required, and evidence summaries have been suggested as a facilitator. Evidence summaries include policy briefs, briefing papers, briefing notes, evidence briefs, abstracts, summary of findings tables, and plain language summaries. There are many organizations developing and disseminating systematic review evidence summaries for different populations or subsets of decision makers. However, evidence on the usefulness and effectiveness of systematic review summaries is lacking. We present an overview of the available evidence on systematic review evidence summaries.Objectives: This systematic review aimed to 1) assess the effectiveness of evidence summaries on policy-makers' use of the evidence and 2) identify the most effective summary components for increasing policy-makers' use of the evidence.Search methods: We searched several online databases (Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, Global Health Library, Popline, Africa-wide, Public Affairs Information Services, Worldwide Political Science Abstracts, Web of Science, and DfiD), websites of research groups and organizations which produce evidence summaries, and reference lists of included summaries and related systematic reviews. These databases were searched in March-April, 2016.Selection criteria: Eligible studies included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-after (CBA) studies, and interrupted time series (ITS) studies. We included studies of policymakers at all levels as well as health system managers. We included studies examining any type of "evidence summary", "policy brief", or other product derived from systematic reviews that presented evidence in a summarized form. These interventions could be compared to active comparators (e.g. other summary formats) or no intervention.The primary outcomes were: 1) use of systematic review summaries decision-making (e.g. self-reported use of the evidence in policy-making, decision-making) and 2) policymaker understanding, knowledge, and/or beliefs (e.g. changes in knowledge scores about the topic included in the summary). We also assessed perceived relevance, credibility, usefulness, understandability, and desirability (e.g. format) of the summaries.Results: Our database search combined with our grey literature search yielded 10,113 references after removal of duplicates. From these, 54 were reviewed in full text and we included 6 studies (reported in 7 papers, 1661 participants) as well as protocols from 2 ongoing studies. Two studies assessed the use of evidence summaries in decision-making and found little to no difference in effect. There was also little to no difference in effect for knowledge, understanding or beliefs (4 studies) and perceived usefulness or usability (3 studies). Summary of Findings tables and graded entry summaries were perceived as slightly easier to understand compared to complete systematic reviews. Two studies assessed formatting changes and found that for Summary of Findings tables, certain elements, such as reporting study event rates and absolute differences were preferred as well as avoiding the use of footnotes. No studies assessed adverse effects. The risks of bias in these studies were mainly assessed as unclear or low however, two studies were assessed as high risk of bias for incomplete outcome data due to very high rates of attrition.Authors' conclusions: Evidence summaries may be easier to understand than complete systematic reviews. However, their ability to increase the use of systematic review evidence in policymaking is unclear.

Implementation strategies for health systems in low-income countries: an overview of systematic reviews

BACKGROUND

A key function of health systems is implementing interventions to improve health, but coverage of essential health interventions remains low in low-income countries. Implementing interventions can be challenging, particularly if it entails complex changes in clinical routines; in collaborative patterns among different healthcare providers and disciplines; in the behaviour of providers, patients or other stakeholders; or in the organisation of care. Decision-makers may use a range of strategies to implement health interventions, and these choices should be based on evidence of the strategies' effectiveness.

OBJECTIVES

To provide an overview of the available evidence from up-to-date systematic reviews about the effects of implementation strategies for health systems in low-income countries. Secondary objectives include identifying needs and priorities for future evaluations and systematic reviews on alternative implementation strategies and informing refinements of the framework for implementation strategies presented in the overview.

METHODS

We searched Health Systems Evidence in November 2010 and PDQ-Evidence up to December 2016 for systematic reviews. We did not apply any date, language or publication status limitations in the searches. We included well-conducted systematic reviews of studies that assessed the effects of implementation strategies on professional practice and patient outcomes and that were published after April 2005. We excluded reviews with limitations important enough to compromise the reliability of the review findings. Two overview authors independently screened reviews, extracted data and assessed the certainty of evidence using GRADE. We prepared SUPPORT Summaries for eligible reviews, including key messages, 'Summary of findings' tables (using GRADE to assess the certainty of the evidence) and assessments of the relevance of findings to low-income countries.

MAIN RESULTS

We identified 7272 systematic reviews and included 39 of them in this overview. An additional four reviews provided supplementary information. Of the 39 reviews, 32 had only minor limitations and 7 had important methodological limitations. Most studies in the reviews were from high-income countries. There were no studies from low-income countries in eight reviews.Implementation strategies addressed in the reviews were grouped into four categories - strategies targeting:1. healthcare organisations (e.g. strategies to change organisational culture; 1 review);2. healthcare workers by type of intervention (e.g. printed educational materials; 14 reviews);3. healthcare workers to address a specific problem (e.g. unnecessary antibiotic prescription; 9 reviews);4. healthcare recipients (e.g. medication adherence; 15 reviews).Overall, we found the following interventions to have desirable effects on at least one outcome with moderate- or high-certainty evidence and no moderate- or high-certainty evidence of undesirable effects.1.Strategies targeted at healthcare workers: educational meetings, nutrition training of health workers, educational outreach, practice facilitation, local opinion leaders, audit and feedback, and tailored interventions.2.Strategies targeted at healthcare workers for specific types of problems: training healthcare workers to be more patient-centred in clinical consultations, use of birth kits, strategies such as clinician education and patient education to reduce antibiotic prescribing in ambulatory care settings, and in-service neonatal emergency care training.3. Strategies targeted at healthcare recipients: mass media interventions to increase uptake of HIV testing; intensive self-management and adherence, intensive disease management programmes to improve health literacy; behavioural interventions and mobile phone text messages for adherence to antiretroviral therapy; a one time incentive to start or continue tuberculosis prophylaxis; default reminders for patients being treated for active tuberculosis; use of sectioned polythene bags for adherence to malaria medication; community-based health education, and reminders and recall strategies to increase vaccination uptake; interventions to increase uptake of cervical screening (invitations, education, counselling, access to health promotion nurse and intensive recruitment); health insurance information and application support.

AUTHORS' CONCLUSIONS

Reliable systematic reviews have evaluated a wide range of strategies for implementing evidence-based interventions in low-income countries. Most of the available evidence is focused on strategies targeted at healthcare workers and healthcare recipients and relates to process-based outcomes. Evidence of the effects of strategies targeting healthcare organisations is scarce.

Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review

BACKGROUND

We completed a scoping review on the barriers and facilitators to use of systematic reviews by health care managers and policy makers, including consideration of format and content, to develop recommendations for systematic review authors and to inform research efforts to develop and test formats for systematic reviews that may optimise their uptake.

METHODS

We used the Arksey and O'Malley approach for our scoping review. Electronic databases (e.g., MEDLINE, EMBASE, PsycInfo) were searched from inception until September 2014. Any study that identified barriers or facilitators (including format and content features) to uptake of systematic reviews by health care managers and policy makers/analysts was eligible for inclusion. Two reviewers independently screened the literature results and abstracted data from the relevant studies. The identified barriers and facilitators were charted using a barriers and facilitators taxonomy for implementing clinical practice guidelines by clinicians.

RESULTS

We identified useful information for authors of systematic reviews to inform their preparation of reviews including providing one-page summaries with key messages, tailored to the relevant audience. Moreover, partnerships between researchers and policy makers/managers to facilitate the conduct and use of systematic reviews should be considered to enhance relevance of reviews and thereby influence uptake.

CONCLUSIONS

Systematic review authors can consider our results when publishing their systematic reviews. These strategies should be rigorously evaluated to determine impact on use of reviews in decision-making.

Dissemination of Evidence From Systematic Reviews Through Academic CME Providers: A Feasibility Study

INTRODUCTION

Although systematic reviews represent a source of best evidence to support clinical decision-making, reviews are underutilized by clinicians. Barriers include lack of awareness, familiarity, and access. Efforts to promote utilization have focused on reaching practicing clinicians, leaving unexplored the roles of continuing medical education (CME) directors and faculty in promoting systematic review use. This study explored the feasibility of working with CME directors and faculty for that purpose.

METHODS

A convenience sample of five academic CME directors and faculty agreed to participate in a feasibility study exploring use in CME courses of systematic reviews from the Agency for Healthcare Research and Quality (AHRQ-SRs). AHRQ-SR topics addressed the comparative effectiveness of health care options. Participants received access to AHRQ-SR reports, associated summary products, and instructional resources. The feasibility study used mixed methods to assess 1) implementation of courses incorporating SR evidence, 2) identification of facilitators and barriers to integration, and 3) acceptability to CME directors, faculty, and learners.

RESULTS

Faculty implemented 14 CME courses of varying formats serving 1700 learners in urban, suburban, and rural settings. Facilitators included credibility, conciseness of messages, and availability of supporting materials; potential barriers included faculty unfamiliarity with SRs, challenges in maintaining review currency, and review scope. SR evidence and summary products proved acceptable to CME directors, course faculty, and learners by multiple measures.

DISCUSSION

This study demonstrates the feasibility of approaches to use AHRQ-SRs in CME courses/programming. Further research is needed to demonstrate generalizability to other types of CME providers and other systemic reviews.

Using a systematic review in clinical decision making: a pilot parallel, randomized controlled trial

BACKGROUND

Evidence suggests that systematic reviews are used infrequently by physicians in clinical decision-making. One proposed solution is to create filtered resources so that information is validated and refined in order to be read quickly. Two shortened systematic review formats were developed to enhance their use in clinical decision-making.

METHODS

To prepare for a full-scale trial, we conducted a pilot study to test methods and procedures in order to refine the processes. A recruitment email was sent to physicians practicing full- or part-time in family medicine or general internal medicine. The pilot study took place in an online environment and eligible physicians were randomized to one of the systematic review formats (shortened or full-length) and instructed to read the document. Participants were asked to provide the clinical bottom line and apply the information presented to a clinical scenario. Participants' answers were evaluated independently by two investigators against "gold standard" answers prepared by an expert panel.

RESULTS

Fifty-six clinicians completed the pilot study within a 2-month period with a response rate of 4.3 %. Agreement between investigators in assessing participants' answers was determined by calculating a kappa statistic. Two questions were assessed separately, and a kappa statistic was calculated at 1.00 (100 % agreement) for each.

CONCLUSIONS

Agreement between investigators in assessing participants' answers is satisfactory. Although recruitment for the pilot study was completed in a reasonable time-frame, response rates were low and will require large numbers of contacts. The results indicate that conducting a full-scale trial is feasible.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02414360 .

A usability study of two formats of a shortened systematic review for clinicians

OBJECTIVE

The aim of this study was to evaluate the usability of two formats of a shortened systematic review for clinicians.

MATERIALS AND METHODS

Usability of the prototypes was assessed using three cycles of iterative testing. 10 participants were asked to complete tasks of locating information or items within two prototypes and 'think aloud' while being audio taped. Interviews were also audio recorded and participants completed a systematic usability scale.

RESULTS

Revisions were made between each iteration in order to address issues identified by participants. Finding information relating to the number of studies in the meta-analysis, and locating the number of studies in the entire systematic review were revealed as areas needing attention during the usability evaluation.

CONCLUSIONS

Iterative testing combined with a multifaceted approach to usability testing offered essential insight into aspects of the prototypes that required modifications. Alterations were made in order to create finalised versions of the two shortened systematic review formats.

Improving the uptake of systematic reviews: a systematic review of intervention effectiveness and relevance

OBJECTIVE

Little is known about the barriers, facilitators and interventions that impact on systematic review uptake. The objective of this study was to identify how uptake of systematic reviews can be improved.

SELECTION CRITERIA

Studies were included if they addressed interventions enhancing the uptake of systematic reviews. Reports in any language were included. All decisionmakers were eligible. Studies could be randomised trials, cluster-randomised trials, controlled-clinical trials and before-and-after studies.

DATA SOURCES

We searched 19 databases including PubMed, EMBASE and The Cochrane Library, covering the full range of publication years from inception to December 2010. Two reviewers independently extracted data and assessed quality according to the Effective Practice and Organisation of Care criteria.

RESULTS

10 studies from 11 countries, containing 12 interventions met our criteria. Settings included a hospital, a government department and a medical school. Doctors, nurses, mid-wives, patients and programme managers were targeted. Six of the studies were geared to improving knowledge and attitudes while four targeted clinical practice.

SYNTHESIS OF RESULTS

Three studies of low-to-moderate risk of bias, identified interventions that showed a statistically significant improvement: educational visits, short summaries of systematic reviews and targeted messaging. Promising interventions include e-learning, computer-based learning, inactive workshops, use of knowledge brokers and an e-registry of reviews. Juxtaposing barriers and facilitators alongside the identified interventions, it was clear that the three effective approaches addressed a wide range of barriers and facilitators.

DISCUSSION

A limited number of studies were found for inclusion. However, the extensive literature search is one of the strengths of this review.

CONCLUSIONS

Targeted messaging, educational visits and summaries are recommended to enhance systematic review uptake. Identified promising approaches need to be developed further. New strategies are required to encompass neglected barriers and facilitators. This review addressed effectiveness and also appropriateness of knowledge uptake strategies.

Development of two shortened systematic review formats for clinicians

BACKGROUND

Systematic reviews provide evidence for clinical questions, however the literature suggests they are not used regularly by physicians for decision-making. A shortened systematic review format is proposed as one possible solution to address barriers, such as lack of time, experienced by busy clinicians. The purpose of this paper is to describe the development process of two shortened formats for a systematic review intended for use by primary care physicians as an information tool for clinical decision-making.

METHODS

We developed prototypes for two formats (case-based and evidence-expertise) that represent a summary of a full-length systematic review before seeking input from end-users. The process was composed of the following four phases: 1) selection of a systematic review and creation of initial prototypes that represent a shortened version of the systematic review; 2) a mapping exercise to identify obstacles described by clinicians in using clinical evidence in decision-making; 3) a heuristic evaluation (a usability inspection method); and 4) a review of the clinical content in the prototypes.

RESULTS

After the initial prototypes were created (Phase 1), the mapping exercise (Phase 2) identified components that prompted modifications. Similarly, the heuristic evaluation and the clinical content review (Phase 3 and Phase 4) uncovered necessary changes. Revisions were made to the prototypes based on the results.

CONCLUSIONS

Documentation of the processes for developing products or tools provides essential information about how they are tailored for the intended user. One step has been described that we hope will increase usability and uptake of these documents to end-users.

A critical second look at integrated knowledge translation

Integrated knowledge translation (IKT) requires active collaboration between researchers and the ultimate users of knowledge throughout a research process, and is being aggressively positioned as an essential strategy to address the problem of underutilization of research-derived knowledge. The purpose of this commentary is to assist potential "knowledge users", particularly those working in policy or service settings, by highlighting some of the more nuanced benefits of the IKT model, as well as some of its potential costs. Actionable outcomes may not be immediately (or ever) forthcoming, but the process of collaboration can result in group-level identity transformation that permits access to different professional perspectives as well as, we suggest, added organizational and social value. As well, the IKT approach provides space for the re-balancing of what is considered "expertise". We offer this paper to help practitioners, administrators and policymakers more realistically assess the potential benefits and costs of engaging in IKT-oriented research.

Interventions to improve the use of systematic reviews in decision-making by health system managers, policy makers and clinicians

BACKGROUND

Systematic reviews provide a transparent and robust summary of existing research. However, health system managers, national and local policy makers and healthcare professionals can face several obstacles when attempting to utilise this evidence. These include constraints operating within the health system, dealing with a large volume of research evidence and difficulties in adapting evidence from systematic reviews so that it is locally relevant. In an attempt to increase the use of systematic review evidence in decision-making a number of interventions have been developed. These include summaries of systematic review evidence that are designed to improve the accessibility of the findings of systematic reviews (often referred to as information products) and changes to organisational structures, such as employing specialist groups to synthesise the evidence to inform local decision-making.

OBJECTIVES

To identify and assess the effects of information products based on the findings of systematic review evidence and organisational supports and processes designed to support the uptake of systematic review evidence by health system managers, policy makers and healthcare professionals.

SEARCH METHODS

We searched The Cochrane Library, MEDLINE, EMBASE, CINAHL, Web of Science, and Health Economic Evaluations Database. We also handsearched two journals (Implementation Science and Evidence and Policy), Cochrane Colloquium abstracts, websites of key organisations and reference lists of studies considered for inclusion. Searches were run from 1992 to March 2011 on all databases, an update search to March 2012 was run on MEDLINE only.

SELECTION CRITERIA

Randomised controlled trials (RCTs), interrupted time-series (ITS) and controlled before-after studies (CBA) of interventions designed to aid the use of systematic reviews in healthcare decision-making were considered.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted the data and assessed the study quality. We extracted the median value across similar outcomes for each study and reported the range of values for each median value. We calculated the median of the two middlemost values if an even number of outcomes were reported.

MAIN RESULTS

We included eight studies evaluating the effectiveness of different interventions designed to support the uptake of systematic review evidence. The overall quality of the evidence was very low to moderate.Two cluster RCTs evaluated the effectiveness of multifaceted interventions, which contained access to systematic reviews relevant to reproductive health, to change obstetric care; the high baseline performance in some of the key clinical indicators limited the findings of these studies. There were no statistically significant effects on clinical practice for all but one of the clinical indicators in selected obstetric units in Thailand (median effect size 4.2%, range -11.2% to 18.2%) and none in Mexico (median effect size 3.5%, range 0.1% to 19.0%). In the second cluster RCT there were no statistically significant differences in selected obstetric units in the UK (median effect RR 0.92; range RR 0.57 to RR 1.10). One RCT evaluated the perceived understanding and ease of use of summary of findings tables in Cochrane Reviews. The median effect of the differences in responses for the acceptability of including summary of findings tables in Cochrane Reviews versus not including them was 16%, range 1% to 28%. One RCT evaluated the effect of an analgesic league table, derived from systematic review evidence, and there was no statistically significant effect on self-reported pain. Only one RCT evaluated an organisational intervention (which included a knowledge broker, access to a repository of systematic reviews and provision of tailored messages), and reported no statistically significant difference in evidence informed programme planning.Three interrupted time series studies evaluated the dissemination of printed bulletins based on evidence from systematic reviews. A statistically significant reduction in the rates of surgery for glue ear in children under 10 years (mean annual decline of -10.1%; 95% CI -7.9 to -12.3) and in children under 15 years (quarterly reduction -0.044; 95% CI -0.080 to -0.011) was reported. The distribution to general practitioners of a bulletin on the treatment of depression was associated with a statistically significant lower prescribing rate each quarter than that predicted by the rates of prescribing observed before the distribution of the bulletin (8.2%; P = 0.005).

AUTHORS' CONCLUSIONS

Mass mailing a printed bulletin which summarises systematic review evidence may improve evidence-based practice when there is a single clear message, if the change is relatively simple to accomplish, and there is a growing awareness by users of the evidence that a change in practice is required. If the intention is to develop awareness and knowledge of systematic review evidence, and the skills for implementing this evidence, a multifaceted intervention that addresses each of these aims may be required, though there is insufficient evidence to support this approach.

Do stakeholders in wound care prefer evidence-based wound care products? A survey in the Netherlands

For several wound products compelling evidence is available on their effectiveness, for example, from systematic reviews. The process of buying, prescribing and applying wound materials involve many stakeholders, who may not be aware of this evidence, although this is essential for uniform and optimum treatment choice. In this survey, we determined the general awareness and use of evidence, based on (Cochrane) systematic reviews, for wound products in open wounds and burns among wound care stakeholders, including doctors, nurses, buyers, pharmacologists and manufacturers. We included 262 stakeholders. Doctors preferred conventional antiseptics (e.g. iodine), while specialised nurses and manufacturers favoured popular products (e.g. silver). Most stakeholders considered silver-containing products as evidence-based effective antiseptics. These were mostly used by specialised nurses (47/57; 82%), although only few of them (9/55; 16%) thought using silver is evidence-based. For burns, silver sulfadiazine and hydrofibre were most popular. The majority of professionals considered using silver sulfadiazine to be evidence-based, which contradicts scientific results. Awareness and use of the Cochrane Library was lower among nurses than among doctors (P < 0.001). Two thirds of the manufacturers were unaware of, or never used, the Cochrane Library. Available compelling evidence in wound care is not equally internalised by stakeholders, which is required to ensure evidence-based decision making.

Feasibility of a knowledge translation CME program: Courriels Cochrane

INTRODUCTION

Systematic literature reviews provide best evidence, but are underused by clinicians. Thus, integrating Cochrane reviews into continuing medical education (CME) is challenging. We designed a pilot CME program where summaries of Cochrane reviews (Courriels Cochrane) were disseminated by e-mail. Program participants automatically received CME credit for each Courriel Cochrane they rated. The feasibility of this program is reported (delivery, participation, and participant evaluation).

METHOD

We recruited French-speaking physicians through the Canadian Medical Association. Program delivery and participation were documented. Participants rated the informational value of Courriels Cochrane using the Information Assessment Method (IAM), which documented their reflective learning (relevance, cognitive impact, use for a patient, expected health benefits). IAM responses were aggregated and analyzed.

RESULTS

The program was delivered as planned. Thirty Courriels Cochrane were delivered to 985 physicians, and 127 (12.9%) completed at least one IAM questionnaire. Out of 1109 Courriels Cochrane ratings, 973 (87.7%) conta-ined 1 or more types of positive cognitive impact, while 835 (75.3%) were clinically relevant. Participants reported the use of information for a patient and expected health benefits in 595 (53.7%) and 569 (51.3%) ratings, respectively.

DISCUSSION

Program delivery required partnering with 5 organizations. Participants valued Courriels Cochrane. IAM ratings documented their reflective learning. The aggregation of IAM ratings documented 3 levels of CME outcomes: participation, learning, and performance. This evaluation study demonstrates the feasibility of the Courriels Cochrane as an approach to further disseminate Cochrane systematic literature reviews to clinicians and document self-reported knowledge translation associated with Cochrane reviews.

Knowledge translation research: the science of moving research into policy and practice

Research findings will not change health outcomes unless health care organizations, systems, and professionals adopt them in practice. Knowledge translation research is the scientific study of the methods to promote the uptake of research findings by patients, health care providers, managers, and policy makers. Many forms of enquiry addressing different questions are needed to develop the evidence base for knowledge translation. In this paper we will present a description of the broad scope of knowledge translation research with a reflection on activities needed to further develop the science of knowledge translation. Consideration of some of the shared research challenges facing the fields of knowledge translation and continuing professional development will also be presented.