Creating clinically relevant knowledge from systematic reviews: the challenges of knowledge translation

Incorporating the Concept of Relevance in Clinical Rehabilitation Research and Its Reviews May Improve Uptake by Stakeholders

ABSTRACT

The "relevance" of research to stakeholders is an important factor in influencing the uptake of new knowledge into practice; however, this concept is neither well defined nor routinely incorporated in clinical rehabilitation research. Developing a uniform definition, measurement standards, stakeholder engagement strategies, and guiding frameworks that bolster relevance may help incorporate the concept as a key element in research planning and design. This article presents a conceptual argument for why relevance matters, proposes a working definition, and suggests strategies for operationalizing the construct in the context of clinical rehabilitation research. We place special emphasis on the importance of promoting relevance to patients, caregivers, and clinicians and provide preliminary frameworks and innovative study designs that can assist clinical rehabilitation researchers in doing so. We argue that researchers who include a direct statement regarding why and to whom a study is relevant and who incorporate considerations of relevance throughout all phases of study design produce more useful research for patients, caregivers, and clinicians, increasing its chance of uptake into practice. Consistent consideration of relevance, particularly to nonacademic audiences, during the conceptualization, study design, presentation, and dissemination of clinical rehabilitation research may promote the uptake of findings by patients, caregivers, and providers.

Evidence transfer: ensuring end users are aware of, have access to, and understand the evidence

The Joanna Briggs Institute Model of evidence-based healthcare (EBHC) states that the main phases of EBHC include evidence synthesis, transfer, and implementation. There has been some confusion regarding the term 'evidence transfer', with this term previously being considered by many as synonymous with knowledge or evidence translation. The aim of this paper is to discuss a proposed definition of evidence transfer and the pivotal role it plays as part of the EBHC process. 'Evidence transfer' can be thought of simply as getting the message (evidence) across and at its core it is all about information delivery. We specifically define evidence transfer as a process that helps communicate or convey the results of research or evidence, or brings evidence to the forefront. It is focused on ensuring people are aware of, have access to and understand evidence.

Using functional electrical stimulation with stroke survivors: A survey of Victorian occupational therapists and physiotherapists

BACKGROUND/AIM

Functional electrical stimulation (FES) improves active movement of the hemiplegic upper and lower limbs following stroke. The use of FES by Australian allied health clinicians in stroke rehabilitation is, however, unknown. The purpose of this study was to understand the use of FES in clinical practice. Reasons for the use of FES and potential variables that influence decision-making were also investigated.

METHODS

Cross-sectional study of Victorian allied health clinicians, using a snowball recruitment method. Ninety-seven eligible therapists completed the anonymous online survey. Data were analysed using frequency distributions.

RESULTS

The majority of respondents were occupational therapists (n = 60; 62%). Approximately half of the respondents (n = 50; 52%) reported using FES in the past two years to improve a stroke survivor's ability to use their arm in daily activities. Respondents suggested that receiving workplace training from colleagues to learn how to use FES is the preferred method of education. Of those who received education (n = 80), 50 participants reported using FES in their practice.

CONCLUSION

There is variable use of FES in stroke rehabilitation to increase active movement after stroke. While there was moderate agreement about when to use FES and useful education approaches for learning to use FES, further research is needed to better understand strategies which could be implemented to support increased FES use in stroke rehabilitation.

Preferences of Knowledge Users for Two Formats of Summarizing Results from Systematic Reviews: Infographics and Critical Appraisals

OBJECTIVES

To examine and compare preferences of knowledge users for two different formats of summarizing results from systematic reviews: infographics and critical appraisals.

DESIGN

Cross-sectional.

SETTING

Annual members' meeting of a Network of Centres of Excellence in Knowledge Mobilization called TREKK (Translating Emergency Knowledge for Kids). TREKK is a national network of researchers, clinicians, health consumers, and relevant organizations with the goal of mobilizing knowledge to improve emergency care for children.

PARTICIPANTS

Members of the TREKK Network attending the annual meeting in October 2013.

OUTCOME MEASURES

Overall preference for infographic vs. critical appraisal format. Members' rating of each format on a 10-point Likert scale for clarity, comprehensibility, and aesthetic appeal. Members' impressions of the appropriateness of the two formats for their professional role and for other audiences.

RESULTS

Among 64 attendees, 58 members provided feedback (91%). Overall, their preferred format was divided with 24/47 (51%) preferring the infographic to the critical appraisal. Preference varied by professional role, with 15/22 (68%) of physicians preferring the critical appraisal and 8/12 (67%) of nurses preferring the infographic. The critical appraisal was rated higher for clarity (mean 7.8 vs. 7.0; p = 0.03), while the infographic was rated higher for aesthetic appeal (mean 7.2 vs. 5.0; p<0.001). There was no difference between formats for comprehensibility (mean 7.6 critical appraisal vs. 7.1 infographic; p = 0.09). Respondents indicated the infographic would be most useful for patients and their caregivers, while the critical appraisal would be most useful for their professional roles.

CONCLUSIONS

Infographics are considered more aesthetically appealing for summarizing evidence; however, critical appraisal formats are considered clearer and more comprehensible. Our findings show differences in terms of audience-specific preferences for presentation of research results. This study supports other research indicating that tools for knowledge dissemination and translation need to be targeted to specific end users' preferences and needs.

The Development and Use of Evidence Summaries for Point of Care Information Systems: A Streamlined Rapid Review Approach

BACKGROUND

A systematic review of evidence is the research method which underpins the traditional approach to evidence-based health care. As systematic reviews follow a rigorous methodology, they can take a substantial amount of time to complete ranging in duration from 6 months to 2 years. Rapid reviews have been proposed as a method to provide summaries of the literature in a more timely fashion.

AIM

The aim of this paper is to outline our experience of developing evidence summaries in the context of a point of care resource as a contribution to the emerging field of rapid review methodologies.

METHODS

Evidence summaries are defined as a synopsis that summarizes existing international evidence on healthcare interventions or activities. These summaries are based on structured searches of the literature and selected evidence-based healthcare databases. Following the search, all studies are assessed for internal validity using an abridged set of critical appraisal tools. Once developed, they undergo three levels of peer review by internal and external experts.

RESULTS

As of November 2014, there are 2458 evidence summaries that have been created across a range of conditions to inform evidence-based healthcare practices. In addition, there is ongoing development of various new evidence summaries on a wide range of topics. Approximately 60-70 new evidence summaries are published every month, covering research in various medical specialty areas. All summaries are updated annually.

LINKING EVIDENCE TO ACTION

Systematic reviews, although the ideal type of research to inform practice, often do not meet the needs of users at the point of care. This article describes the development framework for the creation of evidence summaries, a type of rapid review. Although evidence summaries may result in a less rigorous process of development, they can be useful for improving practice at the point of care.

Improving Chronic Diseases Management Through the Development of an Evidence-Based Resource

OBJECTIVE

There is a large gap between evidence and practice within health care, particularly within the field of chronic disease. To reduce this gap and improve the management of chronic disease, a collaborative partnership between two schools within a large university and two industry partners (a large regional rural hospital and a rural community health center) in rural Victoria, Australia, was developed. The aim of the collaboration was to promote the development of translation science and the implementation of evidence-based health care in chronic disease with a specific focus on developing evidence-based resources that are easily accessed by clinicians.

METHODS

A working group consisting of members of the collaborating organizations and an internationally renowned expert reference group was formed. The group acted as a steering committee and was tasked with developing a taxonomy of the resources. In addition, a peer review process of all resources was established. A corresponding reference group consisting of researchers and clinicians who are clinical experts in various fields was involved in the review process. The resources developed by the group include evidence summaries and recommended practices made available on a web-based database, which can be accessed via subscription by clinicians and researchers worldwide.

RESULTS

As of mid-2014, there were 109 new evidence summaries and 25 recommended practices detailing the best available evidence on topics related to chronic disease management including asthma, diabetes, heart failure, dementia, and others. Training sessions and a newsletter were developed for clinicians within the node to enable them to use the content effectively.

LINKING EVIDENCE TO ACTION

This paper describes the processes involved in the successful development of the collaborative partnership and its evolution into producing a valuable resource for the translation of evidence into practice in the areas of chronic disease management. The resource developed is being used by clinicians to inform practice and support their clinical decision making.

Introduction of new technologies and decision making processes: a framework to adapt a Local Health Technology Decision Support Program for other local settings

PURPOSE

Introducing new health technologies, including medical devices, into a local setting in a safe, effective, and transparent manner is a complex process, involving many disciplines and players within an organization. Decision making should be systematic, consistent, and transparent. It should involve translating and integrating scientific evidence, such as health technology assessment (HTA) reports, with context-sensitive evidence to develop recommendations on whether and under what conditions a new technology will be introduced. However, the development of a program to support such decision making can require considerable time and resources. An alternative is to adapt a preexisting program to the new setting.

MATERIALS AND METHODS

We describe a framework for adapting the Local HTA Decision Support Program, originally developed by the Department of Surgery and Surgical Services (Calgary, AB, Canada), for use by other departments. The framework consists of six steps: 1) development of a program review and adaptation manual, 2) education and readiness assessment of interested departments, 3) evaluation of the program by individual departments, 4) joint evaluation via retreats, 5) synthesis of feedback and program revision, and 6) evaluation of the adaptation process.

RESULTS

Nine departments revised the Local HTA Decision Support Program and expressed strong satisfaction with the adaptation process. Key elements for success were identified.

CONCLUSION

Adaptation of a preexisting program may reduce duplication of effort, save resources, raise the health care providers' awareness of HTA, and foster constructive stakeholder engagement, which enhances the legitimacy of evidence-informed recommendations for introducing new health technologies. We encourage others to use this framework for program adaptation and to report their experiences.

Adapting low back pain guidelines within a multidisciplinary context: a process evaluation

RATIONALE, AIMS AND OBJECTIVES

The Alberta Ambassador Program (AAP) adapted seven clinical practice guidelines on low back pain (LBP) into a single guideline spanning the continuum of care from prevention and diagnosis through to treatment. The Ambassador adaptation process was evaluated to 1 Identify the major challenges encountered and successful strategies utilized; 2 Assess strengths and weaknesses by benchmarking it with the ADAPTE framework; and 3 Identify opportunities for improvement.

METHOD

External consultants reviewed the Ambassador and ADAPTE materials and conducted semi-structured telephone interviews with 29 participants from the AAP committees. All participants were asked about the major challenges encountered and potential areas for improvement.

RESULTS

The response rate was 83% (29/35). There was strong consensus that the Ambassador guideline adaptation process was sound and rigorous all respondents indicated willingness to participate in further iterations of the Program. Key elements of success were identified. The main steps and sequence of the process were closely aligned with the ADAPTE framework, although the AAP incorporated additional enhancements which augmented the process. The main divergences between the two frameworks centred on the organizational structure and the methods used to overcome methodological difficulties.

CONCLUSION

The AAP successfully utilized existing stakeholder interest to create an overarching guideline for managing LBP across multiple primary care disciplines. The study highlighted the strengths and weaknesses of the Program, and identified practical strategies for improvement. Evaluating guideline adaptation processes is pivotal to ensuring that they continue to be an efficient, rigorous and practicable option for producing contextualized, clinically relevant guidelines.

Consensus-based recommendations for investigating clinical heterogeneity in systematic reviews

BACKGROUND

Critics of systematic reviews have argued that these studies often fail to inform clinical decision making because their results are far too general, that the data are sparse, such that findings cannot be applied to individual patients or for other decision making. While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Clinical heterogeneity, true effect heterogeneity, can be defined as variability among studies in the participants, the types or timing of outcome measurements, and the intervention characteristics. The objective of this project was to develop recommendations for investigating clinical heterogeneity in systematic reviews.

METHODS

We used a modified Delphi technique with three phases: (1) pre-meeting item generation; (2) face-to-face consensus meeting in the form of a modified Delphi process; and (3) post-meeting feedback. We identified and invited potential participants with expertise in systematic review methodology, systematic review reporting, or statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity.

RESULTS

Between April and June of 2011, we conducted phone calls with participants. In June 2011 we held the face-to-face focus group meeting in Ann Arbor, Michigan. First, we agreed upon a definition of clinical heterogeneity: Variations in the treatment effect that are due to differences in clinically related characteristics. Next, we discussed and generated recommendations in the following 12 categories related to investigating clinical heterogeneity: the systematic review team, planning investigations, rationale for choice of variables, types of clinical variables, the role of statistical heterogeneity, the use of plotting and visual aids, dealing with outlier studies, the number of investigations or variables, the role of the best evidence synthesis, types of statistical methods, the interpretation of findings, and reporting.

CONCLUSIONS

Clinical heterogeneity is common in systematic reviews. Our recommendations can help guide systematic reviewers in conducting valid and reliable investigations of clinical heterogeneity. Findings of these investigations may allow for increased applicability of findings of systematic reviews to the management of individual patients.

Never the twain shall meet?--a comparison of implementation science and policy implementation research

Background

Many of society’s health problems require research-based knowledge acted on by healthcare practitioners together with implementation of political measures from governmental agencies. However, there has been limited knowledge exchange between implementation science and policy implementation research, which has been conducted since the early 1970s. Based on a narrative review of selective literature on implementation science and policy implementation research, the aim of this paper is to describe the characteristics of policy implementation research, analyze key similarities and differences between this field and implementation science, and discuss how knowledge assembled in policy implementation research could inform implementation science.

Discussion

Following a brief overview of policy implementation research, several aspects of the two fields were described and compared: the purpose and origins of the research; the characteristics of the research; the development and use of theory; determinants of change (independent variables); and the impact of implementation (dependent variables). The comparative analysis showed that there are many similarities between the two fields, yet there are also profound differences. Still, important learning may be derived from several aspects of policy implementation research, including issues related to the influence of the context of implementation and the values and norms of the implementers (the healthcare practitioners) on implementation processes. Relevant research on various associated policy topics, including The Advocacy Coalition Framework, Governance Theory, and Institutional Theory, may also contribute to improved understanding of the difficulties of implementing evidence in healthcare. Implementation science is at a relatively early stage of development, and advancement of the field would benefit from accounting for knowledge beyond the parameters of the immediate implementation science literature.

Summary

There are many common issues in policy implementation research and implementation science. Research in both fields deals with the challenges of translating intentions into desired changes. Important learning may be derived from several aspects of policy implementation research.

Investigating clinical heterogeneity in systematic reviews: a methodologic review of guidance in the literature

BACKGROUND

While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. The objective of this study is to summarize and collate suggested methods for investigating clinical heterogeneity in systematic reviews.

METHODS

We searched databases (Medline, EMBASE, CINAHL, Cochrane Library, and CONSORT, to December 2010) and reference lists and contacted experts to identify resources providing suggestions for investigating clinical heterogeneity between controlled clinical trials included in systematic reviews. We extracted recommendations, assessed resources for risk of bias, and collated the recommendations.

RESULTS

One hundred and one resources were collected, including narrative reviews, methodological reviews, statistical methods papers, and textbooks. These resources generally had a low risk of bias, but there was minimal consensus among them. Resources suggested that planned investigations of clinical heterogeneity should be made explicit in the protocol of the review; clinical experts should be included on the review team; a set of clinical covariates should be chosen considering variables from the participant level, intervention level, outcome level, research setting, or others unique to the research question; covariates should have a clear scientific rationale; there should be a sufficient number of trials per covariate; and results of any such investigations should be interpreted with caution.

CONCLUSIONS

Though the consensus was minimal, there were many recommendations in the literature for investigating clinical heterogeneity in systematic reviews. Formal recommendations for investigating clinical heterogeneity in systematic reviews of controlled trials are required.

The Alberta Health Technology Assessment (HTA) Ambassador Program: The Development of a Contextually Relevant, Multidisciplinary Clinical Practice Guideline for Non-specific Low Back Pain: A Review

PURPOSE

To describe the development of a contextually relevant multidisciplinary clinical practice guideline (CPG) for non-specific low back pain (LBP) and to discuss its value to the management of LBP and the practice of physiotherapy.

METHOD

To mitigate an identified knowledge gap for Alberta primary-care practitioners in the management of non-specific LBP, a collaborative process was developed to engage multidisciplinary health care providers in designing a primary-care CPG for non-specific LBP. A comprehensive review of published LBP guidelines identified the seven highest-quality CPGs; these were used to inform a multidisciplinary guideline development group (GDG) as they developed the CPG.

RESULTS

The GDG constructed a CPG for non-specific LBP along with point-of-care decision-support and patient-education tools.

CONCLUSIONS

The Ambassador Program on Low Back Pain worked with front-line clinicians from across Alberta to review the best available evidence in developing a CPG responsive to the Alberta context. This CPG is intervention specific and provides a wide range of primary-care practitioners with the best available evidence to inform their clinical decisions in managing non-specific LBP.

Facilitating evidence-based practice: process, strategies, and resources

Occupational therapy practitioners may encounter challenges when they try to incorporate evidence into practice. To embrace evidence-based practice (EBP), clinicians must have readily available, relevant, and concisely summarized evidence. Although researchers have described the importance and process of EBP, less has been written about how to efficiently integrate evidence into practice. Clinicians may benefit from examples of reasoning, strategies, and resources to successfully integrate evidence. This article reviews the steps of EBP and offers recommendations to overcome common barriers. For EBP to become integrated into practice, greater communication and collaboration among all stakeholders must occur. EBP and knowledge translation require multiple processes and coordinated efforts. Therefore, everyone from practitioners to employers has a role in increasing EBP and transferring knowledge for practice. To encourage discussion and actions, the article provides implications and recommendations for practitioners, researchers, educators, organizations, and policymakers.

Making the AGREE tool more user-friendly: the feasibility of a user guide based on Boolean operators

Rationale, aims and objectives The Appraisal of Guidelines Research and Evaluation (AGREE) instrument is a generic tool for assessing guideline quality. This feasibility study aimed to reduce the ambiguity and subjectivity associated with AGREE item scoring, and to augment the tool's capacity to differentiate between good- and poor-quality guidelines. Methods A literature review was conducted to ascertain what AGREE instrument adjustments had been reported to date. The AGREE User Guide was then modified by: 1 constructing a detailed set of instructions, or dictionary, using Boolean operators, and 2 overlaying seven criteria to categorize guideline quality. The feasibility of the Boolean-based dictionary was tested by three appraisers using three randomly selected guidelines on low back pain management. The dictionary was then revised and re-tested. Results Of the 52 published studies identified, 14% had modified the instrument by adding or deleting items and 35% had adopted strategies, such as using a consensus approach, to overcome inconsistencies and ensure identical item scoring among appraisers. For the feasibility test, Pearson correlation coefficients ranged from 0.27 to 0.81. Revision and re-testing of the dictionary increased the level of agreement (range 0.41 to 0.94). Application of the revised dictionary not only decreased the variability of the domain scores, but also reduced the tool's reliability among inexperienced appraisers. Conclusion Appraisers found the Boolean-based AGREE User Guide easier to use than the original, which improved their confidence in the tool. Good reliability was achieved in the feasibility test, but the reliability and validity of some of the changes will require further evaluation.

Improving rehabilitation treatment in a local setting: a case study of prosthetic rehabilitation

OBJECTIVE

To contribute to the discussion on the research-practice gap by illustrating obstacles and opportunities that arise in an evidence-informed improvement process of prosthetic rehabilitation in a local setting.

SETTING

Dutch rehabilitation centre. PRESUPPOSITION: The improvement process was considered as a two-way translation process rather than a unidirectional process of knowledge transfer between science and practice.

METHOD

Case study and participatory research methods comprising documentary analysis, treatment observations, individual and focus groups interviews, and literature studies. A qualitative software program (Atlas-ti) was used to triangulate the collected data.

RESULTS

The main concern of local practitioners was identified to be the post-discharge decline in functional capacity in elderly amputees. This was related to a predominantly biomedical and biomechanical approach, and accompanying traditional therapist-patient interactions. The content and underpinnings of prosthetic treatments were scarcely specified in either the scientific literature or the local setting. Generic principles and practices from other fields were useful for treatment innovation for post-discharge problems, such as task- and context-specific training and self-management education. A circuit training focused on motor learning and a problem-solving training focused on social learning were developed by integrating amputation-specific knowledge.

CONCLUSION

Improving rehabilitation practice with the use of available evidence is a heterogeneous and multifaceted scientific enterprise. Such an enterprise requires as much self-reflexivity from researchers as from practitioners.

Can 'Cascades' make guidelines global?

Why are guidelines in medicine so important today? What role do they have? Why and how did the World Gastroenterology Organization (WGO) choose a global focus? What does this mean for guidelines? These are the underlying questions addressed by our article. We argue that the addition of 'Cascades' to guidelines will increase their impact in large parts of the world. By so doing, we hope to add a new dimension to the 'knowledge into action' debate. A number of illustrations shows how raised expectations and resource restrictions pose - or should pose - an enormous challenge for guideline makers. Furthermore, the emphasis on evidence also creates problems for guideline making. If resources are limited it is unlikely gold-standard technologies are available. We believe Cascades can help. A Cascade is a selection of two or more hierarchical diagnostic or therapeutic options, based on proven medical procedures, methods, tools or products for the same disease, condition or diagnosis, aiming to achieve the same outcome and ranked by available resources. The construction of such a cascade is a hazardous intellectual journey that goes, to some extent, against established practice. But lives can be saved by matching options for diagnosis and treatment to available resources. While the optimal strategy, defined through an evidence-based approach, should always be the goal, one must be aware of the resource limitations that confront our colleagues in certain parts of the world and we should endeavour to work with them in the guideline development process to develop strategies that are clinically sound yet economically feasible and dacceptable to their populace.