Rapid Evidence Assessment of the Literature (REAL(©)): streamlining the systematic review process and creating utility for evidence-based health care

Creating synergies among education/research, practice, and policy environments to build capacity for the scholar role in occupational therapy and physiotherapy in the Canadian context

Scholarly practice (SP) is considered a key competency of occupational therapy and physiotherapy. To date, the three sectors-education/research, practice, and policy/regulation-that support SP have been working relatively independently. The goals of this project were to (a) understand how representatives of the three sectors conceptualize SP; (b) define each sector's individual and collective roles in supporting SP; (c) identify factors influencing the enactment of SP and the specific needs of how best to support SP; and (d) co-develop goals and strategies to support SP across all sectors. We used interpretive description methodology. Consistent with an integrated knowledge translation approach, partners representing the three sectors across Canada recruited individuals from each sector, developed the content and questions for three focus groups, and collected and analyzed the data. Inspired by the Consolidated Framework for Implementation Research, we developed the questions for the second focus group. We analyzed the data using an inductive thematic analysis method. Thirty-nine participants from the three sectors participated. Themes related to participants' conceptualization of SP included (a) ongoing process, (b) reflective process, (c) broad concept, and (d) collective effort. Themes describing factors influencing and supporting SP were (a) recognition, (b) appropriate conceptualization, (c) social network, (d) accessibility to resources, and (e) forces outside of practitioners' effort. Goals to support SP included (a) further recognizing SP, (b) sustaining SP competency, and (c) ensuring access to information. SP requires collaborative and integrated intersectoral support and further recognition of its importance through the collaboration of multiple stakeholders.

A scoping review of research on languaging in second language education

With the rapid development of studies on second language acquisition (SLA) and sociocultural theory, languaging has recently gained increasing attention. To assess the existing research on languaging and determine implications for future research, the present study aims to present a scoping review of research on languaging in second language (L2) education. Specifically, this study intends to explore the major subjects of languaging, the effects of languaging, the factors that may influence the effects of languaging, and the ways in which languaging is integrated into the L2 class. In total, 27 relevant peer-reviewed articles were selected for further analysis using the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol (PRISMA-P) for further analysis. The findings of this review show that a) languaging subjects are primarily pertinent to university learners; b) the facilitative effect of languaging on language learning has been verified, with written languaging being the most frequently practiced form of languaging task; c) learner language proficiency, learning style, and corrective feedback are the critical factors contributing to the effect of languaging; and d) three ways of integrating languaging into L2 classes are identified: experiment, pedagogical means, and a mixed mode of experiment and pedagogical techniques. Based on the findings of this review, a model of languaging integration was proposed in four steps: task assignment, languaging with prompts, a post-test, and reflection. This review suggests future avenues for researching and implementing languaging in L2 classes.

A Systematic Review of Machine-Translation-Assisted Language Learning for Sustainable Education

With the rapid development of artificial intelligence, machine translation (MT) has gained popularity in recent years. This study aims to present a systematic review of literature on MT-assisted language learning in terms of main users, theoretical frameworks, users’ attitudes, and the ways in which MT tools are integrated with language teaching and learning. To this end, relevant peer-reviewed articles (n = 26) were selected through the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol (PRISMA-P) for further analysis. The findings revealed that the main MT users were undergraduate and graduate students. Both teachers and students held mixed attitudes for different reasons. It was also found that MT integration followed four steps, i.e., introduction, demonstration, task assignment, and reflection. The procedures of MT integration could be updated and perfected by introducing other features in the future.

Defining Rapid Reviews: a systematic scoping review and thematic analysis of definitions and defining characteristics of rapid reviews

BACKGROUND AND OBJECTIVE

Rapid reviews were first mentioned in the literature in 1997, when Best et al. described the rapid health technology assessment program in the south and west regions of England but did not provide a formal definition. More recently, the only consensus around a rapid review definition is that a formal definition does not exist. The primary aim of this work is to create a repository of existing definitions and to identify key themes, which may help the knowledge synthesis community in defining rapid review products.

METHODS

A systematic scoping review was performed to identify definitions used in journal-published rapid reviews written in English between 2017 and January 2019. We searched Medline, Embase Classic + Embase, PsycINFO, ERIC, Cochrane Library, CINAHL, and Web of Science on December 21, 2018. Two reviewers performed study selection and data extraction using a priori-defined methods published in a protocol. Definitions from rapid review methods articles (published from 1997 onward) identified in another scoping review were added to the results, and all definitions were thematically analyzed using NVivo. A quantitative analysis was also performed around studies cited.

RESULTS

Definitions from 216 rapid reviews and 90 rapid review methods articles were included in the thematic analysis. Eight key themes were identified: accelerated/rapid process or approach, variation in methods shortcuts, focus/depth/breadth of scope, compare and contrast to a full traditional systematic review, stakeholder rationale, resource efficiency rationale, systematic approach, bias/limitations. Secondary referencing was a common occurrence.

CONCLUSION

Thematic analysis performed in this systematic scoping review has allowed for the creation of a suggested definition for rapid reviews that can be used to inform the systematic review community.

The Byzantine Role of Big Data Application in Nursing Science: A Systematic Review

Big data have the potential to determine enhanced decision-making process and to personalize the approach of delivering care when applied in nursing science. So far, the literature on this topic is still not synthesized for the period between 2014 and 2018. Thus, this systematic review aimed to identify and synthesize the most recent evidence on big data application in nursing research. The systematic search was undertaken for the evidence published from January 2014 to May 2018, and the outputs were formatted using the PRISMA Flow Diagram, whereas the quality appraisal was addressed by recommendations consistent with the Critical Appraisal Skills Program. Twelve studies on big data in nursing were included and divided into two themes: the majority of the studies aimed to determine prediction assessment, while only four studies were related to the impact of big data applications to support clinical practice. This review tracks the recent state of knowledge on big data applications in nursing science, revealing the potential for nursing engagement in big data science, even if currently limited to some fields. Big data applications in nursing might have a tremendous potential impact, but are currently underused in research and clinical practice.

Engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process: a scoping review

Background

It is unclear how to engage a wide range of knowledge users in research. We aimed to map the evidence on engaging knowledge users with an emphasis on policy-makers, health system managers, and policy analysts in the knowledge synthesis process through a scoping review.

Methods

We used the Joanna Briggs Institute guidance for scoping reviews. Nine electronic databases (e.g., MEDLINE), two grey literature sources (e.g., OpenSIGLE), and reference lists of relevant systematic reviews were searched from 1996 to August 2016. We included any type of study describing strategies, barriers and facilitators, or assessing the impact of engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process. Screening and data abstraction were conducted by two reviewers independently with a third reviewer resolving discrepancies. Frequency and thematic analyses were conducted.

Results

After screening 8395 titles and abstracts followed by 394 full-texts, 84 unique documents and 7 companion reports fulfilled our eligibility criteria. All 84 documents were published in the last 10 years, and half were prepared in North America. The most common type of knowledge synthesis with knowledge user engagement was a systematic review (36%). The knowledge synthesis most commonly addressed an issue at the level of national healthcare system (48%) and focused on health services delivery (17%) in high-income countries (86%).

Policy-makers were the most common (64%) knowledge users, followed by healthcare professionals (49%) and government agencies as well as patients and caregivers (34%). Knowledge users were engaged in conceptualization and design (49%), literature search and data collection (52%), data synthesis and interpretation (71%), and knowledge dissemination and application (44%). Knowledge users were most commonly engaged as key informants through meetings and workshops as well as surveys, focus groups, and interviews either in-person or by telephone and emails. Knowledge user content expertise/awareness was a common facilitator (18%), while lack of time or opportunity to participate was a common barrier (12%).

Conclusions

Knowledge users were most commonly engaged during the data synthesis and interpretation phases of the knowledge synthesis conduct. Researchers should document and evaluate knowledge user engagement in knowledge synthesis.

Registration details

Open Science Framework (https://osf.io/4dy53/).

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0717-x) contains supplementary material, which is available to authorized users.

Methodological approach to moving nutritional science evidence into practice

The Metabolically Optimized Brain study explored nutritional science believed to be ready to place into practice to help improve US service members' cognitive performance and, thereby, optimize mission-readiness. A transparent, step-wise, research approach was used for informing evidence-based decisions among and for various, diverse stakeholders. A steering committee and subject-matter experts convened to devise the protocol and independent systematic reviews were performed to determine the quality of the evidence for nutritional science in 4 areas relevant to military populations: (1) caffeinated foods and beverages; (2) omega-3 polyunsaturated fatty acids; (3) plant-based foods and beverages or their phytochemical constituents; and (4) whole dietary patterns. A research expert panel was asked to then recommend future research directions and solutions likely to benefit warfighters. An implementation expert panel further considered how to apply sound nutritional science in a cost-effective manner. This article summarizes the methodological processes, high-level results, global research recommendations, and priorities for implementation. Specific results of the individual dietary interventions, as well as recommendations for moving this field of research and practice forward, are detailed throughout the current supplement.

Priorities for implementing nutritional science into practice to optimize military performance

The Metabolically Optimized Brain study explores nutritional science believed to be ready to place into practice to help improve US service member mission-readiness and performance. To this end, an implementation expert panel considered how the US Department of Defense Subsistence Food Service Program, which is operated by each branch of the military in dining facilities within the continental United States, could apply the best nutritional science in a cost-effective manner. The work of this panel was facilitated through a series of thematic conversations guided by evidence generated through systematic reviews, which were performed to identify systems and process gaps and propose possible solutions. The expert panel used a Delphi method of multiple voting, and ultimately proposed 11 systems changes, of which 6 were ranked as highest priority. The proposed highest priority changes were then discussed by the participants with additional stakeholders. The process described here highlights how experts from different sectors operating in a complex system of subsystems can come together to cross talk, identify gaps, and propose mutually beneficial system and process changes to improve the alignment of nutritional science and institutional food-service practice.

Claim Assessment Profile: A Method for Capturing Healthcare Evidence in the Scientific Evaluation and Review of Claims in Health Care (SEaRCH)

Background: Grounding health claims in an evidence base is essential for determining safety and effectiveness. However, it is not appropriate to evaluate all healthcare claims with the same methods. “Gold standard” randomized controlled trials may skip over important qualitative and observational data about use, benefits, side effects, and preferences, issues especially salient in research on complementary and integrative health (CIH) practices. This gap has prompted a move toward studying treatments in their naturalistic settings. In the 1990s, a program initiated under the National Institutes of Health was designed to provide an outreach to CIH practices for assessing the feasibility of conducting retrospective or prospective evaluations. The Claim Assessment Profile further develops this approach, within the framework of Samueli Institute's Scientific Evaluation and Review of Claims in Health Care (SEaRCH) method.

Methods/Design: The goals of a Claim Assessment Profile are to clarify the elements that constitute a practice, define key outcomes, and create an explanatory model of these impacts. The main objective is to determine readiness and capacity of a practice to engage in evaluation of effectiveness. This approach is informed by a variety of rapid assessment and stakeholder-driven methods. Site visits, structured qualitative interviews, surveys, and observational data on implementation provide descriptive data about the practice. Logic modeling defines inputs, processes, and outcome variables; Path modeling defines an analytic map to explore.

Discussion: The Claim Assessment Profile is a rapid assessment of the evaluability of a healthcare practice. The method was developed for use on CIH practices but has also been applied in resilience research and may be applied beyond the healthcare sector. Findings are meant to provide sufficient data to improve decision-making for stakeholders. This method provides an important first step for moving existing promising yet untested practices into comprehensive evaluation.

Scientific Evaluation and Review of Claims in Health Care (SEaRCH): A Streamlined, Systematic, Phased Approach for Determining "What Works" in Healthcare

Background: Answering the question of “what works” in healthcare can be complex and requires the careful design and sequential application of systematic methodologies. Over the last decade, the Samueli Institute has, along with multiple partners, developed a streamlined, systematic, phased approach to this process called the Scientific Evaluation and Review of Claims in Health Care (SEaRCH™). The SEaRCH process provides an approach for rigorously, efficiently, and transparently making evidence-based decisions about healthcare claims in research and practice with minimal bias.

Methods: SEaRCH uses three methods combined in a coordinated fashion to help determine what works in healthcare. The first, the Claims Assessment Profile (CAP), seeks to clarify the healthcare claim and question, and its ability to be evaluated in the context of its delivery. The second method, the Rapid Evidence Assessment of the Literature (REAL^©), is a streamlined, systematic review process conducted to determine the quantity, quality, and strength of evidence and risk/benefit for the treatment. The third method involves the structured use of expert panels (EPs). There are several types of EPs, depending on the purpose and need. Together, these three methods—CAP, REAL, and EP—can be integrated into a strategic approach to help answer the question “what works in healthcare?” and what it means in a comprehensive way.

Discussion: SEaRCH is a systematic, rigorous approach for evaluating healthcare claims of therapies, practices, programs, or products in an efficient and stepwise fashion. It provides an iterative, protocol-driven process that is customized to the intervention, consumer, and context. Multiple communities, including those involved in health service and policy, can benefit from this organized framework, assuring that evidence-based principles determine which healthcare practices with the greatest promise are used for improving the public's health and wellness.

The Impact of Massage Therapy on Function in Pain Populations-A Systematic Review and Meta-Analysis of Randomized Controlled Trials: Part III, Surgical Pain Populations

OBJECTIVE

Pain is multi-dimensional and may be better addressed through a holistic, biopsychosocial approach. Massage therapy is commonly practiced among patients seeking pain management; however, its efficacy is unclear. This systematic review and meta-analysis is the first to rigorously assess the quality of the evidence for massage therapy's efficacy in treating pain, function-related, and health-related quality of life outcomes in surgical pain populations.

METHODS

Key databases were searched from inception through February 2014. Eligible randomized controlled trials were assessed for methodological quality using SIGN 50 Checklist. Meta-analysis was applied at the outcome level. A professionally diverse steering committee interpreted the results to develop recommendations.

RESULTS

Twelve high quality and four low quality studies were included in the review. Results indicate massage therapy is effective for treating pain [standardized mean difference (SMD) = -0.79] and anxiety (SMD = -0.57) compared to active comparators.

CONCLUSION

Based on the available evidence, weak recommendations are suggested for massage therapy, compared to active comparators for reducing pain intensity/severity and anxiety in patients undergoing surgical procedures. This review also discusses massage therapy safety, challenges within this research field, how to address identified research gaps, and next steps for future research.

The Impact of Massage Therapy on Function in Pain Populations-A Systematic Review and Meta-Analysis of Randomized Controlled Trials: Part II, Cancer Pain Populations

PURPOSE

Pain is multi-dimensional and may be better addressed through a holistic, biopsychosocial approach. Massage therapy is commonly practiced among patients seeking pain management; however, its efficacy is unclear. This systematic review and meta-analysis is the first to rigorously assess the quality of massage therapy research and evidence for its efficacy in treating pain, function-related and health-related quality of life in cancer populations.

METHODS

Key databases were searched from inception through February 2014. Eligible randomized controlled trials were assessed for methodological quality using the SIGN 50 Checklist. Meta-analysis was applied at the outcome level. A diverse steering committee interpreted the results to develop recommendations.

RESULTS

Twelve high quality and four low quality studies were subsequently included in the review. Results demonstrate massage therapy is effective for treating pain compared to no treatment [standardized mean difference (SMD)  = -.20] and active (SMD = -0.55) comparators. Compared to active comparators, massage therapy was also found to be beneficial for treating fatigue (SMD = -1.06) and anxiety (SMD = -1.24).

CONCLUSION

Based on the evidence, weak recommendations are suggested for massage therapy, compared to an active comparator, for the treatment of pain, fatigue, and anxiety. No recommendations were suggested for massage therapy compared to no treatment or sham control based on the available literature to date. This review addresses massage therapy safety, research challenges, how to address identified research gaps, and necessary next steps for implementing massage therapy as a viable pain management option for cancer pain populations.

The Impact of Massage Therapy on Function in Pain Populations-A Systematic Review and Meta-Analysis of Randomized Controlled Trials: Part I, Patients Experiencing Pain in the General Population

PURPOSE

Pain is multi-dimensional and may be better addressed through a holistic, biopsychosocial approach. Massage therapy is commonly practiced among patients seeking pain management; however, its efficacy is unclear. This systematic review and meta-analysis is the first to rigorously assess the quality of massage therapy research and evidence for its efficacy in treating pain, function-related and health-related quality of life outcomes across all pain populations.

METHODS

Key databases were searched from inception through February 2014. Eligible randomized controlled trials were assessed for methodological quality using SIGN 50 Checklist. Meta-analysis was applied at the outcome level. A diverse steering committee interpreted the results to develop recommendations.

RESULTS

Sixty high quality and seven low quality studies were included in the review. Results demonstrate massage therapy effectively treats pain compared to sham [standardized mean difference (SMD) = -.44], no treatment (SMD = -1.14), and active (SMD = -0.26) comparators. Compared to active comparators, massage therapy was also beneficial for treating anxiety (SMD = -0.57) and health-related quality of life (SMD = 0.14).

CONCLUSION

Based on the evidence, massage therapy, compared to no treatment, should be strongly recommended as a pain management option. Massage therapy is weakly recommended for reducing pain, compared to other sham or active comparators, and improving mood and health-related quality of life, compared to other active comparators. Massage therapy safety, research challenges, how to address identified research gaps, and necessary next steps for implementing massage therapy as a viable pain management option are discussed.

SEaRCH™ expert panel process: streamlining the link between evidence and practice

BACKGROUND

With rising health care costs and the diversity of scientific and clinical information available to health care providers it is essential to have methodologies that synthesize and distill the quality of information and make it practical to clinicians, patients and policy makers. Too often research synthesis results in the statement that "more and better research is needed" or the conclusions are slanted toward the biases of one type of stakeholder. Such conclusions are discouraging to clinicians and patients who need better guidance on the decisions they make every day.

METHOD

Expert panels are one method for offering valuable insight into the scientific evidence and what experts believe about its application to a given clinical situation. However, with improper management their conclusions can end up being biased or even wrong. There are several types of expert panels, but two that have been extensively involved in bringing evidence to bear on clinical practice are consensus panels, and appropriateness panels. These types of panels are utilized by organizations such as the National Institutes of Health, the Institute of Medicine, RAND, and other organizations to provide clinical guidance. However, there is a need for a more cost effective and efficient approach in conducting these panels. In this paper we describe both types of expert panels and ways to adapt those models to form part of Samueli Institute's Scientific Evaluation and Research of Claims in Health Care (SEaRCH™) program.

DISCUSSION

Expert Panels provide evidence-based information to guide research, practice and health care decision making. The panel process used in SEaRCH seeks to customize, synthesize and streamline these methods. By making the process transparent the panel process informs decisions about clinical appropriateness and research agenda decisions.