A review of the reporting of web searching to identify studies for Cochrane systematic reviews

Expert searchers identified time, team, technology and tension as challenges when carrying out supplementary searches for systematic reviews: A thematic network analysis

BACKGROUND

Systematic reviews require detailed planning of complex processes which can present logistical challenges. Understanding these logistical challenges can help with planning and execution of tasks OBJECTIVES: To describe the perspectives of expert searchers on the main logistical challenges when carrying out supplementary searches for systematic reviews, in particular, forward citation searching and web searching.

METHODS

Qualitative interviews were undertaken with 15 experts on searching for studies for systematic reviews (e.g. information specialists) working in health and social care research settings. Interviews were undertaken by video-call between September 2020 and June 2021. Data analysis used thematic network analysis.

RESULTS

We identified three logistical challenges of using forward citation searching and web searching which were organised under the global theme of 'tension': time, team and technology. Several subthemes were identified which supported the organising themes, including allocating time, justifying time and keeping to time; reviewer expectations and contact with review teams; and access to resources and reference management.

CONCLUSION

Forward citation searching and web searching are logistically challenging search methods for a systematic review. An understanding of these challenges should encourage expert searchers and review teams to maintain open channels of communication, which should also facilitate improved working relationships.

Effectiveness of aerobic and resistance training on the motor symptoms in Parkinson's disease: Systematic review and network meta-analysis

Background/objectives

Aerobic and resistance training are common complementary therapies to improve motor symptoms in people with Parkinson's disease (PD), and there is still a lack of advice on which intensity and period of aerobic or resistance training is more appropriate for people with PD. Therefore, a network meta-analysis was conducted to assess the comparative efficacy of aerobic and resistance training of different intensities and cycles on motor symptoms in patients with Parkinson's disease.

Methods

Based on several biomedical databases, a search strategy system was conducted to retrieve randomized controlled trials (RCTs) without language restrictions. A network meta-analysis with a frequentist approach was conducted to estimate the efficacy and probability rankings of aerobic and resistance training on Parkinson's patients. What's more, a range of analyses and assessments, such as routine meta-analyses and risk of bias, were performed as well.

Results

Twenty trials with 719 patients evaluating 18 different therapies were identified. Through the Unified Parkinson's Disease Motor Rating Scale, (UPDRS III); 6-minute walk test, (6MWT); 10-meter walk test, (TWM); and time up and go (TUG) and Quality of Life Scale-39 (PDQ-39), to explore the effects of different intensity resistance and aerobic exercise on PD. As a result, short period high intensity resistance movement (standard mean difference (SMD) = -0.95, 95% confidence interval (CI) -1.68 to -0.22) had significantly decreased the Unified Parkinson's Disease Motor Rating Scale (UPDRS III). Short period high intensity resistance exercise showed similar superiority in other indices; also, aerobic and resistance training of different cycle intensities produced some efficacy in PD patients, both in direct and indirect comparisons.

Conclusion

For patients with moderate to mild Parkinson's symptoms, short periods high intensity resistance training may provide complementary therapy for PD, and aerobic or resistance training of varying intensity and periodicity may be recommended as exercise prescription for PD patients. However, more large scale and high quality clinical trials are needed to confirm the effectiveness of this exercise therapy in the future.

Systematic Review Registration

https://www.crd.york.ac.uk/PROSPERO/, identifier: CRD42022324824.

Current Evidence in the Management of Premalignant Cystic Lesions of the Pancreas in Patients Undergoing Liver Transplantation

ABSTRACT

Incidental premalignant pancreatic cystic lesions (pPCLs) are increasingly being detected in patients undergoing orthotopic liver transplantation (OLT). The impact of chronic immunosuppression upon pPCLs may elevate risk of progression to pancreatic cancer. This systematic review assesses prevalence, outcome, and management of pPCLs in patients undergoing OLT. Systematic literature searches were performed in accordance with Cochrane review guidelines. Data on 658 patients were identified from 13 articles. Median age was 59 years with a prevalence of 6.2%. Most studies focused on branch-duct intraductal papillary mucinous neoplasms. Average cyst size at diagnosis was 10.3 mm. Six patients (0.9%) underwent pancreatic resection, post-OLT, for suspected "worrisome features" on imaging. One death was due to pancreatic-related cancer, post-OLT. Based on the review, the authors suggest the following: (1) patients with pPCLs undergoing OLT, without "worrisome features," should be followed conservatively; (2) presence of pPCLs alone should not preclude eligibility for OLT, nor should chronic immunosuppression be altered; (3) follow-up should parallel standard approach applied in immunocompetent patients, as development of "worrisome features" of cancer is rare and does not appear to be hastened by immunosuppression; (4) resection is recommended for surgically fit patients without portal hypertension that develop "worrisome features."

Liver Transplantation in Locally Unresectable, Undifferentiated Embryonal Cell Sarcoma

Background.

Undifferentiated embryonal cell sarcoma (UESL) of the liver is the third most common malignant liver disease of childhood presenting as a rapidly enlarging intraabdominal mass. This systematic review explores the practicality of liver transplantation as a viable option in the treatment armamentarium for locally advanced undifferentiated embryonal cell sarcoma.

Methods.

A systematic review of the literature was performed using Medline and Embase, from inception of databases to December 31, 2018. Keywords and MeSH headings used were embryonal sarcoma, mesenchymal sarcoma, and liver transplant. Reviews and manuscripts with incomplete data were excluded.

Results.

Twenty-eight patients had orthotopic liver transplantation (OLT) as a curative treatment option. The median age at presentation was 8 and 27 years in the pediatric and adult population, respectively, with a similar male to female ratio. A majority of the patients presented with abdominal pain, palpable mass, and a normal alpha-feto-protein. The median tumor size was 15 cm mainly affecting the right lobe (62%) of the liver. Eighty-two percent of the patients underwent primary OLT and 5 patients had salvage OLT. One death (3.6%) was due to initial misdiagnosis and management for hepatoblastoma. Recurrence was noted in 7.1% of the population. The median follow-up was noted to be 28.5 months. The documented survival rate post-liver transplant for UESL was 96%.

Conclusions.

Based on available data and the very positive results therein, liver transplantation is a practical and justifiable use of a scarce resource as a treatment option for locally unresectable, undifferentiated embryonal cell sarcoma. The authors propose (accepting existence of different proposals) neoadjuvant therapy before curative resection, and if not achievable, then liver transplantation followed by adjuvant chemotherapy is an option for suitable candidates. For recurrent tumors after surgical resection, adjuvant therapy with salvage liver transplantation is an option.

Dementia Primary Prevention Policies and Strategies and Their Local Implementation: A Scoping Review Using England as a Case Study

Background:

Understanding the policy context and how policy is implemented at the local and clinical level is an important precursor to developing preventive strategies focusing on dementia risk reduction in primary healthcare settings.

Objective:

Using England as a case study, we review policies and strategies relevant to dementia prevention from the national to local level and how these are translated into primary healthcare services.

Methods:

We conducted a scoping review covering: 1) identification of national, regional, and local policies and strategies that include dementia prevention; 2) identification of national guidelines for implementing dementia prevention at the clinical level; and 3) evaluation of the implementation of these at the clinical level.

Results:

Dementia prevention is addressed in national policy, and this filters through to regional and local levels. Focus on dementia prevention is limited and variable. Reference to modifiable risk factors is associated with other non-communicable diseases, placing less emphasis on factors more dementia specific. Evidence of implementation of dementia prevention policies at the clinical level is limited and inconsistent. Available evidence suggests messages about dementia prevention may best be delivered through primary healthcare services such as the National Health Service (NHS) Health Check.

Conclusion:

The limitations identified in this review could be addressed through development of a national policy focused specifically on dementia prevention. This could provide a platform for increasing knowledge and understanding among the general population and healthcare professionals. It would be important for such a policy to cover the full range of modifiable risk factors relevant to dementia.

A Microsoft Excel Approach to Reduce Errors and Increase Efficiency in Systematic Searching

Developing a search strategy for a systematic review is a time-consuming process in which small errors around the formatting and compilation of terms can have large consequences. Microsoft Excel was identified as a potentially useful software to streamline the process and reduce manual errors. Ultimately a spreadsheet was created that largely automates the process of creating a single-line search string with correctly formatted terms, Boolean operators and parentheses.

What are Systematic Reviews?

Systematic reviews are a type of review that uses repeatable analytical methods to collect secondary data and analyse it. Systematic reviews are a type of evidence synthesis which formulate research questions that are broad or narrow in scope, and identify and synthesize data that directly relate to the systematic review question. While some people might associate ‘systematic review’ with 'meta-analysis', there are multiple kinds of review which can be defined as ‘systematic’ which do not involve a meta-analysis. Some systematic reviews critically appraise research studies, and synthesize findings qualitatively or quantitatively. Systematic reviews are often designed to provide an exhaustive summary of current evidence relevant to a research question. For example, systematic reviews of randomized controlled trials are an important way of informing evidence-based medicine, and a review of existing studies is often quicker and cheaper than embarking on a new study. While systematic reviews are often applied in the biomedical or healthcare context, they can be used in other areas where an assessment of a precisely defined subject would be helpful. Systematic reviews may examine clinical tests, public health interventions, environmental interventions, social interventions, adverse effects, qualitative evidence syntheses, methodological reviews, policy reviews, and economic evaluations. An understanding of systematic reviews and how to implement them in practice is highly recommended for professionals involved in the delivery of health care, public health and public policy.

The "realist search": A systematic scoping review of current practice and reporting

The requirement for literature searches that identify studies for inclusion in systematic reviews should be systematic, explicit, and reproducible extends, at least by implication, to other types of literature review. However, realist reviews commonly require literature searches that challenge systematic reporting; searches are iterative and involve multiple search strategies and approaches. Notwithstanding these challenges, reporting of the "realist search" can be structured to be transparent and to facilitate identification of innovative retrieval practices. Our six-component search framework consolidates and extends the structure advanced by Pawson, one of the originators of realist review: formulating the question, conducting the background search, searching for program theory, searching for empirical studies, searching to refine program theory and identifying relevant mid-range theory, and documenting and reporting the search process. This study reviews reports of search methods in 34 realist reviews published within the calendar year of 2016. Data from all eligible reviews were extracted from the search framework. Realist search reports poorly differentiate between the different search components. Review teams often conduct a single "big bang" multipurpose search to fulfill multiple functions within the review. However, it is acknowledged that realist searches are likely to be iterative and responsive to emergent data. Overall, the search for empirical studies appears most comprehensive in conduct and reporting detail. In contrast, searches to identify and refine program theory are poorly conducted, if at all, and poorly reported. Use of this framework offers greater transparency in conduct and reporting while preserving flexibility and methodological innovation.

Conduct and reporting of citation searching in Cochrane systematic reviews: A cross-sectional study

Background

The search for studies for a systematic review should be conducted systematically and reported transparently to facilitate reproduction. This study aimed to report on the conduct and reporting of backward citation searching (ie, checking reference lists) and forward citation searching in a cross section of Cochrane reviews. Citation searching uses the citation network surrounding a source study to identify additional studies.

Methods

Cochrane reviews were identified by searching the Cochrane Database of Systematic Reviews using the wildcard symbol and date limiting to the 3‐month period November 2016 to January 2017. Cochrane reviews thus identified were screened for mention of citation searching. Descriptive detail on the conduct and reporting of citation searching was captured in data extraction forms and described and evaluated.

Results

Two hundred fifteen Cochrane reviews were identified. One hundred seventy‐two reviews reported backward citation searching, and 18 reviews reported forward citation searching. Web of Science was the most frequently reported citation index. The studies used for backward citation searching consisted mainly of studies meeting the inclusion criteria. One‐third of reviews that reported forward citation searching used selected studies of importance. Reporting of citation searching was compliant with the Methodological Expectations of Cochrane Intervention Reviews (MECIR) standards, but full transparency requires additional detail that only a minority of reviews reported.

Conclusion

The conduct of backward citation searching was more uniform than forward citation searching. This might be due to lack of MECIR guidance for forward citation searching. Reporting was generally compliant with MECIR, but this is not always sufficient to ensure full transparency.

Bubble effect: including internet search engines in systematic reviews introduces selection bias and impedes scientific reproducibility

BACKGROUND

Using internet search engines (such as Google search) in systematic literature reviews is increasingly becoming a ubiquitous part of search methodology. In order to integrate the vast quantity of available knowledge, literature mostly focuses on systematic reviews, considered to be principal sources of scientific evidence at all practical levels. Any possible individual methodological flaws present in these systematic reviews have the potential to become systemic.

MAIN TEXT

This particular bias, that could be referred to as (re)search bubble effect, is introduced because of inherent, personalized nature of internet search engines that tailors results according to derived user preferences based on unreproducible criteria. In other words, internet search engines adjust their user's beliefs and attitudes, leading to the creation of a personalized (re)search bubble, including entries that have not been subjected to rigorous peer review process. The internet search engine algorithms are in a state of constant flux, producing differing results at any given moment, even if the query remains identical. There are many more subtle ways of introducing unwanted variations and synonyms of search queries that are used autonomously, detached from user insight and intent. Even the most well-known and respected systematic literature reviews do not seem immune to the negative implications of the search bubble effect, affecting reproducibility.

CONCLUSION

Although immensely useful and justified by the need for encompassing the entirety of knowledge, the practice of including internet search engines in systematic literature reviews is fundamentally irreconcilable with recent emphasis on scientific reproducibility and rigor, having a profound impact on the discussion of the limits of scientific epistemology. Scientific research that is not reproducible, may still be called science, but represents one that should be avoided. Our recommendation is to use internet search engines as an additional literature source, primarily in order to validate initial search strategies centered on bibliographic databases.