Retrieval of overviews of systematic reviews in MEDLINE was improved by the development of an objectively derived and validated search strategy

A global perspective of advanced practice nursing research: A review of systematic reviews

INTRODUCTION

The World Health Organization (WHO) called for the expansion of all nursing roles, including advanced practice nurses (APNs), nurse practitioners (NPs) and clinical nurse specialists (CNSs). A clearer understanding of the impact of these roles will inform global priorities for advanced practice nursing education, research, and policy.

OBJECTIVE

To identify gaps in advanced practice nursing research globally.

MATERIALS AND METHODS

A review of systematic reviews was conducted. We searched CINAHL, Embase, Global Health, Healthstar, PubMed, Medline, Cochrane Library, DARE, Joanna Briggs Institute EBP, and Web of Science from January 2011 onwards, with no restrictions on jurisdiction or language. Grey literature and hand searches of reference lists were undertaken. Review quality was assessed using the Critical Appraisal Skills Program (CASP). Study selection, data extraction and CASP assessments were done independently by two reviewers. We extracted study characteristics, country and outcome data. Data were summarized using narrative synthesis.

RESULTS

We screened 5840 articles and retained 117 systematic reviews, representing 38 countries. Most CASP criteria were met. However, study selection by two reviewers was done inconsistently and language and geographical restrictions were applied. We found highly consistent evidence that APN, NP and CNS care was equal or superior to the comparator (e.g., physicians) for 29 indicator categories across a wide range of clinical settings, patient populations and acuity levels. Mixed findings were noted for quality of life, consultations, costs, emergency room visits, and health care service delivery where some studies favoured the control groups. No indicator consistently favoured the control group. There is emerging research related to Artificial Intelligence (AI).

CONCLUSION

There is a large body of advanced practice nursing research globally, but several WHO regions are underrepresented. Identified research gaps include AI, interprofessional team functioning, workload, and patients and families as partners in healthcare.

PROSPERO REGISTRATION NUMBER

CRD42021278532.

Development of a search filter to retrieve reports of interrupted time series studies from MEDLINE and PubMed

BACKGROUND

Interrupted time series (ITS) studies contribute importantly to systematic reviews of population-level interventions. We aimed to develop and validate search filters to retrieve ITS studies in MEDLINE and PubMed.

METHODS

A total of 1017 known ITS studies (published 2013-2017) were analysed using text mining to generate candidate terms. A control set of 1398 time-series studies were used to select differentiating terms. Various combinations of candidate terms were iteratively tested to generate three search filters. An independent set of 700 ITS studies was used to validate the filters' sensitivities. The filters were test-run in Ovid MEDLINE and the records randomly screened for ITS studies to determine their precision. Finally, all MEDLINE filters were translated to PubMed format and their sensitivities in PubMed were estimated.

RESULTS

Three search filters were created in MEDLINE: a precision-maximising filter with high precision (78%; 95% CI 74%-82%) but moderate sensitivity (63%; 59%-66%), most appropriate when there are limited resources to screen studies; a sensitivity-and-precision-maximising filter with higher sensitivity (81%; 77%-83%) but lower precision (32%; 28%-36%), providing a balance between expediency and comprehensiveness; and a sensitivity-maximising filter with high sensitivity (88%; 85%-90%) but likely very low precision, useful when combined with specific content terms. Similar sensitivity estimates were found for PubMed versions.

CONCLUSION

Our filters strike different balances between comprehensiveness and screening workload and suit different research needs. Retrieval of ITS studies would be improved if authors identified the ITS design in the titles.

Only half of the authors of overviews of exercise-related interventions use some strategy to manage overlapping primary studies-a metaresearch study

OBJECTIVES

The conduct of systematic reviews (SRs) and overviews share several similarities. However, because the unit of analysis for overviews is the SRs, there are some unique challenges. One of the most critical issues to manage when conducting an overview is the overlap of data across the primary studies included in the SRs. This metaresearch study aimed to describe the frequency of strategies to manage the overlap in overviews of exercise-related interventions.

STUDY DESIGN AND SETTING

A systematic search in MEDLINE (Ovid), Embase (Ovid), Cochrane Library, Epistemonikos, and other sources was conducted from inception to June 2022. We included overviews of SRs that considered primary studies and evaluated the effectiveness of exercise-related interventions for any health condition. The overviews were screened by two authors independently, and the extraction was performed by one author and checked by a second. We found 353 overviews published between 2005 and 2022 that met the inclusion criteria.

RESULTS

One hundred and sixty-four overviews (46%) used at least one strategy to visualize, quantify, or resolve overlap, with a matrix (32/164; 20%), absolute frequency (34/164; 21%), and authors' algorithms (24/164; 15%) being the most used methods, respectively. From 2016 onwards, there has been a trend toward increasing the use of some strategies to manage overlap. Of the 108 overviews that used some strategy to resolve the overlap, ie, avoiding double or multiple counting of primary study data, 79 (73%) succeeded. In overviews where no strategies to manage overlap were reported (n = 189/353; 54%), 16 overview authors (8%) recognized this as a study limitation.

CONCLUSION

Although there is a trend toward increasing its use, only half of the authors of the overviews of exercise-related interventions used a strategy to visualize, quantify, or resolve overlap in the primary studies' data. In the future, authors should report such strategies to communicate more valid results.

"Over-reviewing" of research? An analysis of orthodontic reviews

INTRODUCTION

Research overviews may be undertaken to identify gaps in the literature, evaluate existing systematic reviews (SRs), and summarize evidence. This paper aims to profile overviews that have been conducted in orthodontics and related interventions since 2012 and to evaluate the degree of overlap among these overviews.

METHODS

Overviews published between January 1, 2012 and June 20, 2023 were identified using an electronic search involving Google Scholar and PubMed. A descriptive summary was produced, and citation matrices were used to evaluate the percentage of overlap between overviews using corrected covered area and covered area. This was classified as slight, moderate, high, or very high.

RESULTS

A total of 35 overviews were identified across a wide range of topics. Eight overviews included <10 SRs; 21 had 10-20 SRs; and 6 included >20 SRs (median no. of SRs per overview, 15; range, 3-62). Meta-analysis was conducted in only 5 overviews. Overlap between overviews on the same topic ranged from slight (2.7%) to very high (53.8%).

CONCLUSIONS

Almost all overview topics address treatments and their effects, with a wide variation in the number and quality of SRs included. There is considerable overlap in some orthodontic overviews, suggesting unnecessary duplication and research waste. Researchers should be encouraged to focus on primary data collection to add more high-quality data to SRs, which will ultimately enhance the yield from secondary and tertiary orthodontic research.

An Umbrella Review of Self-Management Interventions for Health Conditions With Symptom Overlap With Traumatic Brain Injury

OBJECTIVE

To synthesize evidence for the effectiveness of self-management interventions for chronic health conditions that have symptom overlap with traumatic brain injury (TBI) in order to extract recommendations for self-management intervention in persons with TBI.

DESIGN

An umbrella review of existing systematic reviews and/or meta-analyses of randomized controlled trials or nonrandomized studies targeting self-management of chronic conditions and specific outcomes relevant to persons with TBI.

METHOD

A comprehensive literature search of 5 databases was conducted using PRISMA guidelines. Two independent reviewers conducted screening and data extraction using the Covidence web-based review platform. Quality assessment was conducted using criteria adapted from the Assessing the Methodological Quality of Systematic Reviews-2 (AMSTAR-2).

RESULTS

A total of 26 reviews met the inclusion criteria, covering a range of chronic conditions and a range of outcomes. Seven reviews were of moderate or high quality and focused on self-management in persons with stroke, chronic pain, and psychiatric disorders with psychotic features. Self-management interventions were found to have positive effects on quality of life, self-efficacy, hope, reduction of disability, pain, relapse and rehospitalization rates, psychiatric symptoms, and occupational and social functioning.

CONCLUSIONS

Findings are encouraging with regard to the effectiveness of self-management interventions in patients with symptoms similar to those of TBI. However, reviews did not address adaptation of self-management interventions for those with cognitive deficits or for populations with greater vulnerabilities, such as low education and older adults. Adaptations for TBI and its intersection with these special groups may be needed.

Search strategies (filters) to identify systematic reviews in MEDLINE and Embase

BACKGROUND

Bibliographic databases provide access to an international body of scientific literature in health and medical sciences. Systematic reviews are an important source of evidence for clinicians, researchers, consumers, and policymakers as they address a specific health-related question and use explicit methods to identify, appraise and synthesize evidence from which conclusions can be drawn and decisions made. Methodological search filters help database end-users search the literature effectively with different levels of sensitivity and specificity. These filters have been developed for various study designs and have been found to be particularly useful for intervention studies. Other filters have been developed for finding systematic reviews. Considering the variety and number of available search filters for systematic reviews, there is a need for a review of them in order to provide evidence about their retrieval properties at the time they were developed.

OBJECTIVES

To review systematically empirical studies that report the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE and Embase.

SEARCH METHODS

We searched the following databases from inception to January 2023: MEDLINE, Embase, PsycINFO; Library, Information Science & Technology Abstracts (LISTA) and Science Citation Index (Web of Science).

SELECTION CRITERIA

We included studies if one of their primary objectives is the development, evaluation, or comparison of a search filter that could be used to retrieve systematic reviews on MEDLINE, Embase, or both.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data using a pre-specified and piloted data extraction form using InterTASC Information Specialist Subgroup (ISSG) Search Filter Evaluation Checklist.

MAIN RESULTS

We identified eight studies that developed filters for MEDLINE and three studies that developed filters for Embase. Most studies are very old and some were limited to systematic reviews in specific clinical areas. Six included studies reported the sensitivity of their developed filter. Seven studies reported precision and six studies reported specificity. Only one study reported the number needed to read and positive predictive value. None of the filters were designed to differentiate systematic reviews on the basis of their methodological quality. For MEDLINE, all filters showed similar sensitivity and precision, and one filter showed higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited study reports that may affect accuracy assessments. The report of these studies had some limitations, and the assessments of their accuracy may suffer from indirectness, considering that they were mostly developed before the release of the PRISMA 2009 statement or due to their limited scope in the selection of systematic review topics. Search filters for MEDLINE Three studies produced filters with sensitivity > 90% with variable degrees of precision, and only one of them was developed and validated in a gold-standard database, which allowed the calculation of specificity. The other two search filters had lower levels of sensitivity. One of these produced a filter with higher levels of specificity (> 90%). All filters showed similar sensitivity and precision in the external validation, except for one which was not externally validated and another one which was conceptually derived and only externally validated. Search filters for Embase We identified three studies that developed filters for this database. One of these studies developed filters with variable sensitivity and precision, including highly sensitive strategies (> 90%); however, it was not externally validated. The other study produced a filter with a lower sensitivity (72.7%) but high specificity (99.1%) with a similar performance in the external validation.

AUTHORS' CONCLUSIONS

Studies reporting the development, evaluation, or comparison of search filters to retrieve reports of systematic reviews in MEDLINE showed similar sensitivity and precision, with one filter showing higher levels of specificity. For Embase, filters showed variable sensitivity and precision, with limited information about how the filter was produced, which leaves us uncertain about their performance assessments. Newer filters had limitations in their methods or scope, including very focused subject topics for their gold standards, limiting their applicability across other topics. Our findings highlight that consensus guidance on the conduct of search filters and standardized reporting of search filters are needed, as we found highly heterogeneous development methods, accuracy assessments and outcome selection. New strategies adaptable across interfaces could enhance their usability. Moreover, the performance of existing filters needs to be evaluated in light of the impact of reporting guidelines, including the PRISMA 2009, on how systematic reviews are reported. Finally, future filter developments should also consider comparing the filters against a common reference set to establish comparative performance and assess the quality of systematic reviews retrieved by strategies.

Identifying indicators sensitive to primary healthcare nurse practitioner practice: A review of systematic reviews

AIM

To identify indicators sensitive to the practice of primary healthcare nurse practitioners (PHCNPs).

MATERIALS AND METHODS

A review of systematic reviews was undertaken to identify indicators sensitive to PHCNP practice. Published and grey literature was searched from January 1, 2010 to December 2, 2022. Titles/abstracts (n = 4251) and full texts (n = 365) were screened independently by two reviewers, with a third acting as a tie-breaker. Reference lists of relevant publications were reviewed. Risk of bias was examined independently by two reviewers using AMSTAR-2. Data were extracted by one reviewer and verified by a second reviewer to describe study characteristics, indicators, and results. Indicators were recoded into categories. Findings were summarized using narrative synthesis.

RESULTS

Forty-four systematic reviews were retained including 271 indicators that were recoded into 26 indicator categories at the patient, provider and health system levels. Nineteen reviews were assessed to be at low risk of bias. Patient indicator categories included activities of daily living, adaptation to health conditions, clinical conditions, diagnosis, education-patient, mortality, patient adherence, quality of life, satisfaction, and signs and symptoms. Provider indicator categories included adherence to best practice-providers, education-providers, illness prevention, interprofessional team functioning, and prescribing. Health system indicator categories included access to care, consultations, costs, emergency room visits, healthcare service delivery, hospitalizations, length of stay, patient safety, quality of care, scope of practice, and wait times.

DISCUSSION

Equal to improved care for almost all indicators was found consistently for the PHCNP group. Very few indicators favoured the control group. No indicator was identified for high/low fidelity simulation, cultural safety and cultural sensitivity with people in vulnerable situations or Indigenous Peoples.

CONCLUSION

This review of systematic reviews identified patient, provider and health system indicators sensitive to PHCNP practice. The findings help clarify how PHCNPs contribute to care outcomes.

PROSPERO REGISTRATION NUMBER

CRD42020198182.

Mapping and systematic appraisal of umbrella reviews in epidemiological research: a protocol for a meta-epidemiological study

INTRODUCTION

Umbrella review is one of the terms used to describe an overview of systematic reviews. During the last years, a rapid increase in the number of umbrella reviews on epidemiological studies has been observed, but there is no systematic assessment of their methodological and reporting characteristics. Our study aims to fill this gap by performing a systematic mapping of umbrella reviews in epidemiological research.

METHODS

We will perform a meta-epidemiological study including a systematic review in MEDLINE and EMBASE to identify all the umbrella reviews that focused on systematic reviews of epidemiological studies and were published from inception until December 31, 2022. We will consider eligible any research article which was designed as an umbrella review and summarized systematic reviews and meta-analyses of epidemiological studies. From each eligible article, we will extract information about the research topic, the methodological characteristics, and the reporting characteristics. We will examine whether the umbrella reviews assessed the strength of the available evidence and the rigor of the included systematic reviews. We will also examine whether these characteristics change across time.

DISCUSSION

Our study will systematically appraise the methodological and reporting characteristics of published umbrella reviews in epidemiological literature. The findings of our study can be used to improve the design and conduct of future umbrella reviews, to derive a standardized set of reporting and methodological guidelines for umbrella reviews, and to allow further meta-epidemiological work.

SYSTEMATIC REVIEW REGISTRATION

osf.io/sxzc6.

A methodological review finds mismatch between overall and pairwise overlap analysis in a sample of overviews

OBJECTIVES

Overlap of primary studies is a key methodological challenge for overviews. There are limited reports of methods used to address overlap, and there is no detailed assessment of the corrected covered area (CCA) of a representative sample of overviews. To describe the approaches used to address overlap, and to estimate the overall and pairwise CCA.

METHODS

We searched PubMed for overviews published in 2018. Two authors conducted the screening process. We described the strategy used for assessing overlap, and calculated overall and pairwise CCA for each overview.

RESULTS

We analyzed a random sample of 30 out of 89 eligible articles. Eleven did not address the overlap. Of the remainder, most frequent strategies were visual assessment and discussion of overlap as a limitation. Median overall CCA among the included overviews was 6.7%. The pairwise analysis showed that 52.8% of SR pairs had slight overlap, while 28.3% had very high overlap.

CONCLUSION

Reported strategies for addressing overlap vary considerably among overview authors. The pairwise approach for assessing the CCA revealed highly overlapped pairs of SRs in overviews with overall slight overlap and vice versa. We encourage authors to complement the overall CCA assessment with a pairwise approach.

Overviews of reviews in the cardiovascular field underreported critical methodological and transparency characteristics: a methodological study based on the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement

OBJECTIVES

This study aimed to evaluate the epidemiology, reporting characteristics, and adherence to the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement of overviews of reviews (overviews) of interventions in the cardiovascular field.

STUDY DESIGN AND SETTING

MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched from January 1, 2000, to October 15, 2020. An updated search was performed in MEDLINE, Epistemonikos, and Google Scholar up to August 25, 2022. Overviews of interventions published in English and primarily considering populations, interventions, and outcomes pertinent to the cardiovascular field were eligible. Study selection, data extraction, and PRIOR adherence assessment were performed by two authors independently.

RESULTS

We analyzed 96 overviews. Almost half (43/96 [45%]) were published between 2020 and 2022 and contained a median of 15 systematic reviews (SRs) (interquartile range, 9-28). The commonest title terminology was "overview of (systematic) reviews" (38/96 [40%]). Methods for handling SR overlap were reported in 24/96 (25%), methods for assessing primary study overlap in 18/96 (19%), handling of discrepant data in 11/96 (11%), and methods for methodological quality or risk of bias assessment of the primary studies within SRs in 23/96 (24%). Authors included data sharing statements in 28/96 (29%), complete funding disclosure in 43/96 (45%), protocol registration in 43/96 (45%), and conflict of interest statement in 82/96 (85%) overviews.

CONCLUSION

Insufficient reporting was identified in methodological characteristics unique in overviews' conduct and most transparency markers. Adoption of PRIOR from the research community could ameliorate overviews' reporting.

Strategies used to manage overlap of primary study data by exercise-related overviews: protocol for a systematic methodological review

INTRODUCTION

One of the most conflicting methodological issues when conducting an overview is the overlap of primary studies across systematic reviews (SRs). Overlap in the pooled effect estimates across SRs may lead to overly precise effect estimates in the overview. SRs that focus on exercise-related interventions are often included in overviews aimed at grouping and determining the effectiveness of various interventions for managing specific health conditions. The aim of this systematic methodological review is to describe the strategies used by authors of overviews focusing on exercise-related interventions to manage the overlap of primary studies.

METHODS AND ANALYSIS

A comprehensive search strategy has been developed for different databases and their platforms. The databases to be consulted will be MEDLINE (Ovid), Embase (Ovid), The Cochrane Database of Systematic Reviews (Cochrane Library) and Epistemonikos. Two reviewers will independently screen the records identified through the search strategy and extract the information from the included overviews. The frequency and the type of overlap management strategies of the primary studies included in the SRs will be considered as the main outcome. In addition, the recognition of the lack of use of any overlap management strategy and the congruence between planning and conducting the overview focusing on overlap management strategies will be assessed. A subgroup analysis will be carried out according to the journal impact factor, year of publication and compliance with the Preferred Reporting Items for Overviews of Reviews statement.

ETHICS AND DISSEMINATION

This study will not involve human subjects and therefore does not require ethics committee approval. However, the conduct and reporting of the findings of this review will be conducted in a rigorous, systematic and transparent manner, which relates to research ethics.The findings of this review will be presented at scientific conferences and published as one or more studies in peer-review scientific journals related to rehabilitation or research methods.

Types and associated methodologies of overviews of reviews in health care: a methodological study with published examples

OBJECTIVES

To provide a descriptive insight into the different types of research questions/objectives and associated methodologies of overviews of reviews, supplemented by representative examples from the health care literature.

STUDY DESIGN AND SETTING

We searched in methodological articles for information on types and methodologies used in overviews and we explored the typology of reviews to identify similar types in literature of overviews. We categorized the types of overviews based on the research question/objective and the methodological approach used. Indicative examples for each category were selected from a sample of 2,121 overviews that were retrieved between 2000 and 2022 from MEDLINE, Scopus, and Cochrane Database of Systematic Reviews.

RESULTS

Based on type of research question, overviews were classified as overviews of reviews of interventions, associations, prediction, diagnostic accuracy, prevalence/incidence, experiences/views, economic evaluation, and measurement properties. Based on the methodological approach, we identified a variety of methods (systematic, living, rapid, scoping, evidence mapping, framework, and methodological) used in overviews.

CONCLUSION

The proposed classification and examples provide an essential starting point for future theory-building research on typologies and study designs of overviews of reviews. It is important for methodologists to make vigorous effort to create consensus-based methodological and reporting guidelines to cover these diverse types and key methodological challenges.

A global perspective of advanced practice nursing research: A review of systematic reviews protocol

INTRODUCTION

In 2020, the World Health Organization called for the expansion and greater recognition of all nursing roles, including advanced practice nurses (APNs), to better meet patient care needs. As defined by the International Council of Nurses (ICN), the two most common APN roles include nurse practitioners (NPs) and clinical nurse specialists (CNSs). They help ensure care to communities as well as patients and families with acute, chronic or complex conditions. Moreover, APNs support providers to deliver high quality care and improve access to services. Currently, there is much variability in the use of advanced practice nursing roles globally. A clearer understanding of the roles that are in place across the globe, and how they are being used will support greater role harmonization, and inform global priorities for advanced practice nursing education, research, and policy reform.

OBJECTIVE

To identify current gaps in advanced practice nursing research globally.

MATERIALS AND METHODS

This review of systematic reviews will provide a description of the current state of the research, including gaps, on advanced practice nursing globally. We will include reviews that examine APNs, NPs or CNSs using recognized role definitions. We will search the CINAHL, EMBASE, Global Health, HealthStar, PubMed, Medline, Cochrane Library Database of Systematic Reviews and Controlled Trials Register, Database of Abstracts of Reviews of Effects, Joanna Briggs Institute, and Web of Science electronic databases for reviews published from January 2011 onwards, with no restrictions on jurisdiction or language. We will search the grey literature and hand search the reference lists of all relevant reviews to identify additional studies. We will extract country, patient, provider, health system, educational, and policy/scope of practice data. We will assess the quality of each included review using the CASP criteria, and summarize their findings. This review of systematic reviews protocol was developed following the PRISMA-P recommendations.

PROSPERO REGISTRATION NUMBER

CRD42021278532.

A review found heterogeneous approaches and insufficient reporting in overviews on adverse events

OBJECTIVES

To investigate reporting and methodological characteristics of overviews on adverse (drug-associated) events (AEs) of pharmacological interventions.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews from inception to May 17, 2021 for overviews exclusively investigating AEs of pharmacological interventions. We extracted general, reporting, and methodological characteristics and analyzed data descriptively.

RESULTS

We included 27 overviews, 70% of which were published in 2016 or later. The most common nomenclature in the title was "overview" (56%), followed by "umbrella review" (26%). The median number of included systematic reviews (SRs) in each overview was 15 (interquartile range 7-34). Study selection methods were reported in 52%, methods for data extraction in 67%, and methods for critical appraisal in 63% of overviews. An assessment of methodological quality of included SRs was performed in 70% of overviews. Only 22% of overviews reported strategies for dealing with overlapping SRs. An assessment of the certainty of the evidence was performed in 33% of overviews.

CONCLUSION

To ensure methodological rigor, authors of overviews on AEs should follow available guidance for the conduct and reporting of overviews.

Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement

OBJECTIVE

To develop a reporting guideline for overviews of reviews of healthcare interventions.

DESIGN

Development of the preferred reporting items for overviews of reviews (PRIOR) statement.

PARTICIPANTS

Core team (seven individuals) led day-to-day operations, and an expert advisory group (three individuals) provided methodological advice. A panel of 100 experts (authors, editors, readers including members of the public or patients) was invited to participate in a modified Delphi exercise. 11 expert panellists (chosen on the basis of expertise, and representing relevant stakeholder groups) were invited to take part in a virtual face-to-face meeting to reach agreement (≥70%) on final checklist items. 21 authors of recently published overviews were invited to pilot test the checklist.

SETTING

International consensus.

INTERVENTION

Four stage process established by the EQUATOR Network for developing reporting guidelines in health research: project launch (establish a core team and expert advisory group, register intent), evidence reviews (systematic review of published overviews to describe reporting quality, scoping review of methodological guidance and author reported challenges related to undertaking overviews of reviews), modified Delphi exercise (two online Delphi surveys to reach agreement (≥70%) on relevant reporting items followed by a virtual face-to-face meeting), and development of the reporting guideline.

RESULTS

From the evidence reviews, we drafted an initial list of 47 potentially relevant reporting items. An international group of 52 experts participated in the first Delphi survey (52% participation rate); agreement was reached for inclusion of 43 (91%) items. 44 experts (85% retention rate) completed the second Delphi survey, which included the four items lacking agreement from the first survey and five new items based on respondent comments. During the second round, agreement was not reached for the inclusion or exclusion of the nine remaining items. 19 individuals (6 core team and 3 expert advisory group members, and 10 expert panellists) attended the virtual face-to-face meeting. Among the nine items discussed, high agreement was reached for the inclusion of three and exclusion of six. Six authors participated in pilot testing, resulting in minor wording changes. The final checklist includes 27 main items (with 19 sub-items) across all stages of an overview of reviews.

CONCLUSIONS

PRIOR fills an important gap in reporting guidance for overviews of reviews of healthcare interventions. The checklist, along with rationale and example for each item, provides guidance for authors that will facilitate complete and transparent reporting. This will allow readers to assess the methods used in overviews of reviews of healthcare interventions and understand the trustworthiness and applicability of their findings.

Identifying and addressing conflicting results across multiple discordant systematic reviews on the same question: protocol for a replication study of the Jadad algorithm

INTRODUCTION

An increasing growth of systematic reviews (SRs) presents notable challenges for decision-makers seeking to answer clinical questions. In 1997, an algorithm was created by Jadad to assess discordance in results across SRs on the same question. Our study aims to (1) replicate assessments done in a sample of studies using the Jadad algorithm to determine if the same SR would have been chosen, (2) evaluate the Jadad algorithm in terms of utility, efficiency and comprehensiveness, and (3) describe how authors address discordance in results across multiple SRs.

METHODS AND ANALYSIS

We will use a database of 1218 overviews (2000-2020) created from a bibliometric study as the basis of our search for studies assessing discordance (called discordant reviews). This bibliometric study searched MEDLINE (Ovid), Epistemonikos and Cochrane Database of Systematic Reviews for overviews. We will include any study using Jadad (1997) or another method to assess discordance. The first 30 studies screened at the full-text stage by two independent reviewers will be included. We will replicate the authors' Jadad assessments. We will compare our outcomes qualitatively and evaluate the differences between our Jadad assessment of discordance and the authors' assessment.

ETHICS AND DISSEMINATION

No ethics approval was required as no human subjects were involved. In addition to publishing in an open-access journal, we will disseminate evidence summaries through formal and informal conferences, academic websites, and across social media platforms. This is the first study to comprehensively evaluate and replicate Jadad algorithm assessments of discordance across multiple SRs.

Preconception exposures and adverse pregnancy, birth and postpartum outcomes: Umbrella review of systematic reviews

BACKGROUND

Preconception exposures have been associated with adverse pregnancy, birth and postpartum outcomes. However, the reports, statements and guidelines of national and international health organisations vary in what they recommend individuals should monitor, avoid, reduce or practise in the preconception period.

OBJECTIVES

To synthesise and evaluate the evidence across systematic reviews for associations between exposures before conception and adverse pregnancy, birth and postpartum outcomes.

DATA SOURCES

MEDLINE, Embase, Epistemonikos (to May 2020) and reference lists of included reviews, without language or date restrictions.

STUDY SELECTION, DATA EXTRACTION AND SYNTHESIS

Systematic literature reviews of observational and/or interventional studies reporting associations between preconception exposures in women and/or men of reproductive age and pregnancy, birth or postpartum health outcomes were included. The methodological quality of reviews and the certainty of the evidence underlying each exposure-outcome association were assessed using AMSTAR 2 and the GRADE approach.

RESULTS

We identified 53 eligible reviews reporting 205 unique exposure-outcome associations. Methodological quality was generally low with only two reviews rated as 'high' quality and two as 'moderate'. We found high-certainty, randomised trial evidence that maternal folate supplementation reduces the risk of neural tube defects and anomaly-related terminations. Moderate-certainty, observational evidence was found that maternal physical activity is associated with reduced risk of pre-eclampsia and gestational diabetes, and that paternal age of ≥40 years and maternal body mass index (BMI) and interpregnancy weight gain are associated with increased risk of various adverse pregnancy and birth outcomes. Low- and very low-certainty evidence was found for other associations.

CONCLUSIONS

Clinicians and policymakers can be confident that maternal folate supplementation should be encouraged during the preconception period. There is moderate certainty in the evidence base that maternal physical activity, BMI and interpregnancy weight gain and advanced paternal age are important preconception considerations. High-quality research is required to better understand other exposure-outcome associations.

A new taxonomy was developed for overlap across 'Overviews of systematic reviews': a meta-research study of research waste

Multiple ‘overviews of reviews’ conducted on the same topic (“overlapping overviews”) represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesized reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD‐10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD‐10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time, and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied, which do not need to be duplicated.

Bibliometric study of 'overviews of systematic reviews' of health interventions: Evaluation of prevalence, citation and journal impact factor

Overviews synthesising the results of multiple systematic reviews help inform evidence-based clinical practice. In this first of two companion papers, we evaluate the bibliometrics of overviews, including their prevalence and factors affecting citation rates and journal impact factor (JIF). We searched MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews (CDSR). We included overviews that: (a) synthesised reviews, (b) conducted a systematic search, (c) had a methods section and (d) examined a healthcare intervention. Multivariable regression was conducted to determine the association between citation density, JIF and six predictor variables. We found 1218 overviews published from 2000 to 2020; the majority (73%) were published in the most recent 5-year period. We extracted a selection of these overviews (n = 541; 44%) dated from 2000 to 2018. The 541 overviews were published in 307 journals; CDSR (8%), PLOS ONE (3%) and Sao Paulo Medical Journal (2%) were the most prevalent. The majority (70%) were published in journals with impact factors between 0.05 and 3.97. We found a mean citation count of 10 overviews per year, published in journals with a mean JIF of 4.4. In multivariable analysis, overviews with a high number of citations and JIFs had more authors, larger sample sizes, were open access and reported the funding source. An eightfold increase in the number of overviews was found between 2009 and 2020. We identified 332 overviews published in 2020, which is equivalent to one overview published per day. Overviews perform above average for the journals in which they publish.

Development of an efficient search filter to retrieve systematic reviews from PubMed

Objective:

Locating systematic reviews is essential for clinicians and researchers when creating or updating reviews and for decision-making in health care. This study aimed to develop a search filter for retrieving systematic reviews that improves upon the performance of the PubMed systematic review search filter.

Methods:

Search terms were identified from abstracts of reviews published in Cochrane Database of Systematic Reviews and the titles of articles indexed as systematic reviews in PubMed. Both the precision of the candidate terms and the number of systematic reviews retrieved from PubMed were evaluated after excluding the subset of articles retrieved by the PubMed systematic review filter. Terms that achieved a precision greater than 70% and relevant publication types indexed with MeSH terms were included in the filter search strategy.

Results:

The search strategy used in our filter added specific terms not included in PubMed's systematic review filter and achieved a 61.3% increase in the number of retrieved articles that are potential systematic reviews. Moreover, it achieved an average precision that is likely greater than 80%.

Conclusions:

The developed search filter will enable users to identify more systematic reviews from PubMed than the PubMed systematic review filter with high precision.

Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews

BACKGROUND

Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap.

METHODS

We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use.

RESULTS

Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview - the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview.

CONCLUSIONS

Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.

Global mapping of overviews of systematic reviews in healthcare published between 2000 and 2020: a bibliometric analysis

OBJECTIVE

To conduct a bibliometric analysis using a large sample of overviews of systematic reviews (OoSRs) and reveal research trends and areas of interest about these studies.

STUDY DESIGN AND SETTING

We searched MEDLINE, Scopus and Cochrane Database of Systematic Reviews from 1/1/2000 to 15/10/2020. We used Scopus meta-data and two authors recorded supplementary information independently. We summarized the data using frequencies with percentages.

RESULTS

A total of 1558 studies were considered eligible for analysis. We found that the publications have been increasing yearly and their nomenclature was not uniform (the most frequent label in the title was "overview of systematic reviews"). The largest number of papers and the most cited ones were published by corresponding authors from the UK. The publications were distributed across 737 scholarly journals and many of them were published in the field of complementary/alternative medicine, psychiatry/psychology, nutrition/dietetics, and pediatrics. The co-authorship analysis revealed collaborations among countries. The most common clinical conditions were depression, diabetes, cancer, dementia, pain, cardiovascular disease, stroke, obesity, and schizophrenia.

CONCLUSION

OoSRs have recently become a popular approach of evidence synthesis. International collaborations between overview authors from countries with increased research productivity and countries with less research activity should be encouraged.

Development and application of an outcome-centric approach for conducting overviews of reviews

There are gaps in current guidance concerning how to conduct overviews of systematic reviews in an outcome-centric manner. Herein we summarize the methods and lessons learned from conducting 4 outcome-centric overviews to help inform the Canadian 24-Hour Movement Guidelines for Adults aged 18–64 years and Adults aged 65 years or older on the topics of resistance training, balance and functional training, sedentary behaviour, sleep duration. We defined “critical” and “important” outcomes a priori. We used AMSTAR 2 to assess review quality and sought 1 systematic review per outcome. If multiple reviews were required to address subgroups for an outcome, we calculated the corrected covered area (CCA) to quantify overlap. We report our methodology in a PRISMA table. Across the 4 overviews, authors reviewed 1110 full texts; 45 were retained (low to high quality per AMSTAR 2), representing 950 primary studies, enrolling over 5 385 500 participants. Of 46 outcomes, we identified data for 35. Nineteen outcomes required >1 review (CCA range: 0% to 71.4%). Our outcome-centric overviews addressed unique aspects of overviews, including selection and quality assessment of included reviews, and overlap. Lessons learned included consistent application of methodological principles to minimize bias and optimize reporting transparency.

Novelty Overviews of reviews synthesize systematic reviews in a rigorous and transparent manner. Outcome-centric systematic reviews assess the quality of evidence for primary studies contributing to an outcome. This manuscript describes the development and application of extending the concept of outcome-centric systematic reviews to the design and conduct of outcome-centric overviews.

Does health and social care provision for the community dwelling older population help to reduce unplanned secondary care, support timely discharge and improve patient well-being? A mixed method meta-review of systematic reviews

Background: This study aimed to identify and examine systematic review evidence of health and social care interventions for the community-dwelling older population regarding unplanned hospital admissions, timely hospital discharge and patient well-being. Methods: A meta-review was conducted using Joanna Briggs and PRISMA guidance. A search strategy was developed: eight bibliographic medical and social science databases were searched, and references of included studies checked. Searches were restricted to OECD countries and to systematic reviews published between January 2013-March 2018. Data extraction and quality appraisal was undertaken by one reviewer with a random sample screened independently by two others. Results: Searches retrieved 21,233 records; using data mining techniques, we identified 8,720 reviews. Following title and abstract and full-paper screening, 71 systematic reviews were included: 62 quantitative, seven qualitative and two mixed methods reviews. There were 52 reviews concerned with healthcare interventions and 19 reviews concerned with social care interventions. This meta-review summarises the evidence and evidence gaps of nine broad types of health and social care interventions. It scrutinises the presence of research in combined health and social care provision, finding it lacking in both definition and detail given. This meta-review debates the overlap of some of the person-centred support provided by community health and social care provision. Research recommendations have been generated by this process for both primary and secondary research. Finally, it proposes that research recommendations can be delivered on an ongoing basis if meta-reviews are conducted as living systematic reviews. Conclusions: This meta-review provides evidence of the effect of health and social care interventions for the community-dwelling older population and identification of evidence gaps. It highlights the lack of evidence for combined health and social care interventions and for the impact of social care interventions on health care outcomes. Registration: PROSPERO ID CRD42018087534; registered on 15 March 2018.

Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks

BACKGROUND

A standard or consensus definition of a systematic review does not exist. Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition.

METHODS

We included overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks. We extracted the definitions of SRs, as well as the inclusion and exclusion criteria that could indicate which definition of a SR the authors used. We extracted individual elements of SR definitions, categorised and quantified them.

RESULTS

Among the 535 analysed sources of information, 188 (35%) provided a definition of a SR. The most commonly used reference points for the definitions of SRs were Cochrane and the PRISMA statement. We found 188 different elements of SR definitions and divided them into 14 categories. The highest number of SR definition elements was found in categories related to searching (N = 51), analysis/synthesis (N = 23), overall methods (N = 22), quality/bias/appraisal/validity (N = 22) and aim/question (N = 13). The same five categories were also the most commonly used combination of categories in the SR definitions.

CONCLUSION

Currently used definitions of SRs are vague and ambiguous, often using terms such as clear, explicit and systematic, without further elaboration. In this manuscript we propose a more specific definition of a systematic review, with the ultimate aim of motivating the research community to establish a clear and unambiguous definition of this type of research.

Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data

OBJECTIVES

The aim of the study was to assess the completeness of reporting of methods in overviews.

STUDY DESIGN AND SETTING

Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented.

RESULTS

We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7-32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]).

CONCLUSION

Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.

Primary health care quality indicators: An umbrella review

Nowadays, evaluating the quality of health services, especially in primary health care (PHC), is increasingly important. In a historical perspective, the Department of Health (United Kingdom) developed and proposed a range of indicators in 1998, and lately several health, social and political organizations have defined and implemented different sets of PHC quality indicators. Some systematic reviews in PHC quality indicators are reported but only in specific contexts and conditions. The aim of this study is to characterize and provide a list of indicators discussed in the literature to support managers and clinicians in decision-making processes, through an umbrella review on PHC quality indicators. The methodology was performed according to PRISMA Statement. Indicators from 33 eligible systematic reviews were categorized according to the dimensions of care, function, type of care, domains and condition contexts. Of a total of 727 indicators or groups of indicators, 74.5% (n = 542) were classified in process category and 89.5% (n = 537) with chronic type of care (n = 428; 58.8%) and effective domain (n = 423; 58.1%) with the most frequent values in categorizations by dimensions. The results of this overview of reviews are valuable and imply the need for future research and practice regarding primary health care quality indicators in the most varied conditions and contexts to generate new discussions about their use, comparison and implementation.

Cochrane overviews: how can we optimize their impact on evidence-based rehabilitation?

INTRODUCTION

Overviews (i.e. reviews of multiple systematic reviews) comprise a relatively novel methodology to systematically synthesize research findings. Overviews aim for a beneficial impact on clinical practice, but their methods and pathways to impact have so far not been mapped. The aim of this paper was to inform recommendations for optimizing impact on rehabilitation practice and research by mapping methods and pathways to impact in Cochrane overviews relevant to rehabilitation.

EVIDENCE ACQUISITION

We systematically searched and identified published Cochrane overviews (to June 2018) relevant to rehabilitation. We extracted data and compared overviews on key characteristics, methods of evidence synthesis, statements about impact, and access metrics. We explored one overview in detail regarding beneficiaries, activities and outputs, mapped potential pathways to impact, and, using an iterative process, refined this into a generic map. Through exploration of all synthesized data, we propose further recommendations for planning, conducting and reporting of future overviews in order to optimize impact on rehabilitation.

EVIDENCE SYNTHESIS

We identified seven Cochrane overviews relevant to rehabilitation. Their focus and methods varied, but they were broadly related to rehabilitation interventions for populations of people with diverse long-term conditions. Overviews also varied regarding their intended impact; only 4 overviews identified specific beneficiaries. All overviews included multiple tables and figures, but only one synthesized key findings into a single figure. For five overviews, the Altmetric Attention Score (a weighted count of attention that an output receives based on a range of online sources) was in the top 5% of all research outputs scored by Altmetric. The overview within our worked example had four key impact goals, each with different beneficiaries and required actions; this example led to a generic map of potential pathways to impact for other overviews.

CONCLUSIONS

Cochrane overviews have the potential to play a key role in knowledge translation and therefore to be useful in supporting evidence-based rehabilitation practice. However, current overviews relating to rehabilitation differ in methods, approaches and intended impact, and sometimes fall short of promoting easy access to key information for beneficiaries. Future Cochrane overviews should address topics of importance to key beneficiaries and clearly outline potential pathways to impact in order to have a potential beneficial impact on evidence-based rehabilitation and to improve rehabilitation outcomes.

Service Delivery Models to Maximize Quality of Life for Older People at the End of Life: A Rapid Review

Policy Points We identified two overarching classifications of integrated geriatric and palliative care to maximize older people's quality of life at the end of life. Both are oriented to person-centered care, but with differing emphasis on either function or symptoms and concerns. Policymakers should both improve access to palliative care beyond just the last months of life and increase geriatric care provision to maintain and optimize function. This would ensure that continuity and coordination for potentially complex care needs across the continuum of late life would be maintained, where the demarcation of boundaries between healthy aging and healthy dying become increasingly blurred. Our findings highlight the urgent need for health system change to improve end-of-life care as part of universal health coverage. The use of health services should be informed by the likelihood of benefits and intended outcomes rather than on prognosis.

CONTEXT

In an era of unprecedented global aging, a key priority is to align health and social services for older populations in order to support the dual priorities of living well while adapting to a gradual decline in function. We aimed to provide a comprehensive synthesis of evidence regarding service delivery models that optimize the quality of life (QoL) for older people at the end of life across health, social, and welfare services worldwide.

METHODS

We conducted a rapid scoping review of systematic reviews. We searched MEDLINE, CINAHL, EMBASE, and CDSR databases from 2000 to 2017 for reviews reporting the effectiveness of service models aimed at optimizing QoL for older people, more than 50% of whom were older than 60 and in the last one or two years of life. We assessed the quality of these included reviews using AMSTAR and synthesized the findings narratively.

RESULTS

Of the 2,238 reviews identified, we included 72, with 20 reporting meta-analysis. Although all the World Health Organization (WHO) regions were represented, most of the reviews reported data from the Americas (52 of 72), Europe (46 of 72), and/or the Western Pacific (28 of 72). We identified two overarching classifications of service models but with different target outcomes: Integrated Geriatric Care, emphasizing physical function, and Integrated Palliative Care, focusing mainly on symptoms and concerns. Areas of synergy across the overarching classifications included person-centered care, education, and a multiprofessional workforce. The reviews assessed 117 separate outcomes. A meta-analysis demonstrated effectiveness for both classifications on QoL, including symptoms such as pain, depression, and psychological well-being. Economic analysis and its implications were poorly considered.

CONCLUSIONS

Despite their different target outcomes, those service models classified as Integrated Geriatric Care or Integrated Palliative Care were effective in improving QoL for older people nearing the end of life. Both approaches highlight the imperative for integrating services across the care continuum, with service involvement triggered by the patient's needs and likelihood of benefits. To inform the sustainability of health system change we encourage economic analyses that span health and social care and examine all sources of finance to understand contextual inequalities.

[Publications German-speaking countries in high impact journals: development and validation of a search filter]

Publications German-speaking countries in high impact journals: development and validation of a search filter Abstract.

BACKGROUND

The number of publications in journals with a high impact factor is an indication of a discipline's participation in international discourse. A search filter allows reliable and reproducible searches for specific publications.

AIM

Development and validation of a geographic search filter for publications by nursing scientists affiliated to German-speaking countries in nursing journals with a high impact factor.

METHODS

The search filter was objectively developed following several steps: (i) creation of a development and a validation set, each consisting of relevant and non-relevant publications, (ii) generation of the search filter by means of text analysis of the development set, (iii) internal validation based on the development set and (iv) external validation using the validation set. The validity was examined regarding several accuracy parameters, e. g. sensitivity, specificity, positive predictive value (PPV) and number needed to read (NNR).

RESULTS

The search filter correctly identified 22 of 30 relevant and 16 of 21 non-relevant publications in the development set: sensitivity 80 % (95 % CI 66 - 94), specificity 76 % (95 % CI 58 - 94), PPV 83 % (95 % CI 69 - 97). External validation yielded similar or better results: sensitivity 81 % (95 % CI 67 - 96), specificity 88 % (95 % CI 71 - 100), PPV 88 % (95 % CI 75 - 100). The NNR was 1.2 and 1.1, respectively.

CONCLUSIONS

The search filter has the potential to identify the intended publications.

Toward a comprehensive evidence map of overview of systematic review methods: paper 2-risk of bias assessment; synthesis, presentation and summary of the findings; and assessment of the certainty of the evidence

BACKGROUND

Overviews of systematic reviews (SRs) attempt to systematically retrieve and summarise the results of multiple systematic reviews. This is the second of two papers from a study aiming to develop a comprehensive evidence map of the methods used in overviews. Our objectives were to (a) develop a framework of methods for conducting, interpreting and reporting overviews (stage I)-the Methods for Overviews of Reviews (MOoR) framework-and (b) to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). In the first paper, we reported findings for the four initial steps of an overview (specification of purpose, objectives and scope; eligibility criteria; search methods; data extraction). In this paper, we report the remaining steps: assessing risk of bias; synthesis, presentation and summary of the findings; and assessing certainty of the evidence arising from the overview.

METHODS

In stage I, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews. Based on these studies, we developed a framework of possible methods for overviews, categorised by the steps in conducting an overview. Multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified studies evaluating methods and mapped these evaluations to the framework.

RESULTS

Forty-two stage I studies described methods relevant to one or more of the latter steps of an overview. Six studies evaluating methods were included in stage II. These mapped to steps involving (i) the assessment of risk of bias (RoB) in SRs (two SRs and three primary studies, all reporting evaluation of RoB tools) and (ii) the synthesis, presentation and summary of the findings (one primary study evaluating methods for measuring overlap).

CONCLUSION

Many methods have been described for use in the latter steps in conducting an overview; however, evaluation and guidance for applying these methods is sparse. The exception is RoB assessment, for which a multitude of tools exist-several with sufficient evaluation and guidance to recommend their use. Evaluation of other methods is required to provide a comprehensive evidence map.

Interventions to improve patient flow in emergency departments: an umbrella review

Objectives

Patient flow and crowding are two major issues in ED service improvement. A substantial amount of literature exists on the interventions to improve patient flow and crowding, making it difficult for policymakers, managers and clinicians to be familiar with all the available literature and identify which interventions are supported by the evidence. This umbrella review provides a comprehensive analysis of the evidence from existing quantitative systematic reviews on the interventions that improve patient flow in EDs.

Methods

An umbrella review of systematic reviews published between 2000 and 2017 was undertaken. Included studies were systematic reviews and meta-analyses of quantitative primary studies assessing an intervention that aimed to improve ED throughput.

Results

The search strategy yielded 623 articles of which 13 were included in the umbrella review. The publication dates of the systematic reviews ranged from 2006 to 2016. The 13 systematic reviews evaluated 26 interventions: full capacity protocols, computerised provider order entry, scribes, streaming, fast track and triage. Interventions with similar characteristics were grouped together to produce the following categories: diagnostic services, assessment/short stay units, nurse-directed interventions, physician-directed interventions, administrative/organisational and miscellaneous. The statistical evidence from 14 primary randomised controlled trials (RCTs) was evaluated to determine if correlation or clustering of observations was considered. Only the fast track intervention had moderate evidence to support its use but the RCTs that assessed the intervention did not use statistical tests that considered correlation.

Conclusions

Overall, the evidence supporting the interventions to improve patient flow is weak. Only the fast track intervention had moderate evidence to support its use but correlation/clustering was not taken into consideration in the RCTs examining the intervention. Failure to consider the correlation of the data in the primary studies could result in erroneous conclusions of effectiveness.

Toward a comprehensive evidence map of overview of systematic review methods: paper 1-purpose, eligibility, search and data extraction

BACKGROUND

Overviews of systematic reviews attempt to systematically retrieve and summarise the results of multiple systematic reviews. Methods for conducting, interpreting and reporting overviews are in their infancy. To date, there has been no evidence map of the methods used in overviews, thus making it difficult to determine the gaps and priorities for methods research. Our objectives were to develop and populate a comprehensive framework of methods for conducting, interpreting and reporting overviews (stage I) and to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II).

METHODS

We searched methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program) to identify eligible studies for both stages of this research. In stage I, cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews were used to develop and populate the framework of methods. Drafts and multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified and described studies evaluating overview methods and mapped these evaluations to the framework.

RESULTS

In this paper, we present results for the four initial steps of conducting an overview: (a) specification of the purpose, objectives and scope, (b) specification of the eligibility criteria, (c) search methods and (d) data extraction. Twenty-nine studies mentioned or described methods relevant to one or more of these steps. In the developed framework, identified methods and approaches were grouped according to the steps an overview author would need to undertake. Fifteen studies evaluated identified methods, all of which mapped to the search methods step. These studies either reported the development and evaluation of a new search filter to retrieve systematic reviews or compared the performance of multiple filters.

CONCLUSION

Gaps in the evaluation of methods were found for the majority of steps in the framework. More empirical studies are needed to evaluate the methods outlined and provide a comprehensive evidence map. The framework is useful for planning these evaluations and for planning methods required to deal with challenges that arise when conducting an overview.

Overviews of systematic reviews: great promise, greater challenge

The proliferation of systematic reviews and escalating demand from policy makers has driven a newer form of evidence synthesis-overviews of systematic reviews. Systematic Reviews are publishing a special thematic series on overviews and are encouraging submissions on the development and evaluation of methods for this review type. The authors' of this editorial introduce the series by considering challenges that arise when conducting an overview and what methods guidance is available. They emphasise the importance of evaluating overview methods to understand the trade-offs of using different approaches and propose that a more systematic and coordinated approach to methods development would be beneficial. Finally, they consider the potential for overviews to drive improvements in the conduct and reporting of systematic reviews.

Text mining for search term development in systematic reviewing: A discussion of some methods and challenges

Using text mining to aid the development of database search strings for topics described by diverse terminology has potential benefits for systematic reviews; however, methods and tools for accomplishing this are poorly covered in the research methods literature. We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in 5 overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (term frequency-inverse document frequency and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches. Challenges in using these tools are discussed. The utility of these tools is influenced by the different capabilities of the tools, the way the tools are used, and the text that is analysed. Increased awareness of how the tools perform facilitates the further development of methods for their use in systematic reviews.

Epidemiology and reporting characteristics of overviews of reviews of healthcare interventions published 2012-2016: protocol for a systematic review

BACKGROUND

Overviews of systematic reviews (overviews) attempt to systematically retrieve and summarize the results of multiple systematic reviews (SRs) for a given condition or public health problem. Two prior descriptive analyses of overviews found substantial variation in the methodological approaches used in overviews, and deficiencies in reporting of key methodological steps. Since then, new methods have been developed so it is timely to update the prior descriptive analyses. The objectives are to: (1) investigate the epidemiological, descriptive, and reporting characteristics of a random sample of 100 overviews published from 2012 to 2016 and (2) compare these recently published overviews (2012-2016) to those published prior to 2012 (based on the prior descriptive analyses).

METHODS

Medline, EMBASE, and CDSR will be searched for overviews published 2012-2016, using a validated search filter for overviews. Only overviews written in English will be included. All titles and abstracts will be screened by one review author; those deemed not relevant will be verified by a second person for exclusion. Full-texts will be assessed for inclusion by two reviewers independently. Of those deemed relevant, a random sample of 100 overviews will be selected for inclusion. Data extraction will be either performed by one reviewer with verification by a second reviewer or by one reviewer only depending on the complexity of the item. Discrepancies at any stage will be resolved by consensus or consulting a third person. Data will be extracted on the epidemiological, descriptive, and reporting characteristics of each overview. Data will be analyzed descriptively. When data are available for both time points (up to 2011 vs. 2012-2016), we will compare characteristics by calculating risk ratios or applying the Mann-Whitney test.

DISCUSSION

Overviews are becoming increasingly valuable evidence syntheses, and the number of published overviews is increasing. However, former analyses found limitations in the conduct and reporting of overviews. This update of a recent sample of overviews will inform whether this has changed, while also identifying areas for further improvement.

SYSTEMATIC REVIEW REGISTRATION

The review will not be registered in PROSPERO as it does not meet the eligibility criterion of dealing with health-related outcomes.

No suitable precise or optimized epidemiologic search filters were available for bibliographic databases

OBJECTIVES

To determine a suitable approach to a systematic search for epidemiologic publications in bibliographic databases. For this purpose, suitable sensitive, precise, and optimized filters were to be selected for MEDLINE searches. In addition, the relevance of bibliographic databases was determined.

STUDY DESIGN AND SETTING

Epidemiologic systematic reviews (SRs) retrieved in a systematic search and company dossiers were screened to identify epidemiologic publications (primary studies and SRs) published since 2007. These publications were used to generate a test and validation set. Furthermore, each SR's search strategy was reviewed, and epidemiologic filters were extracted. The search syntaxes were validated using the relative recall method.

RESULTS

The test set comprises 729 relevant epidemiologic publications, of which 566 were MEDLINE-indexed. About 27 epidemiologic filters were extracted. One suitable sensitive filter was identified (Larney et al. 2013: 95.94% sensitivity). Precision was presumably underestimated so that no precise or optimized filters can be recommended. About 77.64% of the publications were found in MEDLINE.

CONCLUSION

There is currently no suitable approach to conducting efficient systematic searches for epidemiologic publications in bibliographic databases. The filter by Larney et al. (2013) can be used for sensitive MEDLINE searches. No robust conclusions can be drawn on precise or optimized filters. Additional search approaches should be considered.

Most overviews of Cochrane reviews neglected potential biases from dual authorship

OBJECTIVE

Some authors of Cochrane overviews have also (co-)authored one or more of the underlying reviews. We examined the extent of dual (co-)authorship in Cochrane overviews, how it is dealt with, and whether the issue is raised in protocols.

STUDY DESIGN

The Cochrane Library was searched for overviews and protocols for overviews in September 2015. Data on dual (co-)authorship were extracted for each review into standard spreadsheets by one author and checked for accuracy by a second.

RESULTS

Twenty overviews and 25 protocols were identified. The overviews included a median of 10 reviews (interquartile range [IQR]: 6-18.5). In 18/20 overviews (90%), at least one of the included reviews was affected by dual (co-)authorship. A median of 5 (IQR, 2.5-7) reviews per overview was affected by dual (co-)authorship. In 8/18 (44%) overviews with dual (co-)authorship, quality assessment was conducted independently. In 7/25 (28%) protocols, dual (co-)authorship was mentioned.

CONCLUSION

Potential biases arising from dual (co-)authorship are often neglected in Cochrane overviews. We argue that authors of Cochrane overviews and Review Groups should pay more attention to the issue, to avoid bias and preserve the good reputation that Cochrane overviews will typically deserve.

Evidence map of studies evaluating methods for conducting, interpreting and reporting overviews of systematic reviews of interventions: rationale and design

BACKGROUND

Overviews of systematic reviews attempt to systematically retrieve and summarise the results of multiple systematic reviews into a single document. Methods for conducting, interpreting and reporting overviews of reviews are in their infancy. To date, there has been no systematic review or evidence map examining the range of methods for overviews nor of the evidence for using these methods. The objectives of the study are to develop and populate a framework of methods that have or may be used in conducting, interpreting and reporting overviews of systematic reviews of interventions (stage I); create an evidence map of studies that have evaluated these methods (stage II); and identify and describe unique methodological challenges of overviews.

METHODS

The research will be undertaken in two stages. For both stages, we plan to search methods collections (e.g. Cochrane Methodology Register, Meth4ReSyn library, AHRQ Effective Health Care Program) to identify eligible studies. These searches will be supplemented by searching reference lists and citation searching. Stage I: Methods used in overviews will be identified from articles describing methods for overviews, methods studies examining a cross section/cohort of overviews, guidance documents and commentaries. The identified methods will populate a framework of available methods for conducting an overview. Two reviewers will independently code included studies to develop the framework. Thematic analysis of the coded data will be used to categorise and describe methods. Stage II: Evaluations of the performance of methods will be identified from systematic reviews of methods studies and methods studies. Evaluations will be described and mapped to the framework of methods identified in stage I.

DISCUSSION

The results of this process will be useful for mapping of methods for overviews of systematic reviews, informing guidance and identifying and prioritising method research in this field.