Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Prevalence of and factors associated with potentially redundant randomized controlled trials: a cross-sectional study

OBJECTIVES

To investigate the prevalence of meta-analyses containing potentially redundant randomized controlled trials (RCTs) and the factors associated with the presence of redundancy.

STUDY DESIGN AND SETTING

This is a cross-sectional study, based on a random sample of references (n = 4500) that were published during 2020 and 2021, indexed in PubMed, Embase, or the Cochrane Database of Systematic Reviews, and retrieved through comprehensive searches using terms about systematic reviews and meta-analysis. From each systematic review, one meta-analysis fulfilling all the following criteria, if available, was included in this study: (1) assessing the effect of the intervention on a primary outcome of the systematic review; (2) combining RCTs only. The primary outcome was prevalence of meta-analyses containing potentially redundant RCTs. Potentially redundant RCTs referred to the trials that started 1 year after the overall effect estimate from cumulative meta-analysis had been statistically robust, as determined by trial sequential analysis when appropriate. The number of potentially redundant trials (if any) in each eligible meta-analysis and the number of participants involved in those trials were documented and contrasted across groups. Logistic regression analysis was conducted to explore the factors associated with presence of potential redundancy.

RESULTS

Of the 448 eligible meta-analyses, 57 (12.7%, 95% confidence interval (CI) 9.8-16.2%) contained potentially redundant RCTs. When limited to the 333 low-heterogeneity meta-analyses, the prevalence was 17.1% (95% CI 13.5-21.5%). The total number of potentially redundant RCTs was 295 (involving 85,385 participants), accounting for 38.5% of the RCTs (and 30.3% of the participants) included in the 57 meta-analyses. In these meta-analyses, the median number of potentially redundant RCTs and the participants involved were 2 (range: 1-50) and 352 (range: 17-26997), respectively. Potentially redundant RCTs were more likely to be present in the meta-analyses evaluating pharmaceutical intervention (odds ratio [OR] 2.31, 95% CI 1.16-4.49), assessing efficacy outcomes (OR 7.25, 95% CI 0.85-61.87), containing more than 5 RCTs (OR 6.47, 95% CI 3.22-12.99), or with the earliest RCT reporting statistically significant effect estimate (OR 5.30, 95% CI 2.64-10.64).

CONCLUSION

This study found that 12.7% to 17.1% of recently published meta-analyses contained potentially redundant RCTs, highlighting the importance of conducting or examining systematic reviews of existing evidence to justify new RCTs. More importantly, the study identified some scenarios in which redundancy was more likely to occur and thus has implications for trialists, funding agencies, ethics committees, and journal editors.

Patterns and Pitfalls of Short-cuts Used in Environmental Management Rapid Reviews

Environmental managers and policy-makers need reliable evidence to make effective decisions. Systematic reviews are one way to provide this information but are time-consuming and may not meet the needs of decision-makers when faced with rapidly changing management requirements or transient policy-windows. Rapid reviews are one type of knowledge synthesis that follow simplified or truncated methods compared to systematic reviews. Rapid reviews on environmentally-relevant topics are growing in prevalence, but it is unclear if rapid reviews use similar short-cuts or follow available guidelines. In this methodological review, we assess 26 rapid reviews published between 2002 and 2023. Numerous rapid review short-cuts and approaches were identified, with few consistencies among studies. Short-cuts were present in all stages of the review process, with some of the most common short-cuts including not developing an a priori review protocol, not including stakeholder involvement, or not conducting critical appraisal of study validity. Poor quality in reporting of methods was observed. Fewer than half of assessed rapid reviews reported using available guidelines when developing their methods. Future rapid reviews should aim for improved reporting and adherence to published guidelines to help increase the useability and evidence-user confidence. This will also enable readers to understand where short-cuts were made and their potential consequences for the conclusions of the review.

What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies

Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were sought included the preparation of a protocol, question formulation, inclusion criteria, searching, selection, data extraction, RoB assessment, synthesis, and reporting. We searched 11 electronic databases in April 2022, and conducted some supplementary searching. Reviewers worked in pairs to screen, select, extract data, and assess the RoB of included reviews and studies. We included 15 systematic reviews, 7 scoping reviews, and 65 primary studies. We found that several commonly used shortcuts in rapid reviews are likely to increase the RoB in the results. These include restrictions based on publication date, use of a single electronic database as a source of studies, and use of a single reviewer for screening titles and abstracts, selecting studies based on the full-text, and for extracting data. Authors of rapid reviews should be transparent in reporting their use of these shortcuts and acknowledge the possibility of them causing bias in the results. This review also highlights shortcuts that can save time without increasing the risk of bias. Further research is needed for both systematic and rapid reviews on faster methods for accurate data extraction and RoB assessment, and on development of more precise search strategies.

Rapid reviews methods series: Guidance on literature search

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

Topic Reviews in Neurosurgical Journals: An Analysis of Publication Trends

BACKGROUND

Topic review articles have become increasingly popular, even as the neurosurgical community looks to peer-reviewed journals as a source of discovery in basic and clinical science. In this study we quantify the prevalence of topic review articles in top neurosurgery journals.

METHODS

The top 20 neurosurgery journals were defined by Google Scholar metrics. The PubMed database quantified the number of topic reviews compared with the total number of articles published; data were analyzed for trends between 1945 and 2022.

RESULTS

All 20 journals have published topic reviews since the start of records on PubMed. Total publications have increased from <500 before 1980 to >8000 in 2022. Topic reviews have increased from <1% before 1980, to 2% by 2000, and to 3%-4% since 2010. The linear trend line equation for the total percentage of reviews in all journals shows a small increase in topic reviews per year. Three journals decreased review publication whereas 4 have reached prevalence >10%. The prevalence of topic reviews increased significantly from the first (2.13) to the last (4.76) year of publication (P = 0.003).

CONCLUSIONS

The increasing prevalence of topic reviews is seen in most neurosurgery journals, reflecting supply and demand. Although there are benefits to these articles, they do not contribute novel data. Actions such as defining and labeling this publication type in journals and databases will improve the transparency of research methods. Academic neurosurgeons should further expand their knowledge and not become focused only on introspection into and review of neurosurgical understanding and practice.

Patients with geriatric syndromes and anti-amyloid therapies: lack of consideration? An exploratory analysis of the literature

Introduction

Patients who should benefit from anti-amyloid therapies (AAT) are found across all geriatric settings. Yet, it remains unclear how the use of AAT in patients with geriatric syndromes, such as frailty and polypharmacy, has so far been discussed in the literature.

Methods

Articles on aducanumab, gantenerumab, lecanemab, donanemab, crenezumab, solanezumab were retrieved in MEDLINE from inception to July 2023. For each article, identified geriatric relevant terms were assigned to five discussion contexts (eligibility of AAT study population, safety, prescription, patient clinical profile, alternative outcomes measurement). Article type and the involvement of geriatric healthcare professionals as an author were further extracted.

Results

Out of 538 articles, 23 (4.27%) were published in journals from the geriatric category, 44 (8.18%) included an author affiliated with a geriatric institution. One hundred and sixteen (21.56%) articles included at least one geriatric relevant term, which were mostly discussed in the context of safety and eligibility. Articles mentioning geriatric syndromes were more frequently authored by a geriatric healthcare professional (p = 0.044).

Discussion

The use of AAT in patients with geriatric syndromes has so far received poor attention in the literature raising concerns on their use in this patient group. The involvement of geriatric healthcare professionals in future studies may increase the relevance of AAT research in patients with geriatric syndromes.

Improving mental ill-health with psycho-social group interventions in South Asia-A scoping review using a realist lens

This scoping review aimed to synthesise current evidence related to psycho-social groups as part of community-based mental health interventions in South Asia. We used a realist lens to pay attention to the contexts and mechanisms supporting positive outcomes. We included studies published from January 2007 to February 2022 that: were based in communities in South Asia, included a group component, reported on interventions with a clear psychosocial component, targeted adults and were implemented by lay community health workers. Two reviewers extracted data on intervention components, groups and facilitators, participant demographics and enabling contexts, mechanisms and outcomes. Expert reference panels including people with lived experience of psycho-social disability, mental health professionals and policy makers confirmed the validity and relevance of initial review findings. The review examined 15 interventions represented by 42 papers. Only four interventions were solely psycho-social and nearly all included psychoeducation and economic support. Only 8 of the 46 quantitative outcome measures used were developed in South Asia. In a context of social exclusion and limited autonomy for people with psychosocial disability, psychosocial support groups triggered five key mechanisms. Trusted relationships undergirded all mechanisms, and provided a sense of inclusion, social support and of being able to manage mental distress due to improved skills and knowledge. Over time group members felt a sense of belonging and collective strength meaning they were better able to advocate for their own well-being and address upstream social health determinants. This led to outcomes of improved mental health and social participation across the realms of intrapersonal, interpersonal and community. Psychosocial groups merit greater attention as an active ingredient in community interventions and also as an effective, relevant, acceptable and scalable platform that can promote and increase mental health in communities, through facilitation by lay community health workers.

Rapid literature review: definition and methodology

Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews. Methods: The Medline and EMBASE databases, as well as the grey literature, were searched using the set of keywords and their combination related to the targeted and rapid review, as well as design, approach, and methodology. Of the 3,898 records retrieved, 12 articles were included. Results: Specific definition of RLRs has only been developed in 2021. In terms of methodology, the RLR should be completed within shorter timeframes using simplified procedures in comparison to SLRs, while maintaining a similar level of transparency and minimizing bias. Inherent components of the RLR process should be a clear research question, search protocol, simplified process of study selection, data extraction, and quality assurance. Conclusions: There is a lack of consensus on the formal definition of the RLR and the best approaches to perform it. The evidence-based supporting methods are evolving, and more work is needed to define the most robust approaches.

The effect of minimum volume standards in hospitals (MIVOS) - protocol of a systematic review

BACKGROUND

The volume-outcome relationship, i.e., higher hospital volume results in better health outcomes, has been established for different surgical procedures as well as for certain nonsurgical medical interventions. Accordingly, many countries such as Germany, the USA, Canada, the UK, and Switzerland have established minimum volume standards. To date, there is a lack of systematically summarized evidence regarding the effects of such regulations.

METHODS

To be included in the review, studies must measure any effects connected to minimum volume standards. Outcomes of interest include the following: (1) patient-related outcomes, (2) process-related outcomes, and (3) health system-related outcomes. We will include (cluster) randomized controlled trials ([C]RCTs), non-randomized controlled trials (nRCTs), controlled before-after studies (CBAs), and interrupted time-series studies (ITSs). We will apply no restrictions regarding language, publication date, and publication status. We will search MEDLINE (via PubMed), Embase (via Embase), CENTRAL (via Cochrane Library), CINHAL (via EBSCO), EconLit (via EBSCO), PDQ evidence for informed health policymaking, health systems evidence, OpenGrey, and also trial registries for relevant studies. We will further search manually for additional studies by cross-checking the reference lists of all included primary studies as well as cross-checking the reference lists of relevant systematic reviews. To evaluate the risk of bias, we will use the ROBINS-I and RoB 2 risk-of-bias tools for the corresponding study designs. For data synthesis and statistical analyses, we will follow the guidance published by the EPOC Cochrane group (Cochrane Effective Practice and Organisation of Care (EPOC), EPOC Resources for review authors, 2019).

DISCUSSION

This systematic review focuses on minimum volume standards and the outcomes used to measure their effects. It is designed to provide thorough and encompassing evidence-based information on this topic. Thus, it will inform decision-makers and policymakers with respect to the effects of minimum volume standards and inform further studies in regard to research gaps.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022318883.

In a pilot study, automated real-time systematic review updates were feasible, accurate, and work-saving

OBJECTIVES

The aim of this study is to describe and pilot a novel method for continuously identifying newly published trials relevant to a systematic review, enabled by combining artificial intelligence (AI) with human expertise.

STUDY DESIGN AND SETTING

We used RobotReviewer LIVE to keep a review of COVID-19 vaccination trials updated from February to August 2021. We compared the papers identified by the system with those found by the conventional manual process by the review team.

RESULTS

The manual update searches (last search date July 2021) retrieved 135 abstracts, of which 31 were included after screening (23% precision, 100% recall). By the same date, the automated system retrieved 56 abstracts, of which 31 were included after manual screening (55% precision, 100% recall). Key limitations of the system include that it is limited to searches of PubMed/MEDLINE, and considers only randomized controlled trial reports. We aim to address these limitations in future. The system is available as open-source software for further piloting and evaluation.

CONCLUSION

Our system identified all relevant studies, reduced manual screening work, and enabled rolling updates on publication of new primary research.

Rapid evidence synthesis to enable innovation and adoption in health and social care

BACKGROUND

The rapid identification and adoption of effective innovations in healthcare is a known challenge. The strongest evidence base for innovations can be provided by evidence synthesis, but this is frequently a lengthy process and even rapid versions of this can be time-consuming and complex. In the UK, the Accelerated Access Review and Academic Health Science Network (AHSN) have provided the impetus to develop a consistently rapid process to support the identification and adoption of high-value innovations in the English NHS.

METHODS

The Greater Manchester Applied Research Collaboration (ARC-GM) developed a framework for a rapid evidence synthesis (RES) approach, which is highly integrated within the innovation process of the Greater Manchester AHSN and the associated healthcare and research ecosystem. The RES uses evidence synthesis approaches and draws on the GRADE Evidence to Decision framework to provide rapid assessments of the existing evidence and its relevance to specific decision problems. We implemented this in a real-time context of decision-making around adoption of innovative health technologies.

RESULTS

Key stakeholders in the Greater Manchester decision-making process for healthcare innovations have found that our approach is both timely and flexible; it is valued for its combination of rigour and speed. Our RES approach rapidly and systematically identifies, appraises and contextualises relevant evidence, which can then be transparently incorporated into decisions about the wider adoption of innovations. The RES also identifies limitations in existing evidence for innovations and this can inform subsequent evaluations. There is substantial interest from other ARCs and AHSNs in implementing a similar process. We are currently exploring methods to make completed RES publicly available. We are also exploring methods to evaluate the impact of using RES as more implementation decisions are made.

CONCLUSIONS

The RES framework we have implemented combines transparency and consistency with flexibility and rapidity. It therefore maximises utility in a real-time decision-making context for healthcare innovations.

A search of just four key databases would identify most randomized controlled trials of acupuncture: a meta-epidemiological study

INTRODUCTION

Little research has been conducted to assess which specific databases should be searched when performing a systematic review (SR) on acupuncture. The current study aimed to identify key databases and the optimal database combination to retrieve randomized controlled trials (RCTs) on acupuncture for inclusion in SRs.

METHODS

A systematic search for SRs in the field of acupuncture was conducted in order to identify target databases and RCTs were extracted from the SRs that had searched all target databases. The proportions of SRs that had achieved 100%, 95% or 90% recall of RCTs and the total recall of RCTs in various combinations of target databases were calculated. Sensitivity analysis was performed on those SRs that included 10 or more RCTs.

RESULTS

CNKI, WanFang, VIP, PubMed, CENTRAL and Embase were regarded as target databases. A total of 4349 acupuncture RCTs were extracted from 286 SRs. Searching all six target databases retrieved 99.3% of all RCTs while 99.1% were recalled by searching the combination of CNKI, WanFang, PubMed and CENTRAL. There were no significant differences on total recall of RCTs (p=0.549) or in the proportion of SRs with 90% recall of RCTs (97.2% vs. 97.6%; p=0.794) between searching the above four and the full six target databases. Most results were similar in the sensitivity analysis.

CONCLUSION

The combined retrieval power of CNKI, WanFang, PubMed and CENTRAL was considered an efficient choice to retrieve acupuncture RCTs included in SRs. This article is protected by copyright. All rights reserved.

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

OBJECTIVE

The academic and scientific community has reacted at pace to gather evidence to help and inform about COVID-19. Concerns have been raised about the quality of this evidence. The aim of this review was to map the nature, scope and quality of evidence syntheses on COVID-19 and to explore the relationship between review quality and the extent of researcher, policy and media interest.

DESIGN AND SETTING

A meta-research: systematic review of reviews.

INFORMATION SOURCES

PubMed, Epistemonikos COVID-19 evidence, the Cochrane Library of Systematic Reviews, the Cochrane COVID-19 Study Register, EMBASE, CINAHL, Web of Science Core Collection and the WHO COVID-19 database, searched between 10 June 2020 and 15 June 2020.

ELIGIBILITY CRITERIA

Any peer-reviewed article reported as a systematic review, rapid review, overview, meta-analysis or qualitative evidence synthesis in the title or abstract addressing a research question relating to COVID-19. Articles described as meta-analyses but not undertaken as part of a systematic or rapid review were excluded.

STUDY SELECTION AND DATA EXTRACTION

Abstract and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met the definition of a systematic review according to six key methodological criteria. For those meeting all criteria, additional data on methods and publication metrics were extracted.

RISK OF BIAS

For articles meeting all six criteria required to meet the definition of a systematic review, AMSTAR-2 ((A MeaSurement Tool to Assess systematic Reviews, version 2.0) was used to assess the quality of the reported methods.

RESULTS

2334 articles were screened, resulting in 280 reviews being included: 232 systematic reviews, 46 rapid reviews and 2 overviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six systematic review criteria. Of these, just 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws: only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication were rapid, with 52 of the 88 systematic reviews reported as being conducted within 3 weeks, and a half published within 3 weeks of submission. Researcher and media interest, as measured by altmetrics and citations, was high, and was not correlated with quality.

DISCUSSION

This meta-research of early published COVID-19 evidence syntheses found low-quality reviews being published at pace, often with short publication turnarounds. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.

INTERPRETATION

Flaws in systematic review methods limit the validity of a review and the generalisability of its findings. Yet, by being reported as 'systematic reviews', many readers may well regard them as high-quality evidence, irrespective of the actual methods undertaken. The challenge especially in times such as this pandemic is to provide indications of trustworthiness in evidence that is available in 'real time'.

PROSPERO REGISTRATION NUMBER

CRD42020188822.

Searching two or more databases decreased the risk of missing relevant studies: a metaresearch study

OBJECTIVE

Assessing changes in coverage, recall, review conclusions and references not found when searching fewer databases.

STUDY DESIGN AND SETTING

In 60 randomly selected Cochrane reviews, we checked included study publications' coverage (indexation) and recall (findability) using different search approaches with MEDLINE, Embase, and CENTRAL and related them to authors' conclusions and certainty. We assessed characteristics of unfound references.

RESULTS

1989/2080 included references, were indexed in ≥1 database (coverage=96%). In reviews where using one of our search approaches would not change conclusions and certainty (n=44-54), median coverage and recall were highest (range 87.9-100.0% and 78.2-93.3%, respectively). Here, searching ≥2 databases reached >95% coverage and ≥87.9% recall. In reviews with unchanged conclusions but less certainty (n=2-8): 63.3-79.3% coverage and 45.0-75.0% recall. In reviews with opposite conclusions (n=1-3): 63.3-96.6% and 52.1-78.7%. In reviews where a conclusion was no longer possible (n=3-7): 60.6%-86.0% and 20.0-53.8%. The 265 references that were indexed but unfound were more often abstractless (30% vs 11%) and older (28% vs. 17% published before 1991) than found references.

CONCLUSION

Searching ≥2 databases improves coverage and recall and decreases the risk of missing eligible studies. If researchers suspect that relevant articles are difficult to find, supplementary search methods should be used.

Differential expression of MicroRNAs in Alzheimer's disease: a systematic review and meta-analysis

Alzheimer's disease (AD) results in progressive cognitive decline owing to the accumulation of amyloid plaques and hyperphosphorylated tau. MicroRNAs (miRNAs) have attracted attention as a putative diagnostic and therapeutic target for neurodegenerative diseases. However, existing meta-analyses on AD and its association with miRNAs have produced inconsistent results. The primary objective of this study is to evaluate the magnitude and consistency of differences in miRNA levels between AD patients, mild cognitive impairment (MCI) patients and healthy controls (HC). Articles investigating miRNA levels in blood, brain tissue, or cerebrospinal fluid (CSF) of AD and MCI patients versus HC were systematically searched in PubMed/Medline from inception to February 16^th, 2021. Fixed- and random-effects meta-analyses were complemented with the I² statistic to measure the heterogeneity, assessment of publication bias, sensitivity subgroup analyses (AD severity, brain region, post-mortem versus ante-mortem specimen for CSF and type of analysis used to quantify miRNA) and functional enrichment pathway analysis. Of the 1512 miRNAs included in 61 articles, 425 meta-analyses were performed on 334 miRNAs. Fifty-six miRNAs were significantly upregulated (n = 40) or downregulated (n = 16) in AD versus HC and all five miRNAs were significantly upregulated in MCI versus HC. Functional enrichment analysis confirmed that pathways related to apoptosis, immune response and inflammation were statistically enriched with upregulated pathways in participants with AD relative to HC. This study confirms that miRNAs' expression is altered in AD and MCI compared to HC. These findings open new diagnostic and therapeutic perspectives for this disorder.

Principles and framework for assessing the risk of bias for studies included in comparative quantitative environmental systematic reviews

The internal validity of conclusions about effectiveness or impact in systematic reviews, and of decisions based on them, depends on risk of bias assessments being conducted appropriately. However, a random sample of 50 recently-published articles claiming to be quantitative environmental systematic reviews found 64% did not include any risk of bias assessment, whilst nearly all that did omitted key sources of bias. Other limitations included lack of transparency, conflation of quality constructs, and incomplete application of risk of bias assessments to the data synthesis. This paper addresses deficiencies in risk of bias assessments by highlighting core principles that are required for risk of bias assessments to be fit-for-purpose, and presenting a framework based on these principles to guide review teams on conducting risk of bias assessments appropriately and consistently. The core principles require that risk of bias assessments be Focused, Extensive, Applied and Transparent (FEAT). These principles support risk of bias assessments, appraisal of risk of bias tools, and the development of new tools. The framework follows a Plan-Conduct-Apply-Report approach covering all stages of risk of bias assessment. The scope of this paper is comparative quantitative environmental systematic reviews which address PICO or PECO-type questions including, but not limited to, topic areas such as environmental management, conservation, ecosystem restoration, and analyses of environmental interventions, exposures, impacts and risks.

Waste to Energy in Developing Countries—A Rapid Review: Opportunities, Challenges, and Policies in Selected Countries of Sub-Saharan Africa and South Asia towards Sustainability

Daily per capita waste generation will increase by 40% and 19%, for developing and developed countries by 2050, respectively. The World Bank estimates that total waste generation is going to triple in Sub-Saharan Africa (SSA) and double in South Asia (SA) by 2050. This article conducts a rapid review and aims to demonstrate the current waste management scenario and the potential of waste to energy generation in the developing world, focusing on SSA and SA. Although many review articles related to waste to energy (WtE) in developing countries are available in the literature, a rapid review particularly focusing on countries in SSA and SA is rarely seen. An analysis of different WtE generation technologies, and current waste management practices in developing countries in SSA and SA are also presented. The analysis shows that about three-fourths of waste is openly dumped in developing countries of SSA and SA. In terms of waste composition, on average, about 48.70% and 51.16% of waste generated in developing economies of SSA and SA are organic. Opportunities to convert this waste into energy for developing countries are highlighted, with a case study of Bangladesh, a country in SA. Major challenges regarding the waste to energy (WtE) projects in the developing world are found to be the composition of waste, absence of waste separation scheme at source, ineffective waste collection method, lack of suitable WtE generation technology in place, lack of financial support and policies related to a WtE project, and absence of coordination between different governmental institutions.

Methodological guidance for rapid reviews in healthcare: a scoping review

OBJECTIVE

The aim of the present work was to identify published methodological guidance for rapid reviews (RRs) and to analyze the recommendations with regard to time-saving measures.

STUDY DESIGN AND SETTING

A literature search was performed in PubMed and EMBASE in November 2020. In addition, a search based on Google Scholar and websites of governmental and non-governmental organizations was conducted. Literature screening was carried out by two researchers independently.

RESULTS

A total of 34 publications were included. These describe 38 distinct RR types. The timeframe to complete the identified RR types ranges from 24 hours to six months (mean time 2.2 months). For most RR types a specific research question (n=21) and a prioritizing search (n=25; preference for e.g. systematic reviews, meta-analyses) is employed. Different approaches such as reduced personnel in literature screening (n=21) and data extraction (n=21) are recommended. The majority of RR types include a bias assessment (n=28) and suggest a narrative report focusing on safety and efficacy.

CONCLUSION

The included RR types are heterogeneous in terms of completion time, considered domains and strategies to alter the standard systematic review methods. A rationale for the recommended shortcuts is rarely presented. This article is protected by copyright. All rights reserved.

Changes in Youth Mental Health, Psychological Wellbeing, and Substance Use During the COVID-19 Pandemic: A Rapid Review

Since the onset of the COVID-19 pandemic, researchers around the world have made efforts to assess its impact on youth mental health; however, the breadth of this topic has impeded a clear assessment of pandemic outcomes. This study aimed to address this gap by reviewing changes in youth (age ≤ 25) mental health, psychological wellbeing, substance use, and the use or delivery of relevant services during the pandemic. PubMed and Embase were searched in May 2021 to conduct a rapid review of the literature. The results encompass 156 primary publications and are reported using a narrative synthesis. Studies of mental health (n = 122) and psychological wellbeing (n = 28) generally indicated poor outcomes in many settings. Publications regarding substance use (n = 41) noted overall declines or unchanged patterns. Studies of service delivery (n = 12) indicated a generally positive reception for helplines and telehealth, although some youth experienced difficulties accessing services. The findings indicate negative impacts of the pandemic on youth mental health, with mixed results for substance use. Services must support marginalized youth who lack access to telehealth.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40894-022-00185-6.

Opportunities to improve reporting of rapid response in health technology assessment

Introduction

Mini health technology assessment (HTA) reports have been used to support policy makers and health systems by providing a timely summary of scientific evidence. The objective of this meta-epidemiologic study was to evaluate the quality of reporting of mini-HTA reports published in Brazil.

Methods

An electronic search for all mini-HTA reports published between 2014 and March 2019 was conducted in the SISREBRATS and CONITEC databases. The study selection and data extraction were performed by two independent assessors. The following data were extracted: bibliographic data; research question; characteristics of the population, health technologies and outcomes assessed; eligibility criteria; information about searches and study selection; risk of bias assessment; quality of evidence assessment; synthesis of results; and recommendation about the technology evaluated. A descriptive analysis was used to summarize the information retrieved from all the included mini-HTA reports.

Results

We included 103 mini-HTA reports, the great majority of which (92.3 percent) focused on the coverage of the technologies in the healthcare system, with more than 60 percent being about drugs. Only five mini-HTA reports (4.8 percent) gave reasons for the choice of outcomes, and fifteen (14.5 percent) discriminated between primary and secondary outcomes. All mini-HTAs reported the databases searched and 99 percent of them reported using Medline. Sixty percent of the mini-HTA reported assessing the risk of bias, and 52 percent reported assessing the quality of evidence.

Conclusion

The quality of reporting of the mini-HTA reports performed in Brazil is insufficient and needs to be improved to guarantee transparency and replicability.

Rapid evidence synthesis approach for limits on the search date: How rapid could it be?

Rapid reviews have been widely employed to support timely decision-making, and limiting the search date is the most popular approach in published rapid reviews. We assessed the accuracy and workload of search date limits on the meta-analytical results to determine the best rapid strategy. The meta-analyses data were collected from the Cochrane Database of Systematic Reviews (CDSR). We emulated the rapid reviews by limiting the search date of the original CDSR to the recent 40, 35, 30, 25, 20, 15, 10, 7, 5, and 3 years, and their results were compared to the full meta-analyses. A random sample of 10% was drawn to repeat the literature search by the same timeframe limits to measure the relative workload reduction (RWR). The relationship between accuracy and RWR was established. We identified 21,363 meta-analyses of binary outcomes and 7683 meta-analyses of continuous outcomes from 2693 CDSRs. Our results suggested that under a maximum tolerance of 5% and 10% on the bias of magnitude, a limit on the recent 20 years can achieve good accuracy and at the same time save the most workload. Under the tolerance of 15% and 20% on the bias, a limit on the recent 10 years and 15 years could be considered. Limiting the search date is a valid rapid method to produce credible evidence for timely decisions. When conducting rapid reviews, researchers should consider both the accuracy and workload to make an appropriate decision.

Prevalence of COPD among population above 30 years in India: A systematic review and meta-analysis

Background

By 2030, Sustainable Development Goal 3.4 aims to reduce the premature mortality caused by non-communicable diseases through prevention and treatment. Chronic obstructive pulmonary disease is the second leading cause of mortality and disability-adjusted life years in India. This review was conducted to estimate the prevalence of COPD using systematic review and meta-analysis technique.

Method

Search was conducted using six databases for studies on COPD among population above 30 years in India between years 2000 to 2020. Cross-sectional and cohort studies reporting prevalence of COPD and associated risk factors were included in the present review. Screening and data extraction was done by two authors independently. Studies were appraised for quality using the modified New Castle Ottawa scale and reporting quality was assessed using STROBE guidelines.

Result

Our search returned 8973 records, from which 23 records fulfilled the eligibility criteria. Overall, the prevalence of COPD among population aged 30 years and above in India was 7%. Risk factors like active and passive smoking, biomass fuel exposure, environmental tobacco smoke, occupational exposure to dust, indoor and outdoor pollution, and increasing age were reported to have a significant association with COPD among Indian population.

Conclusion

Our findings suggest the need for a multicentric national-level research study to understand COPD burden and its contributing risk factors. The findings also suggest the need for COPD sensitive health literacy program focused on early screening and primary prevention of risk factors for COPD, which may help early initiation of self-management practices, that are crucial for better quality of life.

Impact of racism and discrimination on physical and mental health among Aboriginal and Torres Strait islander peoples living in Australia: a systematic scoping review

Background

Racism is increasingly recognised as a significant health determinant that contributes to health inequalities. In Australia efforts have been made to bridge the recognised health gap between Aboriginal and Torres Strait Islander people and other Australians. This systematic scoping review aimed to assess, synthesise, and analyse the evidence in Australia about the impacts of racism on the mental and physical health of Aboriginal and Torrens Strait Islander peoples.

Methods

A systematic search was conducted to locate Australian studies in English published between 2000 and 2020. Five electronic databases were used: PubMed, CINAHL, Embase, Web of Science and the Australia’s National Institute for Aboriginal and Torres Strait Islander Health Research. The search strategy included a combination of key words related with racism, mental health, physical health and Indigenous people. Data were extracted based on review questions and findings were synthesized in a narrative summary.

Results

Of total 338 searched studies from five databases, 12 studies met the inclusion criteria for narrative synthesis where eight were cross-sectional studies and four prospective cohorts. General mental health and general health perception were the most frequently studied outcomes followed by child behaviour, smoking and substance consumption and specific health conditions. The prevalence of racism varied between 6.9 and 97%. The most common health outcomes associated with racism were general poor mental health and poor general health perception. More specific health outcomes such as anxiety, depression, child behaviour, asthma, increased BMI and smoking were also associated with racism but were analysed by a limited number of studies. Three studies analysed psychological distress, negative mental health, sleeping difficulties and negative perceived mental health according to severity of exposition to racism.

Conclusion

Racism is associated with negative overall mental and negative general health outcomes among Aboriginal and Torres Strait Islander peoples. Strategies to prevent all forms and sources of racism are necessary to move forward to bridging the health gap between Aboriginal and Torres Strait Islander peoples and non-Indigenous Australians. Further research is needed to understand in more detail the impact of racism from an Aboriginal and Torres Strait Islander definition of health and wellbeing.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-11363-x.

What works? Interventions to reduce readmission after hip fracture: A rapid review of systematic reviews

BACKGROUND

Hip fracture is a common serious injury in older people and reducing readmission after hip fracture is a priority in many healthcare systems. Interventions which significantly reduce readmission after hip fracture have been identified and the aim of this review is to collate and summarise the efficacy of these interventions in one place.

METHODS

In a rapid review of systematic reviews one reviewer (ELS) searched the Ovid SP version of Medline and the Cochrane Database of Systematic Reviews. Titles and abstracts of 915 articles were reviewed. Nineteen systematic reviews were included. (ELS) used a data extraction sheet to capture data on interventions and their effect on readmission. A second reviewer (RK) verified data extraction in a random sample of four systematic reviews. Results were not meta-analysed. Odds and risk ratios are presented where available.

RESULTS

Three interventions significantly reduce readmission in elderly populations after hip fracture: personalised discharge planning, self-care and regional anaesthesia. Three interventions are not conclusively supported by evidence: Oral Nutritional Supplementation, integration of care, and case management. Two interventions do not affect readmission after hip fracture: Enhanced Recovery pathways and comprehensive geriatric assessment.

CONCLUSIONS

Three interventions are most effective at reducing readmissions in older people: discharge planning, self-care, and regional anaesthesia. Further work is needed to optimise interventions and ensure the most at-risk populations benefit from them, and complete development work on interventions (e.g. interventions to reduce loneliness) and intervention components (e.g. adapting self-care interventions for dementia patients) which have not been fully tested yet.

State of the evidence: a survey of global disparities in clinical trials

Introduction

Ideally, health conditions causing the greatest global disease burden should attract increased research attention. We conducted a comprehensive global study investigating the number of randomised controlled trials (RCTs) published on different health conditions, and how this compares with the global disease burden that they impose.

Methods

We use machine learning to monitor PubMed daily, and find and analyse RCT reports. We assessed RCTs investigating the leading causes of morbidity and mortality from the Global Burden of Disease study. Using regression models, we compared numbers of actual RCTs in different health conditions to numbers predicted from their global disease burden (disability-adjusted life years (DALYs)). We investigated whether RCT numbers differed for conditions disproportionately affecting countries with lower socioeconomic development.

Results

We estimate 463 000 articles describing RCTs (95% prediction interval 439 000 to 485 000) were published from 1990 to July 2020. RCTs recruited a median of 72 participants (IQR 32–195). 82% of RCTs were conducted by researchers in the top fifth of countries by socio-economic development. As DALYs increased for a particular health condition by 10%, the number of RCTs in the same year increased by 5% (3.2%–6.9%), but the association was weak (adjusted R²=0.13). Conditions disproportionately affecting countries with lower socioeconomic development, including respiratory infections and tuberculosis (7000 RCTs below predicted) and enteric infections (9700 RCTs below predicted), appear relatively under-researched for their disease burden. Each 10% shift in DALYs towards countries with low and middle socioeconomic development was associated with a 4% reduction in RCTs (3.7%–4.9%). These disparities have not changed substantially over time.

Conclusion

Research priorities are not well optimised to reduce the global burden of disease. Most RCTs are produced by highly developed countries, and the health needs of these countries have been, on average, favoured.

The Impact of COVID-19 School Closure on Child and Adolescent Health: A Rapid Systematic Review

School closures during pandemics raise important concerns for children and adolescents. Our aim is synthesizing available data on the impact of school closure during the coronavirus disease 2019 (COVID-19) pandemic on child and adolescent health globally. We conducted a rapid systematic review by searching PubMed, Embase, and Google Scholar for any study published between January and September 2020. We included a total of ten primary studies. COVID-19-related school closure was associated with a significant decline in the number of hospital admissions and pediatric emergency department visits. However, a number of children and adolescents lost access to school-based healthcare services, special services for children with disabilities, and nutrition programs. A greater risk of widening educational disparities due to lack of support and resources for remote learning were also reported among poorer families and children with disabilities. School closure also contributed to increased anxiety and loneliness in young people and child stress, sadness, frustration, indiscipline, and hyperactivity. The longer the duration of school closure and reduction of daily physical activity, the higher was the predicted increase of Body Mass Index and childhood obesity prevalence. There is a need to identify children and adolescents at higher risk of learning and mental health impairments and support them during school closures.

Rapid reviews: the pros and cons of an accelerated review process

Although systematic reviews are the method of choice to synthesize scientific evidence, they can take years to complete and publish. Clinicians, managers, and policy-makers often need input from scientific evidence in a more timely and resource-efficient manner. For this purpose, rapid reviews are conducted. Rapid reviews are performed using an accelerated process. However, they should not be less systematic than standard systematic reviews, and the introduction of bias must be avoided. In this article, we describe what rapid reviews are, present their characteristics, give some examples, highlight potential pitfalls, and draw attention to the importance of evidence summaries in order to facilitate adoption in clinical decision-making.

Successful incorporation of single reviewer assessments during systematic review screening: development and validation of sensitivity and work-saved of an algorithm that considers exclusion criteria and count

BACKGROUND

Accepted systematic review (SR) methodology requires citation screening by two reviewers to maximise retrieval of eligible studies. We hypothesized that records could be excluded by a single reviewer without loss of sensitivity in two conditions; the record was ineligible for multiple reasons, or the record was ineligible for one or more specific reasons that could be reliably assessed.

METHODS

Twenty-four SRs performed at CHEO, a pediatric health care and research centre in Ottawa, Canada, were divided into derivation and validation sets. Exclusion criteria during abstract screening were sorted into 11 specific categories, with loss in sensitivity determined by individual category and by number of exclusion criteria endorsed. Five single reviewer algorithms that combined individual categories and multiple exclusion criteria were then tested on the derivation and validation sets, with success defined a priori as less than 5% loss of sensitivity.

RESULTS

The 24 SRs included 930 eligible and 27390 ineligible citations. The reviews were mostly focused on pediatrics (70.8%, N=17/24), but covered various specialties. Using a single reviewer to exclude any citation led to an average loss of sensitivity of 8.6% (95%CI, 6.0-12.1%). Excluding citations with ≥2 exclusion criteria led to 1.2% average loss of sensitivity (95%CI, 0.5-3.1%). Five specific exclusion criteria performed with perfect sensitivity: conference abstract, ineligible age group, case report/series, not human research, and review article. In the derivation set, the five algorithms achieved a loss of sensitivity ranging from 0.0 to 1.9% and work-saved ranging from 14.8 to 39.1%. In the validation set, the loss of sensitivity for all 5 algorithms remained below 2.6%, with work-saved between 10.5% and 48.2%.

CONCLUSIONS

Findings suggest that targeted application of single-reviewer screening, considering both type and number of exclusion criteria, could retain sensitivity and significantly decrease workload. Further research is required to investigate the potential for combining this approach with crowdsourcing or machine learning methodologies.

Burden of COPD among population above 30 years in India: protocol for a systematic review and proposed meta-analysis

BACKGROUND

The Sustainable Development Goals and the National Health Policy of India aim to reduce premature mortality from noncommunicable diseases (NCD) by one-third in the next decade and by 25% by 2025, respectively. Among NCDs globally, chronic obstructive pulmonary disease (COPD) is a major contributor to death and disability. This underscores the need to understand the burden of COPD at the national level by synthesizing evidence and collating the state-wise COPD data to estimate the prevalence of COPD and to highlight the associated risk factors to inform policymakers.

METHOD

The systematic literature search will be carried out in PubMed, Cochrane, Scopus, Web of Science, CINAHL, and ProQuest databases with restrictions for studies published between 2000 and 2020 and available in English. Cross-sectional or cohort studies conducted in and among the Indian population aged 30 years and above will be included. Case reports, randomized trials, meta-analysis, commentaries, and qualitative studies will be excluded from the review. Quality assessment of the included studies will be performed using New Castle Ottawa scale and adherence to reporting standards will be checked using STROBE checklist for Observational Cohort and Cross-Sectional Studies.

DISCUSSION

Prevalence of COPD in the population aged 30 years and above, diagnosed through spirometry and nonspirometry, will be compared and reported and a meta-analysis will be performed to obtain pooled prevalence rates of COPD and the risk factors associated with COPD.

Rapid reviews: A critical perspective

BACKGROUND

The high scientific uncertainty of many far-reaching and serious political decisions during the "coronavirus crisis" underpins the enormous importance of having evidence syntheses that are quickly available and at the same time reliable. As these requirements can only be insufficiently fulfilled by systematic reviews due to the high amount of time required, abbreviated evidence syntheses in the form of rapid reviews are becoming increasingly popular.

PURPOSE

This commentary aims to enhance methodological and methodical discussions and research about abbreviated evidence syntheses.

METHODS

A selective literature search and evaluation focussing on research dealing with rapid reviews.

RESULTS

In rapid reviews, a wide variety of methods can be used to speed up the process of literature search and evaluation, while at the same time maintaining the principles of methodological quality and transparent reporting. But do rapid reviews currently keep what they promise? We discuss the increasing trend towards rapid reviews, giving the currently available evidence on the topic some critical reflection. Following this discussion, we will finally derive demands that go beyond the topic of rapid reviews alone.

Cochrane Rapid Reviews Methods Group offers evidence-informed guidance to conduct rapid reviews

OBJECTIVES

To develop methods guidance to support the conduct of rapid reviews (RRs) produced within Cochrane and beyond, in response to requests for timely evidence syntheses for decision-making purposes including urgent health issues of high priority.

STUDY DESIGN AND SETTING

Interim recommendations were informed by a scoping review of the underlying evidence, primary methods studies conducted, and a survey sent to 119 representatives from 20 Cochrane entities, who were asked to rate and rank RR methods across stages of review conduct. Discussions among those with expertise in RR methods further informed the list of recommendations with accompanying rationales provided.

RESULTS

Based on survey results from 63 respondents (53% response rate), 26 RR methods recommendations are presented for which there was a high or moderate level of agreement or scored highest in the absence of such agreement. Where possible, how recommendations align with Cochrane methods guidance for systematic reviews is highlighted.

CONCLUSION

The Cochrane Rapid Reviews Methods Group offers new, interim guidance to support the conduct of RRs. Because best practice is limited by the lack of currently available evidence for some RR methods shortcuts taken, this guidance will need to be updated as additional abbreviated methods are evaluated.

Low-Value Clinical Practices in Adult Traumatic Brain Injury: An Umbrella Review

Despite numerous interventions and treatment options, the outcomes of traumatic brain injury (TBI) have improved little over the last 3 decades, which raises concern about the value of care in this patient population. We aimed to synthesize the evidence on 14 potentially low-value clinical practices in TBI care. Using umbrella review methodology, we identified systematic reviews evaluating the effectiveness of 14 potentially low-value practices in adults with acute TBI. We present data on methodological quality (Assessing the Methodological Quality of Systematic Reviews), reported effect sizes, and credibility of evidence (I to IV). The only clinical practice with evidence of benefit was therapeutic hypothermia (credibility of evidence II to IV). However, the most recent meta-analysis on hypothermia based on high-quality trials suggested harm (credibility of evidence IV). Meta-analyses on platelet transfusion for patients on antiplatelet therapy were all consistent with harm but were statistically non-significant. For the following practices, effect estimates were consistently close to the null: computed tomography (CT) in adults with mild TBI who are low-risk on a validated clinical decision rule; repeat CT in adults with mild TBI on anticoagulant therapy with no clinical deterioration; antibiotic prophylaxis for external ventricular drain placement; and decompressive craniectomy for refractory intracranial hypertension. We identified five clinical practices with evidence of lack of benefit or harm. However, evidence could not be considered to be strong for any clinical practice as effect measures were imprecise and heterogeneous, systematic reviews were often of low quality, and most included studies had a high risk of bias.

Impact of racism and discrimination on the physical and mental health among Aboriginal and Torres Strait Islander peoples living in Australia: a protocol for a scoping review

BACKGROUND

Racism is increasingly recognised internationally as a key factor contributing to health disparities. A comprehensive body of strong research from international authors has reported negative associations between racism and health outcomes. In Australia, although the literature is more limited, available findings follow global trends. Australia has an identified health gap between Aboriginal and Torres Strait Islander peoples and non-indigenous Australians, and despite efforts to bridge this gap, health inequities continue to exist. This scoping review aims to assess, analyse and synthesise the relationship between racism and discrimination on the physical and mental health of Aboriginal and Torres Strait Islander peoples living in Australia.

METHOD

This is the study protocol for a scoping review. A systematic search will be conducted using five electronic databases: PubMed, CINAHL, Embase, Web of Science and the Australia's National Institute for Aboriginal and Torres Strait Islander Health Research. The database search will include studies published between 2000 and 2020. Reference lists of the included articles will be searched. Outcome measures will include physical and mental health components including chronic conditions, depression and anxiety, psychological distress, social and emotional difficulties, suicide and health-related outcomes such as wellbeing and life satisfaction. Duplications will be removed, and titles and abstracts will be reviewed to select studies. Full-text screening of preselected studies will be performed by four reviewers independently, to select studies according to inclusion criteria. Included studies will be appraised for quality using appropriate tools tailored for each study design. Data will be extracted, and study findings and characteristics synthesised in a narrative summary.

DISCUSSION

Our scoping review will synthesise the evidence on the impacts of racism and discrimination in relation to the physical and mental health of Aboriginal and Torres Strait Islander peoples living in Australia. These findings could guide future health interventions by addressing the exposure of racism and racial discrimination in order to reduce health disparity. It is anticipated the findings to be of interest to policymakers, researchers, Aboriginal and Torres Strait Islander communities and community health organisations and other stakeholders interested in optimising public health interventions for and in partnership with Aboriginal and Strait Torres Islander communities of Australia.

SCOPING REVIEW REGISTRATION

The protocol for this review has been registered on the international prospective register of systematic reviews (PROSPERO). The registration ID is CRD42020186193 .

Abbreviated and comprehensive literature searches led to identical or very similar effect estimates: a meta-epidemiological study

OBJECTIVES

The objective of this study was to assess the agreement of treatment effect estimates from meta-analyses based on abbreviated or comprehensive literature searches.

STUDY DESIGN AND SETTING

This was a meta-epidemiological study. We abbreviated 47 comprehensive Cochrane review searches and searched MEDLINE/Embase/CENTRAL alone, in combination, with/without checking references (658 new searches). We compared one meta-analysis from each review with recalculated ones based on abbreviated searches.

RESULTS

The 47 original meta-analyses included 444 trials (median 6 per review [interquartile range (IQR) 3-11]) with 360045 participants (median 1,371 per review [IQR 685-8,041]). Depending on the search approach, abbreviated searches led to identical effect estimates in 34-79% of meta-analyses, to different effect estimates with the same direction and level of statistical significance in 15-51%, and to opposite effects (or effects could not be estimated anymore) in 6-13%. The deviation of effect sizes was zero in 50% of the meta-analyses and in 75% not larger than 1.07-fold. Effect estimates of abbreviated searches were not consistently smaller or larger (median ratio of odds ratio 1 [IQR 1-1.01]) but more imprecise (1.02-1.06-fold larger standard errors).

CONCLUSION

Abbreviated literature searches often led to identical or very similar effect estimates as comprehensive searches with slightly increased confidence intervals. Relevant deviations may occur.

Assessing the risk of bias in choice of search sources for environmental meta-analyses

Results of meta-analyses are potentially valuable for informing environmental policy and practice decisions. However, selective sampling of primary studies through searches exclusively using widely used bibliographic platform(s) could bias estimates of effect sizes. Such search strategies are common in environmental evidence reviews, and if risk of bias can be detected, this would provide the first empirical evidence that comprehensiveness of searches needs to be improved. We compare the impact of using single and multiple bibliographic platform(s) searches vs more comprehensive searches on estimates of mean effect sizes. We used 137 published meta-analyses, based on multiple source searches, analyzing 9388 studies: 8095 sourced from commercially published articles; and 1293 from grey literature and unpublished data. Single-platform and multiple-platform searches missed studies in 100 and 80 of the meta-analyses, respectively: 52 and 46 meta-analyses provided larger-effect estimates; 32 and 28 meta-analyses provided smaller-effect estimates; eight and four meta-analyses provided opposite direction of estimates; and two each were unable to estimate effects due to missing all studies. Further, we found significant positive log-linear relationships between proportions of studies missed and the deviations of mean effect sizes, suggesting that as the number of studies missed increases, deviation of mean effect size is likely to expand. We also found significant differences in mean effect sizes between indexed and non-indexed studies for 35% of meta-analyses, indicating high risk of bias when the searches were restricted. We conclude that the restricted searches are likely to lead to unrepresentative samples of studies and biased estimates of true effects.

Exclusion of studies with no events in both arms in meta-analysis impacted the conclusions

OBJECTIVES

Classical meta-analyses routinely treated studies with no events in both arms noninformative and excluded them from analyses. This study assessed whether such studies contain information and have an influence on the conclusions of meta-analyses.

STUDY DESIGN AND SETTING

We collected meta-analyses of binary outcomes with at least one study having no events in both arms from Cochrane systematic reviews (2003-2018). We used the generalized linear mixed model to reanalyze these meta-analyses by two approaches: one including studies with no events in both arms and one excluding such studies. The magnitude and direction of odds ratio (OR), P value, and width of 95% confidence interval (CI) were compared. A simulation study was conducted to examine the robustness of results.

RESULTS

We identified 442 meta-analyses. In comparing paired meta-analyses that included studies with no events in both arms vs. those not, 8 (1.80%) resulted in different directions on OR; 41 (9.28%) altered conclusions on statistical significance. Substantial changes occurred on P value (55.66% increased and 44.12% decreased) and the width of 95% CI (50.68% inflated and 49.32% declined) when excluding studies with no events. Simulation study confirmed these findings.

CONCLUSION

Studies with no events in both arms are not necessarily noninformative. Excluding such studies may alter conclusions.

You Can Teach an Old Dog New Tricks: Rank Fusion applied to Coordination Level Matching for Ranking in Systematic Reviews

Coordination level matching is a ranking method originally proposed to rank documents given Boolean queries that is now several decades old. Rank fusion is a relatively recent method for combining runs from multiple systems into a single ranking, and has been shown to significantly improve the ranking. This paper presents a novel extension to coordination level matching, by applying rank fusion to each sub-clause of a Boolean query. We show that, for the tasks of systematic review screening prioritisation and stopping estimation, our method significantly outperforms the state-of-the-art learning to rank and bag-of-words-based systems for this domain. Our fully automatic, unsupervised method has (i) the potential for significant real-world cost savings (ii) does not rely on any intervention from the user, and (iii) is significantly better at ranking documents given only a Boolean query in the context of systematic reviews when compared to other approaches.

Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy

BACKGROUND

The large and increasing number of new studies published each year is making literature identification in systematic reviews ever more time-consuming and costly. Technological assistance has been suggested as an alternative to the conventional, manual study identification to mitigate the cost, but previous literature has mainly evaluated methods in terms of recall (search sensitivity) and workload reduction. There is a need to also evaluate whether screening prioritization methods leads to the same results and conclusions as exhaustive manual screening. In this study, we examined the impact of one screening prioritization method based on active learning on sensitivity and specificity estimates in systematic reviews of diagnostic test accuracy.

METHODS

We simulated the screening process in 48 Cochrane reviews of diagnostic test accuracy and re-run 400 meta-analyses based on a least 3 studies. We compared screening prioritization (with technological assistance) and screening in randomized order (standard practice without technology assistance). We examined if the screening could have been stopped before identifying all relevant studies while still producing reliable summary estimates. For all meta-analyses, we also examined the relationship between the number of relevant studies and the reliability of the final estimates.

RESULTS

The main meta-analysis in each systematic review could have been performed after screening an average of 30% of the candidate articles (range 0.07 to 100%). No systematic review would have required screening more than 2308 studies, whereas manual screening would have required screening up to 43,363 studies. Despite an average 70% recall, the estimation error would have been 1.3% on average, compared to an average 2% estimation error expected when replicating summary estimate calculations.

CONCLUSION

Screening prioritization coupled with stopping criteria in diagnostic test accuracy reviews can reliably detect when the screening process has identified a sufficient number of studies to perform the main meta-analysis with an accuracy within pre-specified tolerance limits. However, many of the systematic reviews did not identify a sufficient number of studies that the meta-analyses were accurate within a 2% limit even with exhaustive manual screening, i.e., using current practice.

Low-value clinical practices in adult traumatic brain injury: an umbrella review protocol

INTRODUCTION

Traumatic brain injury (TBI) leads to 50 000 deaths, 85 000 disabilities and costs $60 billion each year in the USA. Despite numerous interventions and treatment options, the outcomes of TBI have improved little over the last three decades. In a previous scoping review and expert consultation survey, we identified 13 potentially low-value clinical practices in acute TBI. The objective of this umbrella review is to synthesise the evidence on potentially low-value clinical practices in the care of acute TBI.

METHODS AND ANALYSIS

Using umbrella review methodology, we will search Cochrane Central Register of Controlled Trials, Embase, Epistemonikos, International Prospective Register of Systematic Reviews (PROSPERO) and PubMed to identify systematic reviews evaluating the effect of potential intrahospital low-value practices using tailored population, intervention, comparator, outcome and study design questions based on the results of a previous scoping review. We will present data on the methodological quality of these reviews (Assessing the Methodological Quality of Systematic Reviews-2), reported effect sizes and strength of evidence (Grading of Recommendations, Assessment, Development and Evaluation).

ETHICS AND DISSEMINATION

Ethics approval is not required as original data will not be collected. Knowledge users from five healthcare quality organisations and clinical associations are involved in the design and conduct of the study. Results will be disseminated in a peer-reviewed journal, at international scientific meetings and to clinical, healthcare quality and patient-partner associations. This work will support the development of metrics to measure the use of low-value practices, inform policy makers on potential targets for deimplementation and in the long term reduce the use of low-value clinical practices in acute TBI care.

PROSPERO REGISTRATION NUMBER

CRD42019132428.

Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management

BACKGROUND

While systematic review (SR) methods are gaining traction as a method for providing a reliable summary of existing evidence for health risks posed by exposure to chemical substances, it is becoming clear that their value is restricted to a specific range of risk management scenarios - in particular, those which can be addressed with tightly focused questions and can accommodate the time and resource requirements of a systematic evidence synthesis.

METHODS

The concept of a systematic evidence map (SEM) is defined and contrasted to the function and limitations of systematic review (SR) in the context of risk management decision-making. The potential for SEMs to facilitate evidence-based decision-making are explored using a hypothetical example in risk management priority-setting. The potential role of SEMs in reference to broader risk management workflows is characterised.

RESULTS

SEMs are databases of systematically gathered research which characterise broad features of the evidence base. Although not intended to substitute for the evidence synthesis element of systematic reviews, SEMs provide a comprehensive, queryable summary of a large body of policy relevant research. They provide an evidence-based approach to characterising the extent of available evidence and support forward looking predictions or trendspotting in the chemical risk sciences. In particular, SEMs facilitate the identification of related bodies of decision critical chemical risk information which could be further analysed using SR methods, and highlight gaps in the evidence which could be addressed with additional primary studies to reduce uncertainties in decision-making.

CONCLUSIONS

SEMs have strong and growing potential as a high value tool in resource efficient use of existing research in chemical risk management. They can be used as a critical precursor to efficient deployment of high quality SR methods for characterising chemical health risks. Furthermore, SEMs have potential, at a large scale, to support the sort of evidence summarisation and surveillance methods which would greatly increase the resource efficiency, transparency and effectiveness of regulatory initiatives such as EU REACH and US TSCA.

Rapid network meta-analysis using data from Food and Drug Administration approval packages is feasible but with limitations

OBJECTIVE

To test rapid approaches that use Drugs@FDA (a public database of approved drugs) and ClinicalTrials.gov to identify trials and to compare these two sources with bibliographic databases as an evidence base for a systematic review and network meta-analysis (NMA).

STUDY DESIGN AND SETTING

We searched bibliographic databases, Drugs@FDA, and ClinicalTrials.gov for eligible trials on first-line glaucoma medications. We extracted data, assessed risk of bias, and examined the completeness and consistency of information provided by different sources. We fitted random-effects NMA models separately for trials identified from each source and for all unique trials from three sources.

RESULTS

We identified 138 unique trials including 29,394 participants on 15 first-line glaucoma medications. For a given trial, information reported was sometimes inconsistent across data sources. Journal articles provided the most information needed for a systematic review; trial registrations provided the least. Compared to an NMA including all unique trials, we were able to generate reasonably precise effect estimates and similar relative rankings for available interventions using trials from Drugs@FDA alone (but not ClinicalTrials.gov).

CONCLUSIONS

A rapid NMA approach using data from Drugs@FDA is feasible but has its own limitations. Reporting of trial design and results can be improved in both the drug approval packages and on ClinicalTrials.gov.

Challenges of rapid reviews for diagnostic test accuracy questions: a protocol for an international survey and expert consultation

BACKGROUND

Assessment of diagnostic tests, broadly defined as any element that aids in the collection of additional information for further clarification of a patient's health status, has increasingly become a critical issue in health policy and decision-making. Diagnostic evidence, including the accuracy of a medical test for a target condition, is commonly appraised using standard systematic review methodology. Owing to the considerable time and resources required to conduct these, rapid reviews have emerged as a pragmatic alternative by tailoring methods according to the decision maker's circumstances. However, it is not known if streamlining methodological aspects has an impact on the validity of evidence synthesis. Furthermore, due to the particular nature and complexity of the appraisal of diagnostic accuracy, there is need for detailed guidance on how to conduct rapid reviews of diagnostic tests. In this study, we aim to identify the methods currently used by rapid review developers to synthesize evidence on diagnostic test accuracy, as well as to analyze potential shortcomings and challenges related to these methods.

METHODS

We will carry out a two-fold approach: (1) an international survey of professionals working in organizations that develop rapid reviews of diagnostic tests, in terms of the methods and resources used by these agencies when conducting rapid reviews, and (2) semi-structured interviews with senior-level individuals to further explore and validate the findings from the survey and to identify challenges in conducting rapid reviews. We will use STATA 15.0 for quantitative analyses and framework analysis for qualitative analyses. We will ensure protection of data during all stages.

DISCUSSION

The main result of this research will be a map of methods and resources currently used for conducting rapid reviews of diagnostic test accuracy, as well as methodological shortcomings and potential solutions in diagnostic knowledge synthesis that require further research.