The geometric increase in meta-analyses from China in the genomic era

The sound and surprise: overlapping meta-analyses on the topic of safety and efficacy of PD-1 and PD-L1 inhibitors in the treatment of non-small cell lung cancer

PURPOSE

To analyze the characteristics of overlapping meta-analyses based on randomized controlled trials (RCTs) which reported PD-1/PD-L1 inhibitors in non-small cell cancer (NSCLC).

METHODS

Meta-analyses were identified from English and Chinese databases until January 1, 2022. Differences in characteristics of overlapping meta-analyses that conducted in China and other countries were compared to assess their publication propensity. The corrected covered area (CCA) and coverage of relevant RCTs were analyzed for subtopics according to detailed intervention types. The waste and redundancy of evidence were assessed in the case of PD-1/PD-L1 inhibitor monotherapy for second-line treatment for NSCLC.

RESULTS

Fifty-nine meta-analyses published in English and 17 meta-analyses published in Chinese reporting 26 RCTs were identified. Fifty-three (69.74%) meta-analyses were conducted in China. The overlapping meta-analyses in China were more likely to be from hospitals, supported by government funding, integrate first and second-line therapies. Five of the six subtopics had overlapping meta-analyses according to specific types of interventions. The CCA of overlapping meta-analyses ranged from 33.33 to 63.19%, and the coverage of relevant RCTs ranged from 63.64 to 100%. All the conclusions of overlapping meta-analyses have been consistent in the subtopic of PD-1/PD-L1 inhibitor monotherapy for second-line treatment since 2017.

CONCLUSION

Overlapping meta-analyses of PD-1/PD-L1 inhibitors in NSCLC hints that meta-analyses under this topic probably exist serious redundancy. Future research should focus on prospective registration of protocols for systematic reviews/meta-analyses, scientific designed PICO, and cumulative meta-analysis to reduce redundant and wasted studies. Journals should strengthen the requirement for reviewing previously published evidence in manuscript review.

Factors associated with successful publication for systematic review protocol registration: an analysis of 397 registered protocols

BACKGROUND

Meta-analyses are on top of the evidence-based medicine pyramid, yet many of them are not completed after they are begun. Many factors impacting the publication of meta-analysis works have been discussed, and their association with publication likelihood has been investigated. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication. In our current review, we aim to investigate these various factors and their impact on the likelihood of publication. A comprehensive review of 397 registered protocols retrieved from five databases was performed to investigate the different factors that might affect the likelihood of publication. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication.

RESULTS

We found that corresponding authors in developed countries and English-speaking countries had higher likelihoods of publication: 206/320 (p = 0.018) and 158/236 (p = 0.006), respectively. Factors affecting publications are the countries of corresponding author (p = 0.033), whether they are from developed countries (OR: 1.9, 95% CI: 1.2-3.1, p = 0.016), from English-speaking countries (OR: 1.8, 95% CI: 1.2-2.7, p = 0.005), update status of the protocol (OR: 1.6, 95% CI: 1.0-2.6, p = 0.033), and external funding (OR: 1.7, 95% CI: 1.1-2.7, p = 0.025). Multivariable regression retains three variables as significant predictors for the publication of a systematic review: whether it is the corresponding author from developed countries (p = 0.013), update status of the protocol (p = 0.014), and external funding (p = 0.047).

CONCLUSION

Being on top of the evidence hierarchy, systematic review and meta-analysis are the keys to informed clinical decision-making. Updating protocol status and external funding are significant influences on their publications. More attentions should be paid to the methodological quality of this type of publication.

High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study

BACKGROUND

Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations.

METHODS

PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics.

RESULTS

A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001).

CONCLUSION

Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.

COVID 19 Scoping: A Systemic Review and Meta-Analysis

Introduction:

The world has faced the pandemic of COVID-19 in the march of 2020 and still it continues to effect in 2021. Hence, in the present study we aim to evaluate the gulps in the research so that certain recommendations can be made for the future research. We conducted a scoping review of the COVID meta-analysis.

Materials and Methods:

Online data was collected from the search engines of EBSCO, PubMed, Google Scholar, and Scopus. The searched terms were COVID-19, CORONA, SARS-CoV-2, clinical features, Wuhan, etc. The study articles were collected that from January 2020 to February 2021. Based on the PRISMA guidelines, the meta-analysis was performed.

Results:

In the present study, we finalized 316 articles. On February 2020, the first article was published. We observed a spike in the meta-analysis later on. Most of the meta-analysis were issued in the virology and infection magazines. As expected, the majority studies were from Wuhan. The other countries that published the meta-analysis were the USA, the UK, and Italy. The studies included in each meta-analysis were nearly 25 and the subjects were approximately 16 thousand. However, we noticed a poor quality in majority of these meta-analysis and <10% of all the meta-analysis showed higher confidence.

Conclusion:

A poor quality of the meta-analysis has predominated the data and very few are of high quality. All the journal editors and the reviewing team should verify and thoroughly organize the protocol so that only high quality meta-analysis are encouraged.

Clinical Epidemiology in China series. Paper 3: The methodological and reporting quality of systematic reviews and meta-analyses published by China' researchers in English-language is higher than those published in Chinese-language

OBJECTIVE

To assess the methodological and reporting quality of Chinese- and English -language systematic reviews and meta-analyses (SRs/MAs) published by Chinese authors between 2016 and 2018.

STUDY DESIGN AND SETTING

We searched MEDLINE and Chinese Science Citation Database (CSCD) for SRs/MAs led by Chinese authors published between 2016 and 2018. We used random sampling to select 10% of the eligible SRs/MAs published in each year from CSCD, and then matched the same number of SRs/MAs in MEDLINE. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR-2) tool. Stratified analyses were conducted to compare the differences of quality between Chinese- and English language SRs/MAs.

RESULTS

We identified 336 SRs/MAs (168 in Chinese and 168 in English). The reporting quality in Chinese-language SRs/MAs was slightly lower than English-language SRs/MAs (mean PRISMA scores: 20.58 vs. 21.71 in 2016, 19.87 vs. 21.24 in 2017, and 21.29 vs. 22.38 in 2018). Less than half of both Chinese- and English-language SRs/MAs complied with item 5 (protocol and registration), item 7 (information sources), item 8 (search) and item 27 (funding)). The methodological quality in Chinese -language SRs/MAs was also slightly lower than English -language SRs/MAs (mean AMSTAR-2 scores: 8.07 vs. 9.36 in 2016; 9.21 vs. 10.26 in 2017; 8.86 vs. 9.28 in 2018). Three items (item 2: established a protocol; item 4: use a comprehensive literature search; and item 10: report the sources of funding) were adhered to by less than 10% of both Chinese- and English -language SRs/MAs. Only one (0.6%) Chinese-language SRs/MA and nine (5.4%) English-language SRs/MAs were rated as high methodological quality.

CONCLUSION

The reporting and methodological quality of English-language SRs/MAs conducted by authors from China between 2016 and 2018 were slightly better than those of Chinese -language SRs/MAs.

Publication of meta-analyses in sleep medicine: a scoping review

STUDY OBJECTIVES

Sleep research has grown substantially in recent decades, producing a large amount of data and an increasing number of meta-analyses. This study sought to establish the volume of meta-analyses in this area and assess how this level of material has developed over time.

METHODS

A bibliographic search of the Web of Science database was conducted (1945-2019). The total number of articles and the total number of meta-analyses were extracted for both sleep medicine and a combination of 6 other medical specialties (cardiology, neurology, psychiatry, pulmonology, otorhinolaryngology, and pediatrics).

RESULTS

A total of 262,384 articles and 1,152 meta-analyses related to sleep medicine were identified. Considering the whole period under analysis, meta-analyses represented 0.44% of the total number of sleep medicine-related articles. Throughout this period, the proportion of meta-analyses published has been increasing in both sleep medicine and the other fields, but it is greater in the other fields. In 2019, meta-analyses in sleep medicine represented 1.10% of the publication output in this area but represented 1.62% of the other areas. However, sleep medicine's growth rate has been consistently higher than in the other fields. The United States, China, and the United Kingdom have been the top meta-analysis producers.

CONCLUSIONS

Meta-analyses in sleep medicine are underused. As a recent medical field, sleep medicine has more potential to grow and is likely to grow faster than other fields. Researchers should be encouraged to perform and publish meta-analyses on sleep medicine, as long as the analyses are reasonable and feasible from methodological, statistical. and practical perspectives.

COVID-19 meta-analyses: a scoping review and quality assessment

OBJECTIVE

To carry out a scoping review of the meta-analyses published regarding about coronavirus disease 2019 (COVID-19), evaluating their main characteristics, publication trends and methodological quality.

METHODS

A bibliometric search was performed in PubMed®, Scopus and Web of Science, focusing on meta-analyses about COVID-2019 disease. Bibliometric and descriptive data for the included articles were extracted and the methodological quality of the included meta-analyses was evaluated using A Measurement Tool to Assess Systematic Reviews.

RESULTS

A total of 348 meta-analyses were considered eligible. The first meta-analysis about COVID-19 disease was published on February 26, 2020, and the number of meta-analyses has grown rapidly since then. Most of them were published in infectious disease and virology journals. The greatest number come from China, followed by the United States, Italy and the United Kingdom. On average, these meta-analyses included 23 studies and 15,200 participants. Overall quality was remarkably low, and only 8.9% of them could be considered as of high confidence level.

CONCLUSION

Although well-designed meta-analyses about COVID-19 disease have already been published, the majority are of low quality. Thus, all stakeholders playing a role in COVID-19 deseases, including policy makers, researchers, publishers and journals, should prioritize well-designed meta-analyses, performed only when the background information seem suitable, and discouraging those of low quality or that use suboptimal methods.

Current Issues on Research Conducted to Improve Women's Health

There are varied lessons to be learned regarding the current methodological approaches to women’s health research. In the present scheme of growing medical literature and inflation of novel results claiming significance, the sheer amount of information can render evidence-based practice confusing. The factors that classically determined the impact of discoveries appear to be losing ground: citation count and publication rates, hierarchy in author lists according to contribution, and a journal’s impact factor. Through a comprehensive literature search on the currently available data from theses, opinion, and original articles and reviews on this topic, we seek to present to clinicians a narrative synthesis of three crucial axes underlying the totality of the research production chain: (a) critical advances in research methodology, (b) the interplay of academy and industry in a trial conduct, and (c) review- and publication-associated developments. We also provide specific recommendations on the study design and conduct, reviewing the processes and dissemination of data and the conclusions and implementation of findings. Overall, clinicians and the public should be aware of the discourse behind the marketing of alleged breakthrough research. Still, multiple initiatives, such as patient review and strict, supervised literature synthesis, have become more widely accepted. The “bottom-up” approach of a wide dissemination of information to clinicians, together with practical incentives for stakeholders with competing interests to collaborate, promise to improve women’s healthcare.

Redundant meta-analyses are common in genetic epidemiology

OBJECTIVES

The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology.

STUDY DESIGN AND SETTING

Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association.

RESULTS

Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones.

CONCLUSION

These results suggest that duplication is common in meta-analyses of genetic associations.

Genetic variations in MicroRNA genes and cancer risk: A field synopsis and meta-analysis

BACKGROUND

Cancer risk has been associated with certain gene variations in microRNA (miRNA), but conflicting evidence warrants re-assessing of significant results in meta-analyses. We summarized published meta-analyses that assess the associations between miRNA polymorphism and cancers to show the validity of the findings.

METHOD

We searched PubMed and investigated the results of meta-analyses published through November 2018. We re-assessed the results based on false-positive report probability (FPRP) to test the noteworthiness of the associations.

RESULTS

Sixty-eight miRNA polymorphisms in 45 meta-analyses associated with cancer were included. Four (7.4%) and sixteen (25.0%) single nucleotide polymorphisms (SNPs) were noteworthy (FPRP < 0.2) at a prior probability of 0.001 for interesting candidate genes and a statistical power to detect an odds ratio (OR) of 1.1 and 1.5, respectively. The four miRNA SNPs noteworthy at an OR of 1.1 were as follows: miR-146a/rs2910164 Cvs.G; miR-27a/rs895819 Cvs.T; miR-423/rs6505162 Cvs.A; and miR-605/rs2043556 Cvs.T. The 16 SNPs noteworthy at an OR of 1.5 include the four genotype comparisons at an OR of 1.1, and the additional 12 genotype comparisons were as follows: miR-196a2/rs11614913 Tvs.C; miR-27a/rs895819 GGvs.AA + AG; miR-196a2/rs11614913 C vs.T; miR-146a/rs2910164 Gvs.C; miR-196a2/rs11614913 Tvs.C; miR-146a/rs2910164 Cvs.G; miR-499/rs3746444 homozygous model; miR-146a/rs2910164 CCvs.GG + GC; miR-499/rs3746444 TCvs.TT; miR-499/rs3746444 GAvs.AA; miR-146a/rs2910164 CCvs.GG; and miR-499/rs3746444 Gvs.A. No association was noteworthy at a prior probability of 0.000001.

CONCLUSION

Out of 68 published associations of miRNA polymorphisms with cancer, sixteen have shown noteworthiness in our re-assessing meta-analysis. Our findings summarize the results of meta-analyses on the association of cancer with SNPs and underline the importance of interpreting results with caution.

Mendel's laws, Mendelian randomization and causal inference in observational data: substantive and nomenclatural issues

We respond to criticisms of Mendelian randomization (MR) by Mukamal, Stampfer and Rimm (MSR). MSR consider that MR is receiving too much attention and should be renamed. We explain how MR links to Mendel's laws, the origin of the name and our lack of concern regarding nomenclature. We address MSR's substantive points regarding MR of alcohol and cardiovascular disease, an issue on which they dispute the MR findings. We demonstrate that their strictures with respect to population stratification, confounding, weak instrument bias, pleiotropy and confounding have been addressed, and summarise how the field has advanced in relation to the issues they raise. We agree with MSR that "the hard problem of conducting high-quality, reproducible epidemiology" should be addressed by epidemiologists. However we see more evidence of confrontation of this issue within MR, as opposed to conventional observational epidemiology, within which the same methods that have demonstrably failed in the past are simply rolled out into new areas, leaving their previous failures unexamined.

An observational study found large methodological heterogeneity in systematic reviews addressing prevalence and cumulative incidence

OBJECTIVES

The objective of this study was to assess reporting and methodological aspects of systematic reviews (SRs) on prevalence and cumulative incidence data.

STUDY DESIGN AND SETTING

We searched PubMed up to 18 April, 2018, and drew a random sample of eligible SRs.

RESULTS

The included 215 SRs were reported in 187 different journals. 58.1% were published between 2015 and 2018. Few SRs were registered with PROSPERO (5.6%). One-quarter considered articles without languages restrictions (25.1%). Regional restrictions of included studies were applied in 22.8%. A meta-analysis was carried out in 40.5% of the SRs. One hundred and six studies (49.3%) assessed risk of bias or study quality. A total of 41 different existing tools as well as 15 tools developed by the authors themselves were used. The most commonly applied tools were the Newcastle-Ottawa Scale (15.1%), STROBE (13.5%), and the Cochrane Collaboration's tool for assessing risk of bias (7.9%).

CONCLUSION

We found large heterogeneity in characteristics, reporting, and methodological aspects of SRs on prevalence and cumulative incidence data, especially when compared with other types of SRs. Newly developed or revised guidance on how to conduct and report SRs as well as instruments for critical appraisal should consider the diversity of review types.

Characteristics, trend, and methodological quality of systematic reviews and meta-analyses in nuclear medicine: A bibliometric analysis of studies published between 2005 and 2016

To evaluate the characteristics, trend, and quality of systematic reviews and meta-analyses in nuclear medicine.We performed a PubMed search to identify systematic reviews and meta-analyses published between 2005 and 2016 in the field of nuclear medicine. The following data were extracted: journal name, impact factor, type of study, topics with cancer type, imaging modalities, authors (number, country, affiliation, presence of nuclear medicine specialists and statisticians, discordance between the first and corresponding authors), funding, methodological quality, methods used for quality assessment, and statistical methods.We included 185 nuclear medicine articles. Meta-analyses (n = 164; 88.6%) were published about 7 times more frequently than systematic reviews. Oncology was the most commonly studied topic (n = 125, 67.6%). The first authors were most frequently located in China (n = 73; 39.5%). PET was the most commonly used modality (n = 150; 81.1%). Both the number of authors and the ratio of discordance between the first and corresponding authors tended to progressively increase over time.The mean AMSTAR score increased over time (5.77 in 2005-2008, 6.71 in 2009-2012, and 7.44 in 2013-2016). The proportion of articles with quality assessment increased significantly (20/26 in 2005-2008, 54/65 in 2009-2012, and 79/94 in 2013-2016). The most commonly used assessment tool was quality assessment of diagnostic accuracy studies (n = 85; 54.9%).The number and quality of systematic reviews and meta-analyses in nuclear medicine have significantly increased over the review period; however, the quality of these articles varies. Efforts to overcome specific weaknesses of the methodologies can provide opportunities for quality improvement.

Evaluation of Italian radiotherapy research from 1985 to 2005: preliminary analysis

AIM

The difficulty in conducting meaningful clinical research is a multifactorial issue, involving political, financial and cultural problems, which can lead to unexpected negative long-term consequences, in terms of knowledge advancement and impact on patient care. The aims of the present review were to evaluate the publications of Italian radiotherapy (RT) groups during a 20-year period and to verify whether research is still appealing to young radiation oncologists (ROs) in Italy.

METHODS

PubMed database was searched for English-language articles published by Italian groups from January 1985 to December 2005. Analyzed variables were: publication/year, kind of study, geographical area and age of the first author.

RESULTS

The systematic review identified 3291 articles: 1207 papers fulfilled the inclusion criteria. The number of Italian published papers increased during the examined period. Retrospective analyses, prospective phase I-II trials and literature reviews were 44, 20 and 14.5% of all published manuscripts, respectively. Randomized trials showed a mild increase from 2000 to 2005, but their absolute number remained low respect to other types of studies (4%). Northern Italy produced the very most of Italian research papers (58.7%). The age of the first/second author was evaluated on 716 papers: In more than 50% of cases, the first author was younger than 40.

CONCLUSION

Despite a general gradual improvement, RT clinical research suffers in Italy (as elsewhere) from insufficient funding, with a negative impact on evidence production. It is worth noting that clinical research is still appealing and accessible to junior Italian RO.

Inadequate diversity of information resources searched in US-affiliated systematic reviews and meta-analyses: 2005-2016

OBJECTIVE

Systematic reviews and meta-analyses (SRMAs) rely upon comprehensive searches into diverse resources that catalog primary studies. However, since what constitutes a comprehensive search is unclear, we examined trends in databases searched from 2005-2016, surrounding the publication of search guidelines in 2013, and associations between resources searched and evidence of publication bias in SRMAs involving human subjects.

STUDY DESIGN

To ensure comparability of included SRMAs over the 12 years in the face of a near 100-fold increase of international SRMAs (mainly genetic studies from China) during this period, we focused on USA-affiliated SRMAs, manually reviewing 100 randomly selected SRMAs from those published in each year. After excluding articles (mainly for inadequate detail or out-of-scope methods), we identified factors associated with the databases searched, used network analysis to see which resources were simultaneously searched, and used logistic regression to link information sources searched with a lower chance of finding publication bias.

RESULTS

Among 817 SRMA articles studied, the common resources used were Medline (95%), EMBASE (44%), and Cochrane (41%). Methods journal SRMAs were most likely to use registries and grey literature resources. We found substantial co-searching of resources with only published materials, and not complemented by searches of registries and the grey literature. The 2013 guideline did not substantially increase searching of registries and grey literature resources to retrieve primary studies for the SRMAs. When used to augment Medline, Scopus (in all SRMAs) and ClinicalTrials.gov (in SRMAs with safety outcomes) were negatively associated with publication bias.

CONCLUSIONS

Even SRMAs that search multiple sources tend to search similar resources. Our study supports searching Scopus and CTG in addition to Medline to reduce the chance of publication bias.

A review of recent publication trends from top publishing countries

BACKGROUND

Evidence-based medicine relies on current best evidence from the medical literature, the patient's history, and the clinician's own experience to provide the best care for patients. Systematic reviews and meta-analysis are considered the highest levels of evidence for informing clinical decisions. Recently, reports have shown an increase in the number but a decrease in quality of meta-analysis publications. We reviewed publication trends and determined the countries with the most journal articles and types of publications in PubMed from 1995 to 2015.

METHODS

We examined journal entries in PubMed from 1995 to 2015 from top publishing countries for total number of publications and citations in core clinical journals and in specific publication types (systematic reviews, meta-analysis, randomized controlled trials).

RESULTS

Yearly, only 30 countries generated 94.6% of all publications and 98.1% of core clinical journals worldwide. All publication types increased but with a significant increase in meta-analysis publications from China. Collaborative and co-authored papers among the 30 countries also showed an increasing trend.

CONCLUSION

The USA leads in all publication citations and specific publication types, except for meta-analysis where China publishes more. Collaborative publishing among international collaborators is also increasing.

Meta-analyses in environmental and occupational health

OBJECTIVES

Meta-analyses are considered generally as the highest level of evidence, but concerns have been voiced about their massive, low-quality production. This paper aimed to evaluate the landscape of meta-analyses in the field of occupational and environmental health and medicine.

METHODS

Using relevant search terms, all meta-analyses were searched for, but those published in 2015 were assessed for their origin, whether they included randomised trials and individual-level data and whether they had authors from the industry or consultancy firms.

RESULTS

PubMed searches (last update February 2017) identified 1251 eligible meta-analyses in this field. There was a rapid increase over time (n=16 published in 1995 vs n=163 published in 2015). Of the 163 eligible meta-analyses published in 2015, 49 were from China, followed at a distance by the USA (n=19). Only 16 considered randomised (intervention) trials and 13 included individual-level data. Only 1 of the 150 meta-analyses had industry authors and none had consultancy firm authors. As an example of conflicting findings, 12 overlapping meta-analyses addressed mobile phones and brain cancer risk and they differed substantially in number of studies included, eligibility criteria and conclusions.

CONCLUSIONS

There has been a major increase in the publication of meta-analyses in occupational and environmental health over time, with the majority of these studies focusing on observational data, while a commendable fraction used individual-level data. Authorship is still limited largely to academic and non-profit authors. With massive production of meta-analyses, redundancy needs to be anticipated and efforts should be made to safeguard quality and protect from bias.

MetaGenyo: a web tool for meta-analysis of genetic association studies

Background

Genetic association studies (GAS) aims to evaluate the association between genetic variants and phenotypes. In the last few years, the number of this type of study has increased exponentially, but the results are not always reproducible due to experimental designs, low sample sizes and other methodological errors. In this field, meta-analysis techniques are becoming very popular tools to combine results across studies to increase statistical power and to resolve discrepancies in genetic association studies. A meta-analysis summarizes research findings, increases statistical power and enables the identification of genuine associations between genotypes and phenotypes. Meta-analysis techniques are increasingly used in GAS, but it is also increasing the amount of published meta-analysis containing different errors. Although there are several software packages that implement meta-analysis, none of them are specifically designed for genetic association studies and in most cases their use requires advanced programming or scripting expertise.

Results

We have developed MetaGenyo, a web tool for meta-analysis in GAS. MetaGenyo implements a complete and comprehensive workflow that can be executed in an easy-to-use environment without programming knowledge. MetaGenyo has been developed to guide users through the main steps of a GAS meta-analysis, covering Hardy-Weinberg test, statistical association for different genetic models, analysis of heterogeneity, testing for publication bias, subgroup analysis and robustness testing of the results.

Conclusions

MetaGenyo is a useful tool to conduct comprehensive genetic association meta-analysis. The application is freely available at http://bioinfo.genyo.es/metagenyo/.

Electronic supplementary material

The online version of this article (10.1186/s12859-017-1990-4) contains supplementary material, which is available to authorized users.

The methodological and reporting quality of systematic reviews from China and the USA are similar

OBJECTIVES

To compare the methodological and reporting quality of systematic reviews by authors from China and those from the United States (USA).

STUDY DESIGN AND SETTING

From systematic reviews of randomized trials published in 2014 in English, we randomly selected 100 from China and 100 from the USA. The methodological quality was assessed using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) tool, and reporting quality assessed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) tool.

RESULTS

Compared with systematic reviews from the USA, those from China were more likely to be a meta-analysis, published in low-impact journals, and a non-Cochrane review. The mean summary Assessing the Methodological Quality of Systematic Reviews score was 6.7 (95% confidence interval: 6.5, 7.0) for reviews from China and 6.6 (6.1, 7.1) for reviews from the USA, and the mean summary Preferred Reporting Items for Systematic Reviews and Meta-analyses score was 21.2 (20.7, 21.6) for reviews from China and 20.6 (19.9, 21.3) for reviews from the USA. The differences in summary quality scores between China and the USA were statistically nonsignificant after adjusting for multiple review factors.

CONCLUSION

The overall methodological and reporting quality of systematic reviews by authors from China are similar to those from the USA, although the quality of systematic reviews from both countries could be further improved.

Trends in published meta-analyses in cancer research, 2008-2013

In order to capture trends in the contribution of epidemiology to cancer research, we describe an online meta-analysis database resource for cancer clinical and population research and illustrate trends and descriptive detail of cancer meta-analyses from 2008 through 2013. A total of 4,686 cancer meta-analyses met our inclusion criteria. During this 6-year period, a fivefold increase was observed in the yearly number of meta-analyses. Fifty-six percent of meta-analyses concerned observational studies, mostly of cancer risk, more than half of which were genetic studies. The major cancer sites were breast, colorectal, and digestive. This online database for Cancer Genomics and Epidemiology Navigator will be continuously updated to allow investigators to quickly navigate the meta-analyses emerging from cancer epidemiology studies and cancer clinical trials.

The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses

POLICY POINTS

Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta-analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta-analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.

CONTEXT

Currently, most systematic reviews and meta-analyses are done retrospectively with fragmented published information. This article aims to explore the growth of published systematic reviews and meta-analyses and to estimate how often they are redundant, misleading, or serving conflicted interests.

METHODS

Data included information from PubMed surveys and from empirical evaluations of meta-analyses.

FINDINGS

Publication of systematic reviews and meta-analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as "systematic reviews" and 58,611 as "meta-analyses." Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta-analyses versus only 153% for all PubMed-indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta-analyses of randomized trials have overlapping, redundant meta-analyses; same-topic meta-analyses may exceed 20 sometimes. Some fields produce massive numbers of meta-analyses; for example, 185 meta-analyses of antidepressants for depression were published between 2007 and 2014. These meta-analyses are often produced either by industry employees or by authors with industry ties and results are aligned with sponsor interests. China has rapidly become the most prolific producer of English-language, PubMed-indexed meta-analyses. The most massive presence of Chinese meta-analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes. Furthermore, many contracting companies working on evidence synthesis receive industry contracts to produce meta-analyses, many of which probably remain unpublished. Many other meta-analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta-analyses are both non-misleading and useful.

CONCLUSIONS

The production of systematic reviews and meta-analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted.

Long noncoding RNAs as novel predictors of survival in human cancer: a systematic review and meta-analysis

Background

Expression of various long noncoding RNAs (lncRNAs) may affect cancer prognosis. Here, we aim to gather and examine all evidence on the potential role of lncRNAs as novel predictors of survival in human cancer.

Methods

We systematically searched through PubMed, to identify all published studies reporting on the association between any individual lncRNA or group of lncRNAs with prognosis in human cancer (death or other clinical outcomes). Where appropriate, we then performed quantitative synthesis of those results using meta-analytic methods to identify the true effect size of lncRNAs on cancer prognosis. The reliability of those results was then examined using measures of heterogeneity and testing for selective reporting biases.

Results

Three hundred ninety-two studies were screened to eventually identify 111 eligible studies on 127 datasets. In total, these represented 16,754 independent participants pertaining to 53 individual and 6 grouped lncRNAs within a total of 19 cancer sites. Overall, 83 % of the studies we identified addressed overall survival and 32 % of the studies addressed recurrence-free survival. For overall survival, 96 % (88/92) of studies identified a statistically significant association of lncRNA expression to prognosis. Meta-analysis of 6 out of 7 lncRNAs for which three or more studies were available, identified statistically significant associations with overall survival. The lncRNA HOTAIR was by far the most broadly studied lncRNA (n = 29; of 111 studies) and featured a summary hazard ratio (HR) of 2.22 (95 % confidence interval (CI), 1.86–2.65) with modest heterogeneity (I² = 49 %; 95 % CI, 14–79 %). Prominent excess significance was demonstrated across all meta-analyses (p-value = 0.0003), raising the possibility of substantial selective reporting biases.

Conclusions

Multiple lncRNAs have been shown to be strongly associated with prognosis in diverse cancers, but substantial bias cannot be excluded in this field and larger studies are needed to understand whether these prognostic information may eventually be useful.

Electronic supplementary material

The online version of this article (doi:10.1186/s12943-016-0535-1) contains supplementary material, which is available to authorized users.

Triangulating meta-analyses: the example of the serotonin transporter gene, stressful life events and major depression

Background

Meta-analysis is intended as a tool for the objective synthesis of evidence across a literature, in order to obtain the best evidence as to whether or not an association or effect is robust. However, as the use of meta-analysis has proliferated it has become increasingly clear that the results of a meta-analysis can be critically sensitive to methodological and analytical choices, so that different meta-analyses on the same topic can arrive at quite different conclusions.

Results

We demonstrate the variability in results of different meta-analyses on the same topic, using the example of the literature on the putative moderating effect of 5-HTTLPR genotype on the association between stressful life events and major depression. We also extend on previous work by including a P-curve analysis of studies from this literature, drawn from a previous meta-analysis, in an attempt to resolve the discrepant conclusions arrived at by previous meta-analyses.

Conclusions

We highlight the divergent conclusions that can be reached when different methodological and analytical choices are taken, and argue that triangulating evidence using multiple evidence synthesis methods is preferable where possible, and that every effort should be made for meta-analyses to be as unbiased as possible (e.g., conducted by methodologists or as part of an adversarial collaboration between authors from opposing camps).

Electronic supplementary material

The online version of this article (doi:10.1186/s40359-016-0129-0) contains supplementary material, which is available to authorized users.

Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study

BACKGROUND

Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs.

METHODS AND FINDINGS

We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics.

CONCLUSIONS

An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.

Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer

BACKGROUND

Multiple treatments are frequently available for a given condition, and clinicians and patients need a comprehensive, up-to-date synthesis of evidence for all competing treatments. We aimed to quantify the waste of research related to the failure of systematic reviews to provide a complete and up-to-date evidence synthesis over time.

METHODS

We performed a series of systematic overviews and networks of randomized trials assessing the gap between evidence covered by systematic reviews and available trials of second-line treatments for advanced non-small cell lung cancer. We searched the Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, MEDLINE, EMBASE, and other resources sequentially by year from 2009 to March 2, 2015. We sequentially compared the amount of evidence missing from systematic reviews to the randomized evidence available for inclusion each year. We constructed cumulative networks of randomized evidence over time and evaluated the proportion of trials, patients, treatments, and treatment comparisons not covered by systematic reviews on December 31 each year from 2009 to 2015.

RESULTS

We identified 77 trials (28,636 patients) assessing 47 treatments with 54 comparisons and 29 systematic reviews (13 published after 2013). From 2009 to 2015, the evidence covered by existing systematic reviews was consistently incomplete: 45 % to 70 % of trials; 30 % to 58 % of patients; 40 % to 66 % of treatments; and 38 % to 71 % of comparisons were missing. In the cumulative networks of randomized evidence, 10 % to 17 % of treatment comparisons were partially covered by systematic reviews and 55 % to 85 % were partially or not covered.

CONCLUSIONS

We illustrate how systematic reviews of a given condition provide a fragmented, out-of-date panorama of the evidence for all treatments. This waste of research might be reduced by the development of live cumulative network meta-analyses.

What is the future of biomedical research?

Randomized controlled trials require hard work and financial commitment, whereas meta-analyses and systematic reviews can be relatively easy to perform and often get published in high impact journals. Many researchers might decide to devote themselves to the latter approach, resulting in a negative impact on clinical research. We have reviewed the number of indexed meta-analyses and systematic reviews on PubMed and compared it with the number of randomized controlled trials over the same period. Statistical analysis showed an exponential increase of synthetic studies with respect to randomized trials. The ratio between RCTs and synthetic studies is quickly decreasing. These results suggest that a growing number of researchers might prefer to commit themselves to synthetic studies more than be involved in more time consuming and funds demanding observational trials. If we are unable to invert this trend, in the future we will have a growing number of synthetic studies utilizing someone else's original data and fewer raw data to base our knowledge upon.

Systematic reviews and meta-analyses: Why are they clinically significant?

This review aims to clarify the clinical significance of systematic reviews and meta-analyses by illustrating several classical examples. Firstly, systematic reviews can provide the highest level of evidence for clinical decisions. Secondly, systematic reviews can propose unresolved issues and future directions. Thirdly, systematic reviews can avoid harm to the human body. Fourthly, systematic reviews can prevent a waste of resources. Generally speaking, clinical researchers should be encouraged to perform systematic reviews and meta-analyses.

The influence of social context on the treatment outcomes of complementary and alternative medicine: the case of acupuncture and herbal medicine in Japan and the U.S

Background

Complementary and alternative medicine (CAM), such as acupuncture and herbal medicine, is popular in many countries. Yet, treatment outcomes of CAM are found to vary significantly between medical trials in different social environments. This paper addresses how the social organization of medicine affects medical treatment outcomes. In particular, it examines the extent to which two popular complementary and alternative medicine (CAM) interventions (acupuncture and herbal medicine) are coordinated with biomedicine and how coordination characteristics are related to the treatment outcomes of the two CAM interventions.

Methods

This paper conducts an archival analysis of the institutional settings of the CAM interventions in Japan and the U.S. It also conducts a systematic content analysis of the treatment outcomes in 246 acupuncture reports and 528 herbal medicine reports that are conducted in Japan or the U.S. and registered in the Cochrane Library’s Central Register of Controlled Trials (CENTRAL), and 716 acupuncture reports and 3,485 herbal medicine reports that are from Japan or the U.S. and listed in MEDLINE. It examines the association between the treatment outcomes of the two interventions and the geographical location of the reports; it also explores how the institutional settings of the interventions are related to the treatment outcomes.

Results

Japanese herbal medicine is integrated into the national medical system the most and American herbal medicine the least; American acupuncture and Japanese acupuncture fall in the middle. Treatment outcomes are the most favorable for Japanese herbal medicine and the least favorable for American herbal medicine. The outcomes of American acupuncture and Japanese acupuncture fall in the middle.

Conclusions

The co-utilization of CAM with biomedicine can produce difficulties due to tensions between CAM and biomedicine. These difficulties and subsequent CAM treatment outcomes vary, depending on how CAM is institutionalized in relation to biomedicine in the national medical system. Coordinated CAM interventions are more likely to be effective and synergic with biomedicine, when compared to uncoordinated ones.