Publication bias in dermatology systematic reviews and meta-analyses

A Cross-Sectional Evaluation of Publication Bias in the Plastic Surgery Literature

BACKGROUND

Publication bias (PB) is the preferential publishing of studies with statistically significant results. PB can skew findings of systematic reviews (SRs) and meta-analyses (MAs), with potential consequences for patient care and health policy. This study aims to determine the extent to which SRs and MAs in the plastic surgery literature evaluate and report PB.

METHODS

This cross-sectional study assessed PB reporting and analysis from plastic surgery studies published between January 1, 2015, and June 19, 2020. Full texts of SRs and MAs were assessed by two reviewers for PB assessment methodology and analysis. Post hoc assessment of studies that did not originally analyze PB was performed using Egger regression, Duval, Tweedie trim-and-fill, and Copas selection models.

RESULTS

There were 549 studies evaluated, of which 531 full texts were included. PB was discussed by 183 studies (34.5%), and formally assessed by 97 studies (18.3%). Among SRs and MAs that formally assessed PB, PB was present in 24 studies (10.7%), not present in 52 (23.1%), and inconclusive in eight (3.6%); 141 studies (62.7%) did not report the results of their PB assessment. Funnel plots were the most common assessment method [ n = 88 (39.1%)], and 60 studies (68.2%) published funnel plots. The post hoc assessment revealed PB in 17 of 20 studies (85.0%).

CONCLUSIONS

PB is inadequately reported and analyzed among studies in the plastic surgery literature. Most studies that assessed PB found PB, as did post hoc analysis of nonreporting studies. Increased assessment and reporting of PB among SRs and MAs would improve the quality of evidence in plastic surgery.

A Review of Meta-Analyses in Plastic Surgery: Need for Adequate Assessment of Publication Bias

INTRODUCTION

Publication bias describes a phenomenon in which significant positive results have a higher likelihood of being published compared to negative or nonsignificant results. Publication bias can confound the estimated therapeutic effect in meta-analyses and needs to be adequately assessed in the surgical literature.

METHODS

A review of meta-analyses published in five plastic surgery journals from 2002 to 2022 was conducted. The inclusion criteria for meta-analyses were factors that demonstrated an obligation to assess publication bias, such as interventions with comparable treatment groups and enough power for statistical analysis. Acknowledgment of publication bias risk, quality of bias assessment, methods used in assessment, and individual article factors were analyzed.

RESULTS

318 unique meta-analyses were identified in literature search, and after full-text reviews, 143 met the inclusion criteria for obligation to assess publication bias. 64% of eligible meta-analyses acknowledged the confounding potential of publication bias, and only 46% conducted a formal assessment. Of those who conducted an assessment, 49% used subjective inspection of funnel plots alone, while 47% used any statistical testing in analysis. Overall, only 9/143 (6.3%) assessed publication bias and attempted to correct for its effect. Journals with a higher average impact factor were associated with mention and assessment of publication bias, but more recent publication year and higher number of primary articles analyzed were not.

CONCLUSIONS

This review identified low rates of proper publication bias assessment in meta-analyses published in five major plastic surgery journals. Assessment of publication bias using objective statistical testing is necessary to ensure quality literature within surgical disciplines.

Quality of systematic reviews on the treatment of vesiculobullous skin diseases. A meta-epidemiological study

BACKGROUND

Systematic reviews of Randomized Controlled Trials (RCTs) are considered high-level evidence to support a decision on therapeutic interventions, and their methodological quality is essential to provide reliable and applicable results.

OBJECTIVE

This meta-epidemiological study aimed to map and critically appraise systematic reviews assessing treatments for vesiculobullous skin diseases.

METHODS

We conducted a comprehensive search strategy on MEDLINE (via Pubmed) in December 2022 without restrictions to find systematic reviews evaluating pharmacological interventions for vesiculobullous skin diseases. The methodological quality was assessed using the AMSTAR-2 tool, and additional information was extracted. We identified nine systematic reviews published between 2002 and 2021, seven assessing pemphigus.

RESULTS

According to the AMSTAR-2 tool, 55.6% were classified as critically low quality, 22.2% as moderate quality, 11.1% as low and 11.1% as high quality. No review assessed the certainty of the evidence (GRADE); 86% of pemphigus reviews had at least two overlapping RCTs. There were some limitations regarding methodological flaws and the AMSTAR-2 tool use CONCLUSIONS: These findings reveal a frail methodological quality of systematic reviews about vesiculobullous diseases treatment that may impact the results. Therefore, methodological rigor is mandatory for future systematic reviews to avoid duplication of effort and increase the certainty of the evidence supporting decision-making.

Publication bias in otorhinolaryngology meta-analyses in 2021

INTRODUCTION

One concern in meta-analyses is the presence of publication bias (PB) which leads to the dissemination of inflated results. In this study, we assessed how much the meta-analyses in the field of otorhinolaryngology in 2021 evaluated the presence of PB.

METHODS

Six of the most influential journals in the field were selected. A search was conducted, and data were extracted from the included studies. In cases where PB was not assessed by the authors, we evaluated the risk of its presence by designing funnel plots and performing statistical tests.

RESULTS

Seventy-five systematic reviews were included. Fifty-one percent of them used at least one method for assessing the risk of PB, with the visual inspection of a funnel plot being the most frequent method used. Twenty-nine percent of the studies reported a high risk of PB presence. We replicated the results of 11 meta-analyses that did not assess the risk of PB and found that 63.6% were at high risk. We also found that a considerable proportion of the systematic reviews that found a high risk of PB did not take it into consideration when making conclusions and discussing their results.

DISCUSSION

Our results indicate that systematic reviews published in some of the most influential journals in the field do not implement enough measures in their search strategies to reduce the risk of PB, nor do they assess the risk of its presence or take the risk of its presence into consideration when inferring their results.

Publication bias in meta-analyses of the therapeutic efficacy of remdesivir interventions for patients with COVID-19

Purpose

No comprehensive statistical assessment of publication bias has been conducted in remdesivir-based intervention research for COVID-19 patients. This study aims to examine all meta-analyses of the efficacy of remdesivir interventions in COVID-19 patients and perform a statistical assessment of publication bias.

Design/methodology/approach

This is an analytic study conducted to assess the impact of publication bias on the results of meta-analyses of remdesivir-based interventions in patients infected with COVID-19. All English full-text meta-analyses published in peer-reviewed journals in 2019–2021 were included. A computerized search of PubMed and Web of Science electronic databases was performed on December 24, 2021. The trim-and-fill method calculated the number of missing studies and the adjusted cumulative effect sizes.

Findings

The final analysis comprised 21 studies with 88 outcomes. The investigation revealed missing studies in 46 outcomes (52%). Seventy-six missing studies were replaced in the outcomes using the trim-and-fill procedure. The adjusted recalculated effect sizes of the 27 outcomes increased by an average of 0.04. In comparison, the adjusted effect size of 18 outcomes fell by an average of 0.036. Only 14 out of 46 outcomes with publication bias were subjected to a gray literature search (30%). To discover related research, no gray literature search was conducted in most outcomes with publication bias (n = 32; 70%). In conclusion, the reported effect estimates regarding the effect of remdesivir in COVID-19 patients are only slightly affected by publication bias and can be considered authentic. Health-care decision-makers in COVID-19 should consider current research results when making clinical decisions.

Research limitations/implications

Most health decisions are based on the effect sizes revealed in meta-analyses. When deciding on remdesivir-based treatment for COVID-19 patients, therefore, the outcomes of this investigation may be of paramount importance to health policymakers, leading to better treatment strategies.

Practical implications

According to the results, no major publication bias and missing studies were detected on average. Therefore, the calculated effect sizes of remdesivir-based interventions on meta-analyses can be used as authentic and unbiased benchmarks by health-care decision-makers in treating patients with COVID-19.

Originality/value

This is the first study to examine the effect of publication bias and gray literature searches on the results of meta-analyses of treatment with COVID-19 (remdesivir).

Impact of searching clinical trials registers in systematic reviews of pharmaceutical and non-pharmaceutical interventions: Reanalysis of meta-analyses

Systematic reviewers are advised to search trials registers to minimise risk of reporting biases. However, there has been little research on the impact of searching trials registers on the results of meta-analyses. We aimed to evaluate the impact of searching clinical trials registers for systematic reviews of pharmaceutical or non-pharmaceutical interventions. We searched PubMed, Scopus, Science Citation Index and Social Sciences Citation Index, and Education Collection for systematic reviews with meta-analyses indexed from 2 November to 2 December 2020. A random sample of systematic reviews was initially drawn, and for reviews which considered randomised trials eligible for inclusion, which had not searched a trials register, we searched ClinicalTrials.gov, EudraCT, ANZCTR, and the WHO ICTRP search portal for eligible trials. We compared meta-analytic effect estimates before and after including results from additional trials identified. We found additional trials for 63% (63/101) of eligible reviews; however, trials with results that could contribute to a meta-analysis were identified for only 20% (20/101) of the reviews. On average, there was no difference in the meta-analytic effect estimates before versus after adding the new trials. In summary, searching clinical trial registers led to identification of additional trials for many reviews; however, very few trials had results available for inclusion in meta-analyses. Including results from the new trials led to no change in the meta-analytic estimates, on average. Trials registers would be even more valuable to systematic reviewers if more trialists made use of them (i.e., registered their trials and posted results in a timely manner).

Publication bias in pharmacogenetics of adverse reaction to antiseizure drugs: An umbrella review and a meta-epidemiological study

Publication bias may lead to a misestimation in the association between pharmacogenetic biomarkers (PGx) and antiseizure drug's adverse effects (AEs). We aimed to assess its prevalence in this field. We searched for systematic reviews assessing PGx of antiseizure drug's AEs. For each unique association between a PGx, a drug and its AE, we used the available odds ratio (ORs) to generate corresponding funnel plots. We estimated the prevalence of publication bias using visual inspections and asymmetry tests. We explored the impact of publication bias using ORs adjusted for potential publication bias. Twenty-two associations were available. Our visual analysis suggested a publication bias in five out twenty-two funnel plots (23% [95%CI: 8; 45]). The Egger's test showed a significant publication bias in one (HLA-B*15:02 and phenytoin-induced Stevens-Johnson syndrome or toxic epidermal necrolysis, p = 0.03) out of nine (11% [95%CI: 0; 48]) and the Begg's test in one (HLA-B*15:02 and carbamazepine-induced serious cutaneous reactions, p = 0.02) out of ten (10% [95%CI: 0; 45]) assessable funnel plots. Adjusting for publication bias may reduce by half the ORs of the pharmacogenetics associations. Publication bias in the pharmacogenetic of antiseizure drug's AEs is not uncommon and may affect the estimation of the effect of such biomarkers. When conducting pharmacogenetic studies, it is critical to publish also the negative one.

Evaluating publication bias for clinical trials supporting new dermatologic drug approvals from 2003 to 2018

The degree of publication bias and impact of the Food and Drug Administration Amendments Act (FDAAA) of 2007, which aimed to improve clinical trial transparency, has yet to be examined for recent dermatologic drugs. The objective of our study was to estimate the degree of publication bias for clinical trials supporting FDA approval of new dermatologic drugs. This retrospective cohort study examined all phase II and III efficacy trials supporting approval of new dermatologic drugs from 2003 to 2018. FDA drug approval documents were reviewed for supportive clinical trial information, and publications were matched using PubMed and Google Scholar searches. Ratios of relative risks (RRR) comparing positive versus non-positive trials before and after FDAAA enactment served to estimate publication bias. We found that the likelihood of publishing positive versus non-positive drug trials in dermatology was unchanged before and after FDAAA enactment (RRR 0.87, 95% CI 0.37-2.08), as was the likelihood of publishing without misleading interpretation (RRR 1.51, 95% CI 0.22-10.50). There was no measurable publication bias for efficacy trials supporting new drug approvals in dermatology over the past 15 years. Fewer pre-FDAAA trials (n = 21) compared to post-FDAAA trials (n = 106) met inclusion criteria. Though not analyzed in this study, safety and secondary efficacy results are other potential sources for publication bias.

Association between CDH1 methylation and esophageal cancer risk: a meta-analysis and bioinformatics study

OBJECTIVE

The aim is to evaluate the association of CDH1 methylation with esophageal cancer (EC) risk.

METHODS

The PubMed, Embase, Web of Science, and Cochrane Library databases were systematically searched to identify relevant articles. Pooled odds ratios (ORs) with 95% confidence interval (CI) were estimated using the fixed- or random-effects models. The pooled sensitivity and specificity were calculated to assess the diagnostic value of CDH1 methylation for EC. The results of the meta-analysis were validated using The Cancer Genome Atlas and Gene Expression Omnibus databases.

RESULTS

Thirteen studies consisting of 1,633 samples were included. A high CDH1 methylation was significantly associated with an increased risk of EC (OR = 10.40, 95% CI = 6.29-17.18). Furthermore, CDH1 methylation status was related to tumor status, lymph node status, and metastasis. For the diagnosis of EC, the pooled sensitivity and specificity of CDH1 methylation were 0.57 (95% CI = 0.39-0.74) and 0.89 (95% CI = 0.81-0.94), respectively. Bioinformatics analysis showed that CDH1 methylation occurred more frequently in EC tissues than in normal controls, in good agreement with the results of the meta-analysis.

CONCLUSION

A significant association was found between CDH1 methylation and EC risk. We therefore suggest that CDH1 methylation can serve as a promising diagnostic marker for EC.

Evaluation of comprehensive benefits of sponge cities using meta-analysis in different geographical environments in China

With the rapid progress in urbanization, frequent urban waterlogging and non-point source pollution are threatening the living and health of human beings. Sponge city construction has become an effective means to curb urban waterlogging. Although related studies have explored the comprehensive benefits of sponge cities, few studies have been conducted on the effects of different geographical environments on runoff control and suspended solid (SS) removal. Based on 76 cities with sponge cities in China, this study used the meta-analysis method to evaluate the relationships of climate, terrain, underlying surface conditions, and construction area with the increase in the total annual runoff control rate and SS removal rate. The results reveal that the runoff control benefit can be significantly improved by sponge cities under the combined conditions of average annual precipitation of approximately 1000 mm, high fractional vegetation cover, sufficient soil fertility, a terrain slope i of ≤2%, and a permeability coefficient of strata of 100-200 m/d, especially in northern China, where the weight representing the quantity of comprehensive benefits was calculated to be 25.5%. In addition, the study results assist in reforming unfavorable geographical environments in the construction of sponge city, thus providing more effective solutions for tackling SS pollution. The most significant benefits of SS removal were obtained in north central China, where the weight was 21.4%. This study comprehensively investigated the effects of geographical environmental factors on the comprehensive benefits of sponge city reflected by the improvement in the total annual runoff control rate and the SS removal rate. The results will provide guidance for the planning and design of global sponge cities and effectively optimize the practice, scale, and location of existing construction based on specific geographical environments.

Assessing the compliance of systematic review articles published in leading dermatology journals with the PRISMA statement guidelines: A systematic review

Background

Reporting quality of systematic reviews and meta-analyses is of critical importance in dermatology because of their key role in informing health care decisions.

Objective

To assess the compliance of systematic reviews and meta-analyses in leading dermatology journals with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement guidelines.

Methods

This review was carried out in accordance with PRISMA guidelines. Included studies were reviews published across 6 years in the top 4 highest-impact-factor dermatology journals of 2017. Records and full texts were screened independently. Data analysis was conducted with univariate multivariable linear regression. The primary outcome was to assess the compliance of systematic reviews and meta-analyses in leading dermatology journals with the PRISMA statement.

Results

A total of 166 studies were included and mean PRISMA compliance across all articles was 73%. Compliance significantly improved over time (β = .016; P = <.001). The worst reported checklist item was item 5 (reporting on protocol existence), with a compliance of 15% of articles.

Conclusion

PRISMA compliance within leading dermatology journals could be improved; however, it is steadily improving.

Quality Assessment of Published Systematic Reviews in High Impact Cardiology Journals: Revisiting the Evidence Pyramid

Objective: Systematic reviews are increasingly used as sources of evidence in clinical cardiology guidelines. In the present study, we aimed to assess the quality of published systematic reviews in high impact cardiology journals.

Methods: We searched PubMed for systematic reviews published between 2010 and 2019 in five general cardiology journals with the highest impact factor (according to Clarivate Analytics 2019). We extracted data on eligibility criteria, methodological characteristics, bias assessments, and sources of funding. Further, we assessed the quality of retrieved reviews using the AMSTAR tool.

Results: A total of 352 systematic reviews were assessed. The AMSTAR quality score was low or critically low in 71% (95% CI: 65.7–75.4) of the assessed reviews. Sixty-four reviews (18.2%, 95% CI: 14.5–22.6) registered/published their protocol. Only 221 reviews (62.8%, 95% CI: 57.6–67.7) reported adherence to the EQUATOR checklists, 208 reviews (58.4%, 95% CI: 53.9–64.1) assessed the risk of bias in the included studies, and 177 reviews (52.3%, 95% CI: 45.1–55.5) assessed the risk of publication bias in their primary outcome analysis. The primary outcome was statistically significant in 274 (79.6%, 95% CI: 75.1–83.6) and had statistical heterogeneity in 167 (48.5%, 95% CI: 43.3–53.8) reviews. The use and sources of external funding was not disclosed in 87 reviews (24.7%, 95% CI: 20.5–29.5). Data analysis showed that the existence of publication bias was significantly associated with statistical heterogeneity of the primary outcome and that complex design, larger sample size, and higher AMSTAR quality score were associated with higher citation metrics.

Conclusion: Our analysis uncovered widespread gaps in conducting and reporting systematic reviews in cardiology. These findings highlight the importance of rigorous editorial and peer review policies in systematic review publishing, as well as education of the investigators and clinicians on the synthesis and interpretation of evidence.

Efficacy and safety of adalimumab in hidradenitis suppurativa: A systematic review and meta-analysis of randomized controlled trials

BACKGROUND

Adalimumab is used as a first-line biologic agent in the management of moderate-to-severe hidradenitis suppurativa (HS). The objective of the present study was to evaluate the efficacy and safety of adalimumab in patients with moderate-to-severe HS.

METHODS

We performed a systematic review and meta-analysis according to Preferred Reporting Items for Systematic reviews and Meta-Analysis guidelines. Pooled estimates, namely standardized mean difference (SMD) and relative risk (RR), were calculated using random-effect model with trial sequential analysis. Small study effects were examined using the Doi plot. Certainty of evidence (CoE) was assessed using "The Grading of Recommendations Assessment, Development, and Evaluation" approach, and number-needed-to-treat (NNT) was calculated.

RESULTS

Five randomized controlled trials, involving 1014 patients, were included. We performed subgroup analysis of adalimumab administered subcutaneously both weekly and every other week. Adalimumab administered weekly was associated with better clinical response achievement (RR 1.76, 95% confidence interval [95% CI] 1.35-2.29; trial sequential analysis TSA-adjusted CI 1.01-3.08; CoE: low; NNT = 5) and a significant improvement in modified Sartorius score (SMD = -0.45, 95% CI = -0.76 to -0.13; CoE: very low; NNT = 10) and dermatology life quality index (DLQI) (SMD -0.47, 95% CI -0.61 to -0.32; CoE: low; NNT = 10). Nevertheless, adalimumab administered every other week showed an improvement only in modified Sartorius score. The pooled RRs of adverse events in both groups revealed no statistical significance when compared with the placebo.

CONCLUSIONS

Adalimumab administered weekly resulted in not only better clinical responses than placebo but also significantly improved disease severity and quality of life of patients with moderate-to-severe HS. Our study provides supporting evidence to the current guidelines and aids decision-making in the application of adalimumab in HS management.

Bone Mineral Density, Osteoporosis, and Fracture Risk in Adult Patients with Psoriasis or Psoriatic Arthritis: A Systematic Review and Meta-Analysis of Observational Studies

INTRODUCTION

Awareness of psoriasis-related comorbidities has been established in the current guidelines; however, evidence regarding the association of bone density or bone fragility with psoriatic disease remains inconclusive.

METHODS

We conducted a systematic review and meta-analysis to assess bone mineral density and the risk of osteoporosis and fractures in patients with psoriatic disease, including those with cutaneous psoriasis and psoriatic arthritis. We searched electronic databases for published observational studies. A meta-analysis was performed using the random-effect model. Pooled estimates and their confidence intervals (CIs) were calculated. Small-study effects were examined using the Doi plot and Luis Furuya-Kanamori index.

RESULTS

The analysis of the standardized mean difference in the absolute value of bone mineral density at different measuring sites (lumbar spine, femoral neck, and total hip) revealed no significant difference between patients with psoriatic disease and non-psoriatic controls. The pooled results of the adjusted odds ratios (ORs) demonstrated no increased risk of osteoporosis in patients with psoriatic disease. Notably, patients with psoriatic disease had a higher OR of developing bone fractures (adjusted OR: 1.09; 95% CI: 1.06 to 1.12; I²: 0%).

CONCLUSION

Patients with psoriatic disease may be more likely to develop fractures compared with non-psoriatic controls. This higher risk for fracture may not necessarily be associated with lower bone mineral density nor a higher risk for osteoporosis.

Generic outcome set for the international registry on Laser trEAtments in Dermatology (LEAD): a protocol for a Delphi study to achieve consensus on what to measure

INTRODUCTION

While laser technology has expanded the armamentarium of treatment for various skin diseases during the past years, heterogeneity in study outcomes hampers comparability and appropriate evidence synthesis. Part of these issues can be addressed by developing a generic outcome set. Using the Delphi method, this study aims to seek consensus between key stakeholders on relevant generic outcomes (what to measure) for implementation in the international registry on Laser trEAtments in Dermatology (LEAD). The registry is focused on collecting research data on various laser treatments for skin disorders.

METHODS AND ANALYSIS

By reviewing the literature and involvement of key stakeholder groups and adult patients in need or after laser surgery and health professionals, a preliminary list of outcomes will be generated and categorised into domains. Using these outcomes, an international three-round Delphi study will be performed to rate the importance of outcomes in the selection of a generic outcome set. Participants are allowed to provide new outcomes to the preliminary list for revisions during the first Delphi round. Finally, results will be discussed during a consensus meeting to agree on generic outcomes to be used in the LEAD registry.

ETHICS AND DISSEMINATION

An ethics approval was not applicable (W19_290 # 18.336). The study is registered with the Cochrane Skin Core OUtcome Set INitiative) and the Core Outcome Measures in Effectiveness Trials initiative. Procedures will be conducted according to the Declaration of Helsinki. The findings will be disseminated through peer-reviewed publications and conference presentations.

An Evaluation of Publication Bias in High-Impact Orthopaedic Literature

Background:

Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses (“false positives”) and (2) studies without significant results may not be included in systematic reviews and meta-analyses (“false negatives”) because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted.

Methods:

We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves.

Results:

Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively.

Conclusions:

Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias.

Clinical Relevance:

By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.

Assessing the compliance of systematic review articles published in leading dermatology journals with the PRISMA statement guidelines: A systematic review protocol

Introduction

Systematic reviews and meta-analyses are methodologically rigorous studies that are said to form the reference standard for summarising evidence to guide health care. Reporting quality of reviews is of critical importance in order to judge the quality and risk of bias in a review to ensure sound healthcare decisions are made. This is particularly important in the field of dermatology due to the growing number of systematic reviews and their key role in informing healthcare decision within dermatology. A contemporary and comprehensive review of the compliance of dermatology systematic reviews and meta-analyses with the PRISMA checklist, in the highest impact factor dermatology journals, has not yet been assessed. To our knowledge, our review represents the most extensive study assessing reporting quality of systematic reviews and meta-analyses published within dermatology to date.

Methods and analysis

Our protocol is reported in line with the Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 guidelines. MEDLINE will be searched to look for systematic reviews and meta-analysis in selected years within the top four highest impact factor dermatology journals in 2017. Records and full texts will be screened independently by five researchers. Data will be extracted onto a standard data extraction database. A training session will take place to ensure accurate data extraction and scoring of studies with the PRISMA checklist. The data will be analysed and outcomes will be determined. Primary outcome will be the compliance of reviews with the PRISMA checklist.

Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review

BACKGROUND

Several scales, checklists and domain-based tools for assessing risk of reporting biases exist, but it is unclear how much they vary in content and guidance. We conducted a systematic review of the content and measurement properties of such tools.

METHODS

We searched for potentially relevant articles in Ovid MEDLINE, Ovid Embase, Ovid PsycINFO and Google Scholar from inception to February 2017. One author screened all titles, abstracts and full text articles, and collected data on tool characteristics.

RESULTS

We identified 18 tools that include an assessment of the risk of reporting bias. Tools varied in regard to the type of reporting bias assessed (eg, bias due to selective publication, bias due to selective non-reporting), and the level of assessment (eg, for the study as a whole, a particular result within a study or a particular synthesis of studies). Various criteria are used across tools to designate a synthesis as being at 'high' risk of bias due to selective publication (eg, evidence of funnel plot asymmetry, use of non-comprehensive searches). However, the relative weight assigned to each criterion in the overall judgement is unclear for most of these tools. Tools for assessing risk of bias due to selective non-reporting guide users to assess a study, or an outcome within a study, as 'high' risk of bias if no results are reported for an outcome. However, assessing the corresponding risk of bias in a synthesis that is missing the non-reported outcomes is outside the scope of most of these tools. Inter-rater agreement estimates were available for five tools.

CONCLUSION

There are several limitations of existing tools for assessing risk of reporting biases, in terms of their scope, guidance for reaching risk of bias judgements and measurement properties. Development and evaluation of a new, comprehensive tool could help overcome present limitations.

Flaws in the application and interpretation of statistical analyses in systematic reviews of therapeutic interventions were common: a cross-sectional analysis

OBJECTIVES

The objective of the study was to investigate the application and interpretation of statistical analyses in a cross-section of systematic reviews (SRs) of therapeutic interventions, without restriction by journal, clinical condition, or specialty.

STUDY DESIGN AND SETTING

We evaluated a random sample of SRs assembled previously, which were indexed in MEDLINE® during February 2014, focused on a treatment or prevention question, and reported at least one meta-analysis. The reported statistical methods used in each SR were extracted from articles and online appendices by one author, with a 20% random sample extracted in duplicate.

RESULTS

We evaluated 110 SRs; 78/110 (71%) were non-Cochrane SRs and 55/110 (50%) investigated a pharmacological intervention. The SRs presented a median of 13 (interquartile range: 5-27) meta-analytic effects. When considering the index (primary or first reported) meta-analysis of each SR, just over half (62/110 [56%]) used the random-effects model, but few (5/62 [8%]) interpreted the meta-analytic effect correctly (as the average of the intervention effects across all studies). A statistical test for funnel plot asymmetry was reported in 17/110 (15%) SRs; however, in only 4/17 (24%) did the test include the recommended number of at least 10 studies of varying size. Subgroup analyses accompanied 42/110 (38%) index meta-analyses, but findings were not interpreted with respect to a test for interaction in 29/42 (69%) cases, and the issue of potential confounding in the subgroup analyses was not raised in any SR.

CONCLUSIONS

There is scope for improvement in the application and interpretation of statistical analyses in SRs of therapeutic interventions. The involvement of statisticians on the SR team and establishment of partnerships between researchers with specialist expertise in SR methods and journal editors may help overcome these shortcomings.

A review of publication bias in the gastroenterology literature

In systematic reviews and meta-analyses, publication bias is particularly problematic, given that combining only statistically significant outcomes is likely to overestimate the true effect of an intervention since non-significant findings have been omitted. We examined practices for evaluating publication bias from gastroenterology literature. We performed a PubMed search to identify systematic reviews published in American Journal of Gastroenterology, Gut, and Gastroenterology from 2005 to 2015. Of the 304 found, 215 studies were eligible for inclusion based on relevant study characteristics. There were 190 systematic reviews which used at least one method to evaluate publication bias and/or included ten or more primary studies. There were 115/190 (60.53%) systematic reviews which used at least one method to evaluate publication bias. Most (105/115, 91.27%) qualified reviews used at least one method to evaluate publication bias and 78/115 (67.83%) used a combination of methods. The most common methods were funnel plot (100/115, 86.96%), Egger's regression (67/115, 58.26%), and Begg's (28/115, 24.35%). Of the 115 reviews that performed evaluations, 26 (22.61%) conducted these analyses with fewer than ten primary studies, and a minority (24/115, 20.87%) reached the conclusion that publication bias was present in their work. While methods to assess publication bias were frequently noted among qualified systematic reviews, these methods are limited in value and could be improved by incorporating approaches that assess the degree of publication bias severity.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool

OBJECTIVES

No gold standard exists to assess methodological quality of systematic reviews (SRs). Although Assessing the Methodological Quality of Systematic Reviews (AMSTAR) is widely accepted for analyzing quality, the ROBIS instrument has recently been developed. This study aimed to compare the capacity of both instruments to capture the quality of SRs concerning psoriasis interventions.

STUDY DESIGN AND SETTING

Systematic literature searches were undertaken on relevant databases. For each review, methodological quality and bias risk were evaluated using the AMSTAR and ROBIS tools. Descriptive and principal component analyses were conducted to describe similarities and discrepancies between both assessment tools.

RESULTS

We classified 139 intervention SRs as displaying high/moderate/low methodological quality and as high/low risk of bias. A high risk of bias was detected for most SRs classified as displaying high or moderate methodological quality by AMSTAR. When comparing ROBIS result profiles, responses to domain 4 signaling questions showed the greatest differences between bias risk assessments, whereas domain 2 items showed the least.

CONCLUSION

When considering SRs published about psoriasis, methodological quality remains suboptimal, and the risk of bias is elevated, even for SRs exhibiting high methodological quality. Furthermore, the AMSTAR and ROBIS tools may be considered as complementary when conducting quality assessment of SRs.

Methodological quality of systematic reviews referenced in clinical practice guidelines for the treatment of opioid use disorder

INTRODUCTION

With efforts to combat opioid use disorder, there is an increased interest in clinical practice guidelines (CPGs) for opioid use disorder treatments. No literature exists examining the quality of systematic reviews used in opioid use disorder CPGs. This study aims to describe the methodological quality and reporting clarity of systematic reviews (SRs) used to create CPGs for opioid use disorder.

METHODS

From June to July 2016 guideline clearinghouses and medical literature databases were searched for relevant CPGs used in the treatment of opioid use disorder. Included CPGs must have been recognized by a national organization. SRs from the reference section of each CPG was scored by using AMSTAR (a measurement tool to assess the methodological quality of systematic reviews) tool and PRISMA (preferred reporting items for systematic reviews and meta-analyses) checklist.

RESULTS

Seventeen CPGs from 2006-2016 were included in the review. From these, 57 unique SRs were extracted. SRS comprised 0.28% to 17.92% of all references found in the CPGs. All SRs obtained moderate or high methodological quality score on the AMSTAR tool. All reviews met at least 70% of PRISMA criteria. In PRISMA, underperforming areas included accurate title labeling, protocol registration, and risk of bias. Underperforming areas in AMSTAR included conflicts of interest, funding, and publication bias. A positive correlation was found between AMSTAR and PRISMA scores (r = .79).

CONCLUSION

Although the SRs in the CPGs were of good quality, there are still areas for improvement. Systematic reviewers should consult PRISMA and AMSTAR when conducting and reporting reviews. It is important for CPG developers to consider methodological quality as a factor when developing CPG recommendations, recognizing that the quality of systematic reviews underpinning guidelines does not necessarily correspond to the quality of the guideline itself.

Selective outcome reporting in obesity clinical trials: a cross-sectional review

Selective outcome reporting is a form of bias resulting from discrepancies between outcomes presented in a trial's registration and the published report. We investigate this selective bias in obesity clinical trials. A PubMed search was conducted to identify randomized controlled trials (RCTs) published in four obesity journals from 2013 to 2015. Primary, secondary and tertiary outcomes were recorded for each trial and compared to pre-specified outcomes in each trial's registration. Of the 392 identified articles, 142 were included in the final analysis; 22 (15%) RCTs demonstrated major outcome discrepancies between registration and publication: No primary outcomes were demoted to a secondary or tertiary outcome; 14 (36.84%) primary outcomes were omitted; 14 (36.84%) primary outcomes were added: 5 (13.16%) secondary outcomes were upgraded to primary outcomes; and timing of assessment for a primary outcome changed 5 (13.16%) times. Out of the 63 prospectively registered studies, 53 had no discrepancies. A total of 76 of the studies (29.80%) were unregistered or did not have an associated registration number. Our results suggest that selective outcome reporting may be a concern in obesity clinical trials. As selective outcome reporting may distort clinical findings and limit outcomes in systematic reviews, we encourage trialists and journal editors to work towards solutions to mitigate this issue.

The Indirect Efficacy Comparison of DNA Methylation in Sputum for Early Screening and Auxiliary Detection of Lung Cancer: A Meta-Analysis

Background: DNA methylation in sputum has been an attractive candidate biomarker for the non-invasive screening and detection of lung cancer. Materials and Methods: Databases including PubMed, Ovid, Cochrane library, Web of Science databases, Chinese Biological Medicine (CBM), Chinese National Knowledge Infrastructure (CNKI), Wanfang, Vip Databases and Google Scholar were searched to collect the diagnostic trials on aberrant DNA methylation in the screening and detection of lung cancer published until 1 December 2016. Indirect comparison meta-analysis was used to evaluate the diagnostic value of the included candidate genes. Results: The systematic literature search yielded a total of 33 studies including a total of 4801 subjects (2238 patients with lung cancer and 2563 controls) and covering 32 genes. We identified that methylated genes in sputum samples for the early screening and auxiliary detection of lung cancer yielded an overall sensitivity of 0.46 (0.41–0.50) and specificity of 0.83 (0.80–0.86). Combined indirect comparisons identified the superior gene of SOX17 (sensitivity: 0.84, specificity: 0.88), CDO1 (sensitivity: 0.78, specificity: 0.67), ZFP42 (sensitivity: 0.87, specificity: 0.63) and TAC1 (sensitivity: 0.86, specificity: 0.75). Conclusions: The present meta-analysis demonstrates that methylated SOX17, CDO1, ZFP42, TAC1, FAM19A4, FHIT, MGMT, p16, and RASSF1A are potential superior biomarkers for the screening and auxiliary detection of lung cancer.

Methods for Developing Evidence Reviews in Short Periods of Time: A Scoping Review

INTRODUCTION

Rapid reviews (RR), using abbreviated systematic review (SR) methods, are becoming more popular among decision-makers. This World Health Organization commissioned study sought to summarize RR methods, identify differences, and highlight potential biases between RR and SR.

METHODS

Review of RR methods (Key Question 1 [KQ1]), meta-epidemiologic studies comparing reliability/ validity of RR and SR methods (KQ2), and their potential associated biases (KQ3). We searched Medline, EMBASE, Cochrane Library, grey literature, and checked reference lists, used personal contacts, and crowdsourcing (e.g. email listservs). Selection and data extraction was conducted by one reviewer (KQ1) or two reviewers independently (KQ2-3).

RESULTS

Across all KQs, we identified 42,743 citations through the literature searches. KQ1: RR methods from 29 organizations were reviewed. There was no consensus on which aspects of the SR process to abbreviate. KQ2: Studies comparing the conclusions of RR and SR (n = 9) found them to be generally similar. Where major differences were identified, it was attributed to the inclusion of evidence from different sources (e.g. searching different databases or including different study designs). KQ3: Potential biases introduced into the review process were well-identified although not necessarily supported by empirical evidence, and focused mainly on selective outcome reporting and publication biases.

CONCLUSION

RR approaches are context and organization specific. Existing comparative evidence has found similar conclusions derived from RR and SR, but there is a lack of evidence comparing the potential of bias in both evidence synthesis approaches. Further research and decision aids are needed to help decision makers and reviewers balance the benefits of providing timely evidence with the potential for biased findings.

The Correlation of MGMT Promoter Methylation and Clinicopathological Features in Gastric Cancer: A Systematic Review and Meta-Analysis

The silencing of the tumor suppressor gene O-6-methylguanine-DNA methyltransferase (MGMT) by promoter methylation commonly occurs in human cancers. The relationship between MGMT promoter methylation and gastric cancer (GC) remains inconsistent. This study aimed to evaluate the potential value of MGMT promoter methylation in GC patients. Electronic databases were searched to identify eligible studies. The pooled odds ratio (OR) and the corresponding 95% confidence interval (95% CI) were used to evaluate the effects of MGMT methylation on GC risk and clinicopathological characteristics. In total, 31 eligible studies including 2988 GC patients and 2189 nonmalignant controls were involved in meta-analysis. In the pooled analysis, MGMT promoter methylation was significantly associated with GC risk (OR = 3.34, P < 0.001) and substantial heterogeneity (P < 0.001). Meta-regression and subgroup analyses based on the testing method, sample material and ethnicity failed to explain the sources of heterogeneity. Interestingly, MGMT methylation showed a trend associated with gender, and methylation is lower in males compared with females (OR = 0.76, 95% CI = 0.56–1.03). We did not find a significant association in relation to tumor types, clinical stage, age status or H. pylori status in cancer (all P > 0.1). MGMT promoter methylation may be correlated with the prognosis of GCs in disease free survival (DFS) or overall survival (OS) for univariate analysis. MGMT promoter methylation may play a crucial role in the carcinogenesis and prognosis of GC. MGMT methylation was not correlated with tumor types, clinical stage, age status, H. pylori status. However, the result of the association of MGMT methylation and gender should be considered with caution.

The detection accuracy of cone beam CT for osseous defects of the temporomandibular joint: a systematic review and meta-analysis

The purpose of this review was to evaluate whether cone-beam computed tomography (CBCT) is reliable for the detection of bone changes of the temporomandibular joint (TMJ). Studies collected from the PubMed, Web of Science, Cochrane Library, ScienceDirect, Embase, Wanfang and CNKI databases were searched, and the publishing time was limited from January 1990 to December 2015. Eight studies (23 experimental research groups) were eventually included for further analysis. The pooled sensitivity was 0.67 and the pooled specificity was 0.87, which leads to a relatively large area (0.84) under the Receiver Operating Characteristic (ROC) curve. The related pooled positive likelihood ratio (+LR) and the pooled negative likelihood ratio (−LR) were 5.2 and 0.38, respectively. The subgroup analysis was conducted for four subgroups categorized by voxel size (≤0.2; >0.2, ≤0.3; >0.3, ≤0.4; >0.4, and ≤0.5 (mm)), and the “>0.4, ≤0.5” subgroup had a higher pooled sensitivity and pooled specificity than the other groups. The present study demonstrates that CBCT has a relatively high diagnostic accuracy for TMJ bone changes, although its reliability is limited. Voxel size did not play a role in the accuracy of CBCT.