Quality of reporting in infertility journals

Challenges in Interpreting Obstetrics and Gynecology Literature

The field of obstetrics and gynecology is constantly replenished with the newest research findings. In an era of rapidly available study publications, there are a number of challenges to interpreting the obstetrics and gynecology literature. Common pitfalls include the over reliance on the dichotomized P-value, lack of transparency, bias in study reporting, limitations of resources, absence of standardized practices and outcomes in study design, and the rare concerns for data integrity. We review these predominant challenges and their potential solutions, in interpreting the obstetrics and gynecology literature.

Systematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017

To gain insight into changes of scholarly journals' recommendations, we conducted a systematic review of studies that analysed journals' Instructions to Authors (ItAs). We summarised results of 153 studies, and meta-analysed how often ItAs addressed: 1) authorship, 2) conflicts of interest, 3) data sharing, 4) ethics approval, 5) funding disclosure, and 6) International Committee of Medical Journal Editors' Uniform Requirements for Manuscripts. For each topic we found large between-study heterogeneity. Here, we show six factors that explained most of that heterogeneity: 1) time (addressing of topics generally increased over time), 2) country (large differences found between countries), 3) database indexation (large differences found between databases), 4) impact factor (topics were more often addressed in highest than in lowest impact factor journals), 5) discipline (topics were more often addressed in Health Sciences than in other disciplines), and 6) sub-discipline (topics were more often addressed in general than in sub-disciplinary journals).

Rigor, reproducibility, and transparency of randomized controlled trials in obstetrics and gynecology

BACKGROUND

Randomized controlled trials are considered the highest level of evidence but fewer than half are reproducible. A rigorous methodology improves trial quality, but reproducibility may be limited by a lack of transparency in reporting. The Consolidated Standards of Reporting Trials guidelines define reporting standards, and pretrial registration requires a predefined methodology and predefined outcomes.

OBJECTIVE

We evaluated obstetrics and gynecology trials published in 6 journals in terms of their adherence to the Consolidated Standards of Reporting Trials guidelines. Second, we evaluated pretrial registration compliance and concordance between the registry and publication. Furthermore, we evaluated the differences in trial characteristics among randomized controlled trials with the highest level of compliance and those with lower levels of compliance and adherence to guidelines by journal type.

STUDY DESIGN

This was a cross-sectional study of obstetrics and gynecology trials published between 2017 and 2019 in 6 journals (American Journal of Obstetrics & Gynecology, BJOG: An International Journal of Obstetrics and Gynaecology, Obstetrics & Gynecology, The Journal of the American Medical Association, The Lancet, and The New England Journal of Medicine). Randomized controlled trials were identified via PubMed and manual journal archive searches. The primary outcome was adequate compliance with the Consolidated Standards of Reporting Trials guidelines defined as ≥80% of the checklist items present. Secondary outcomes included completion of pretrial registration and concordance between the pretrial registration and publication in terms of the outcomes and sample size. We compared the characteristics between trials with adequate compliance and those with inadequate compliance. Secondary analyses included comparisons of characteristics of the trials in the top quartile for compliance with the Consolidated Standards of Reporting Trials guidelines with those of the trials in lower quartiles and compliance with guidelines in obstetrics-gynecology vs non-obstetrics-gynecology journals. In an exploratory analysis, trends in compliance with the Consolidated Standards of Reporting Trials guidelines across the study period were assessed. A post hoc sensitivity analysis evaluated the outcomes after the exclusion of 2 retracted trials.

RESULTS

Of the 170 trials included, 80% (95% confidence interval, 74%-86%) were adequately compliant with the Consolidated Standards of Reporting Trials manuscript guidelines and 66% (95% confidence interval, 59%-73%) were compliant with the abstract guidelines. Nearly all trials (98%) reported pretrial registration. Concordance between pretrial registration and publication in terms of the primary outcomes was identified for 77% of the trials, concordance in terms of the secondary outcomes was observed in 32% of the trials, and concordance in terms of sample size was observed in 60% of the trials. Trials with adequate compliance were more likely to be preregistered, include an a priori power calculation, and use an intent to treat analysis. Trials in the top quartile for compliance with the Consolidated Standards of Reporting Trials guidelines were more likely to be multicenter, international, and government funded. More trials from non-obstetrics-gynecology journals were in the top quartile for compliance with the Consolidated Standards of Reporting Trials guidelines than trials from obstetrics-gynecology journals (64.9% vs 25.7%; P<.001). No significant trends in adequate compliance were identified across the study period. Results did not differ significantly in the sensitivity analysis.

CONCLUSION

Of all the trials included, 20% of obstetrics-gynecology trials published in 6 high-impact journals were not compliant with the Consolidated Standards of Reporting Trials guidelines, and there were major discrepancies between pretrial registration and publication. Transparency, reproducibility, and scientific rigor in obstetrics and gynecology trial reporting needs to be improved.

Trends and predictors of biomedical research quality, 1990-2015: a meta-research study

OBJECTIVE

To measure the frequency of adequate methods, inadequate methods and poor reporting in published randomised controlled trials (RCTs) and test potential factors associated with adequacy of methods and reporting.

DESIGN

Retrospective analysis of RCTs included in Cochrane reviews. Time series describes the proportion of RCTs using adequate methods, inadequate methods and poor reporting. A multinomial logit model tests potential factors associated with methods and reporting, including funding source, first author affiliation, clinical trial registration status, study novelty, team characteristics, technology and geography.

DATA

Risk of bias assessments for random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data and selective reporting, for each RCT, were mapped to bibliometric and funding data.

OUTCOMES

Risk of bias on six methodological dimensions and RCT-level overall assessment of adequate methods, inadequate methods or poor reporting.

RESULTS

This study analysed 20 571 RCTs. 5.7% of RCTs used adequate methods (N=1173). 59.3% used inadequate methods (N=12 190) and 35.0% were poorly reported (N=7208). The proportion of poorly reported RCTs decreased from 42.5% in 1990 to 30.2% in 2015. The proportion of RCTs using adequate methods increased from 2.6% in 1990 to 10.3% in 2015. The proportion of RCTs using inadequate methods increased from 54.9% in 1990 to 59.5% in 2015. Industry funding, top pharmaceutical company affiliation, trial registration, larger authorship teams, international teams and drug trials were associated with a greater likelihood of using adequate methods. National Institutes of Health funding and university prestige were not.

CONCLUSION

Even though reporting has improved since 1990, the proportion of RCTs using inadequate methods is high (59.3%) and increasing, potentially slowing progress and contributing to the reproducibility crisis. Stronger incentives for the use of adequate methods are needed.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Misleading reporting and interpretation of results in major infertility journals

OBJECTIVE

To evaluate the proportion of randomized controlled trials (RCTs) published in top infertility journals indexed on PubMed that reported their results with proper effect estimates and their precision estimation, while correctly interpreting both measures.

DESIGN

Cross-sectional study evaluating all the RCTs published in top infertility journals during 2014.

SETTING

Not applicable.

PATIENT(S)

Not applicable.

INTERVENTION(S)

Not applicable.

MAIN OUTCOME MEASURE(S)

Proportion of RCTs that reported both relative and absolute effect size measures and its precision.

RESULT(S)

Among the 32 RCTs published in 2014 in the top infertility journals reviewed, 37.5% (95% confidence interval [CI], 21.1-56.3) did not mention in their abstracts whether the difference among the study arms was statistically or clinically significant, and only 6.3% (95% CI, 0.8-20.8) used a CI of the absolute difference. Similarly, in the results section, these elements were observed in 28.2% (95% CI, 13.7-46.7) and 15.6% (95% CI, 5.3-32.8), respectively. Only one study clearly expressed the minimal clinically important difference in their methods section, but we found related proxies in 53% (95% CI, 34.7-70.9). None of the studies used CIs to draw conclusions about the clinical or statistical significance. We found 13 studies where the interpretation of the findings could be misleading.

CONCLUSION(S)

Recommended reporting items are underused in top infertility journals, which could lead to misleading interpretations. Authors, reviewers, and editorial boards should emphasize their use to improve reporting quality.