Number and quality of randomized controlled trials in obstetrics published in the top general medical and obstetrics and gynecology journals

Retracted vs non-retracted obstetrical randomized trials: Which quality criteria are most associated with retraction for untrustworthiness?

BACKGROUND

Randomized control trials (RCTs) are an essential pillar of scientific knowledge and medical practice, and their integrity has important implications for reliable systemic reviews and meta-analyses. However, the number of article retractions due to falsified data and scientific misconduct has increased in recent years. In response, the scientific community has pursued the creation of quality criteria that can be utilized to promote trustworthiness.

METHODS

After a quality criteria checklist was created by a team of experts, retracted and nonretracted studies were evaluated for adherence to assess the checklist's usefulness and identify potential pitfalls. Retracted studies published in obstetric literature and retracted between 1994-2024 were identified using the online Retraction Watch Database. A previously created database of non-retracted obstetric RCTs published between 2018-2020 was used for the control group.

RESULTS

A total of 173 studies were identified, 136 non-retracted and 37 retracted. Overall, 13 of 17 (76.5 %) criteria were statistically different between retracted and non-retracted articles. A cutoff of ≤ 11 total quality criteria granted 94.9 % (95 % CI, 89.7 - 97.9) sensitivity and 78.4 % (95 % CI, 61.8 - 90.2) specificity in distinguishing non-retracted from retracted studies.

CONCLUSIONS

Retracted studies were significantly less likely to adhere to the 17-quality criteria checklist compared to non-retracted studies, providing useful insight to peer-reviewed scientific journals about what to evaluate for in an RCT manuscript prior to publication. The authors recommend that journal editors play close attention to criteria related to research ethics, data falsification, and risk of bias.

Antenatal Physical Activity Interventions and Pregnancy Outcomes: A Systematic Review and Meta-Analysis With a Focus on Trial Quality

BACKGROUND

Guidelines recommending regular physical activity in pregnancy for improving pregnancy outcomes are informed by published meta-analyses. Inclusion of randomised trials of poor methodological quality may bias effect estimates.

OBJECTIVES

To assess the validity of these recommendations by focusing on trial quality.

SEARCH STRATEGY

Systematic search of PubMed, PubMed Central, Ovid Medline, Embase, Cochrane Central Register of Controlled Trials, and CINAHL from inception to 14 December 2023.

SELECTION CRITERIA

Randomised trials evaluating an antenatal physical activity intervention alone, compared with no such intervention.

DATA COLLECTION AND ANALYSIS

Trial quality was assessed using the Cochrane Risk of Bias tool. Independent of this, studies were grouped based on degree of deviation from the intention to treat principle. Sequential meta-analysis was performed in which greater degrees of potential bias were allowed. Between intervention group comparisons used, relative risks or mean differences with 95% confidence intervals for dichotomous outcomes and continuous outcomes, respectively.

MAIN RESULTS

Overall, the quality of trial reporting was low. Only 5 trials (12.5%) were performed and analysed in keeping with the intention to treat principle. When considering only those trials performed rigorously, there was no evidence that antenatal physical activity improves pregnancy outcomes or limits gestational weight gain (WMD -0.60 kg; 95% CI -2.17, 0.98 WMD -0.60 kg; 95% CI -2.17, 0.98).

CONCLUSIONS

When considering only trials at no/negligible risk of bias, antenatal physical activity interventions were not associated with improved pregnancy outcomes. Most trials were not methodologically rigorous. Incorporation of such meta-analyses into pregnancy care guidelines may result in inaccurate recommendations.

Exploring the Limits of Artificial Intelligence for Referencing Scientific Articles

OBJECTIVE

 To evaluate the reliability of three artificial intelligence (AI) chatbots (ChatGPT, Google Bard, and Chatsonic) in generating accurate references from existing obstetric literature.

STUDY DESIGN

 Between mid-March and late April 2023, ChatGPT, Google Bard, and Chatsonic were prompted to provide references for specific obstetrical randomized controlled trials (RCTs) published in 2020. RCTs were considered for inclusion if they were mentioned in a previous article that primarily evaluated RCTs published by the top medical and obstetrics and gynecology journals with the highest impact factors in 2020 as well as RCTs published in a new journal focused on publishing obstetric RCTs. The selection of the three AI models was based on their popularity, performance in natural language processing, and public availability. Data collection involved prompting the AI chatbots to provide references according to a standardized protocol. The primary evaluation metric was the accuracy of each AI model in correctly citing references, including authors, publication title, journal name, and digital object identifier (DOI). Statistical analysis was performed using a permutation test to compare the performance of the AI models.

RESULTS

 Among the 44 RCTs analyzed, Google Bard demonstrated the highest accuracy, correctly citing 13.6% of the requested RCTs, whereas ChatGPT and Chatsonic exhibited lower accuracy rates of 2.4 and 0%, respectively. Google Bard often substantially outperformed Chatsonic and ChatGPT in correctly citing the studied reference components. The majority of references from all AI models studied were noted to provide DOIs for unrelated studies or DOIs that do not exist.

CONCLUSION

 To ensure the reliability of scientific information being disseminated, authors must exercise caution when utilizing AI for scientific writing and literature search. However, despite their limitations, collaborative partnerships between AI systems and researchers have the potential to drive synergistic advancements, leading to improved patient care and outcomes.

KEY POINTS

· AI chatbots often cite scientific articles incorrectly.. · AI chatbots can create false references.. · Responsible AI use in research is vital..

Are reporting guidelines used in infectious diseases publications? An analysis of more than 1,000 articles

Objective

To assess whether 16 reporting guidelines of Enhancing the QUAlity and Transparency Of Health Research (EQUATOR) were used in infectious diseases research publications.

Design

This cross-sectional, audit-type study assessed articles published in five infectious diseases journals in 2019.

Methods

All articles were manually reviewed to assess if a reporting guideline was advisable and searched for the names and acronyms of 16 reporting guidelines. An "advisable use rate" was calculated.

Results

We reviewed 1,251 manuscripts across five infectious diseases journals. Guideline use was advisable for 973 (75%) articles. Reporting guidelines were used in 85 articles, 6.1% of total articles, and 8% (95% CI 6%-9%) of articles for which guidelines were advised. The advisable use rate ranged from 0.06 to 0.17 for any guideline, 0-0.08 for CONSORT, 0.53-1 for Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and 0-0.66 for Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) : The TRIPOD statement. No trends were observed across the five journals.

Conclusions

The use of EQUATOR-related reporting guidelines is infrequent, despite journals and publishers promoting their usage. Whether this finding is attributable to knowledge, acceptance, or perceived usefulness of the guidelines still needs to be clarified.

Standard quality criteria in retracted vs nonretracted obstetrical randomized controlled trials

BACKGROUND

The number of retracted articles in peer-reviewed journals is increasing within the field of obstetrics. The most common reason for article retraction is scientific misconduct. Unfortunately, article retraction often occurs years after publication, allowing inaccurate data to be widely distributed to readers. There exists a great need for validated screening criteria for obstetric journals to use when reviewing randomized controlled trials for scientific misconduct.

OBJECTIVE

This study aimed to compare retracted obstetric randomized controlled trials with nonretracted randomized controlled trials with regard to their inclusion of 7 quality metrics: prospective trial registration, trial registration number, ethics approval statement, name of the approving committee, statement of informed consent, adherence to the Consolidated Standards of Reporting Trials guidelines, and a data sharing statement.

STUDY DESIGN

Obstetric randomized controlled trials retracted between 1995 and 2021 identified through Retraction Watch were compared with nonretracted randomized controlled trials published between 2018 and 2020 with regard to inclusion of the 7 quality metrics. The main outcome was the difference in prospective trial registration. Secondary outcomes were the percentage of individual criteria met and the screening performance of quality criteria in predicting article retraction.

RESULTS

A total of 150 randomized controlled trials were identified, of which 14 (9.3%) were retracted and 136 (90.7%) nonretracted. Retracted randomized controlled trials were less likely than nonretracted randomized controlled trials to be prospectively registered (14.3% vs 80.1%; P<.001). The median number of quality criteria met was lower for retracted randomized controlled trials (3 vs 6; P<.01). Using a cutoff of ≤4 criteria granted 85.7% (95% confidence interval, 57.2-98.2) sensitivity and 92.0% (95% confidence interval, 86.2-96.0) specificity in distinguishing the retracted randomized controlled trials from nonretracted studies.

CONCLUSION

Retracted obstetric randomized controlled trials were less likely to include the 7 quality metrics required on submission by most top obstetrics and gynecology journals.

How many US obstetrical trials reach publication? A cross-sectional analysis of ClinicalTrials.gov and PubMed from 2007 to 2019

BACKGROUND

Obstetrical clinical trials are the foundation of evidence-based medicine during pregnancy. As more obstetrical trials are conducted, understanding the publication characteristics of these trials is of utmost importance to advance obstetrical health.

OBJECTIVE

This study aimed to characterize the frequency of publication and trial characteristics associated with publication among obstetrical clinical trials in the United States. We additionally sought to examine time from trial completion to publication.

STUDY DESIGN

This was a cross-sectional analysis of completed obstetrical trials with an intervention design and at least 1 site in the United States registered to ClinicalTrials.gov from 2007 to 2019. Trial characteristics were cross-referenced with PubMed to determine publication status up to 2021 using the National Clinical Trial identification number. Bivariable analyses were conducted to determine trial characteristics associated with publication. Multivariable logistic regression models controlling for prespecified covariates were generated to estimate the relationship between funding, primary purpose, and therapeutic foci with publication. Additional exploratory analyses of other trial characteristics were conducted. Time to publication was analyzed using Kaplan-Meier curves and Cox regression models.

RESULTS

Of the 1879 obstetrical trials with registered completion, a total of 575 (30.6%) had at least 1 site in the United States, were completed before October 1, 2019, and were included in this analysis. Between October 2007 and October 2019, fewer than two-thirds (N=348, 60.5%) of trials reached publication. Annual rates of publication ranged from 46.4% in 2018 to 70.0% in 2007. No difference was observed in publication by funding, primary purpose, or therapeutic foci (all P>.05). Trials with characteristics indicating high trial quality-including randomized allocation scheme, ≥50 participants enrolled, ≥2 sites, and presence of a data safety monitoring committee-had increased odds of publication compared with those without such characteristics (all P<.05). For example, studies with randomized allocation of intervention had 2-fold greater odds of publication than nonrandomized studies (adjusted odds ratio, 2.09; 95% confidence interval, 1.30-3.37). Studies with ≥150 participants had nearly 8-fold odds of publication (adjusted odds ratio, 7.90; 95% confidence interval, 3.78-17.49) relative to studies with <50 participants. Temporal analysis demonstrated variability in time to publication among obstetrical trials, with a median time of 20.1 months after trial completion, and with most trials that reached publication having been published by 40 months. No difference was observed in time to publication by funding, primary purpose, or therapeutic foci (all P>.05).

CONCLUSION

Publication of obstetrical trials remains suboptimal, with significant differences observed between trials with indicators of high quality and those without. Most trials that reach publication are published within 2 years of registered completion on ClinicalTrials.gov.