Selective outcome reporting is present in randomized controlled trials in lung cancer immunotherapies

Selective outcome reporting in randomized clinical trials using the third molar surgery model

Selective outcome reporting (SOR) can threaten the validity of results found in clinical trials. Some studies in the literature have analyzed SOR in dentistry, but there is no study that has observed SOR in clinical trials in oral and maxillofacial surgery. Impacted third molar surgery is one of the most used models in clinical trials to study mainly analgesic and anti-inflammatory drug interventions. Our study aimed to evaluate the prevalence of SOR in publications employing the third molar extraction clinical trial model, and to verify whether there was an association between the statistical significance of outcomes and other characteristics that could lead to SOR. A systematic search was performed on the ClinicialTrials.gov platform for randomized clinical trial protocols, using the condition of third molar extraction. The corresponding published articles were sourced in PubMed, Scopus, and Embase databases, and compared with the registered protocols regarding the methodological data, in terms of: sample calculation, primary outcome identification, end-point periods, insertion of new outcomes in the publication, and results of outcomes. 358 protocol records were retrieved; 87 presented their corresponding articles. SOR was identified in 28.74% of the publications, and had a significant relationship with changes in the protocol, insertions of new outcomes, and discrepancies in the types of study. General risk of bias was found to be low. There were associations between SOR and the discrepancies in terms of the type of study, the choice of new outcome, and changes in the history of protocol records. The prevalence of SOR in clinical research using the third molar extraction surgery model is moderate. The quality of the scientific reporting of the results and, consequently, the certainty of evidence relating to the intervention tested can be overstated, increasing the chances of misinterpretation by health professionals.

Estimating the prevalence of discrepancies between study registrations and publications: a systematic review and meta-analyses

OBJECTIVES

Prospectively registering study plans in a permanent time-stamped and publicly accessible document is becoming more common across disciplines and aims to reduce risk of bias and make risk of bias transparent. Selective reporting persists, however, when researchers deviate from their registered plans without disclosure. This systematic review aimed to estimate the prevalence of undisclosed discrepancies between prospectively registered study plans and their associated publication. We further aimed to identify the research disciplines where these discrepancies have been observed, whether interventions to reduce discrepancies have been conducted, and gaps in the literature.

DESIGN

Systematic review and meta-analyses.

DATA SOURCES

Scopus and Web of Knowledge, published up to 15 December 2019.

ELIGIBILITY CRITERIA

Articles that included quantitative data about discrepancies between registrations or study protocols and their associated publications.

DATA EXTRACTION AND SYNTHESIS

Each included article was independently coded by two reviewers using a coding form designed for this review (osf.io/728ys). We used random-effects meta-analyses to synthesise the results.

RESULTS

We reviewed k=89 articles, which included k=70 that reported on primary outcome discrepancies from n=6314 studies and, k=22 that reported on secondary outcome discrepancies from n=1436 studies. Meta-analyses indicated that between 29% and 37% (95% CI) of studies contained at least one primary outcome discrepancy and between 50% and 75% (95% CI) contained at least one secondary outcome discrepancy. Almost all articles assessed clinical literature, and there was considerable heterogeneity. We identified only one article that attempted to correct discrepancies.

CONCLUSIONS

Many articles did not include information on whether discrepancies were disclosed, which version of a registration they compared publications to and whether the registration was prospective. Thus, our estimates represent discrepancies broadly, rather than our target of undisclosed discrepancies between prospectively registered study plans and their associated publications. Discrepancies are common and reduce the trustworthiness of medical research. Interventions to reduce discrepancies could prove valuable.

REGISTRATION

osf.io/ktmdg. Protocol amendments are listed in online supplemental material A.

Selective outcome reporting bias is highly prevalent in randomized clinical trials of nonsurgical periodontal therapy

Selective outcome reporting (SOR) is a type of bias that can compromise the validity of results and affect evidence-based practice. SOR can overestimate the effect of an intervention and lead to conclusions that a treatment is effective when it is not. This study aimed to investigate the prevalence of SOR in publications of RCTs on nonsurgical periodontal therapy (NSPT) and to verify associated factors. The protocols were searched and selected on the www.clinicaltrials.gov platform up to January 16, 2022. Corresponding publications were identified, and data extraction and discrepancy analysis were performed. The risk of bias was assessed according to the RoB2 tool. One hundred forty-five studies (174 publications) were included. The prevalence of SOR was 49.7% and was unclear in nearly one third of studies (27.6%). Only 31.7% of the primary outcomes were completely described in the publications. The overall risk of bias was high in 60% of the included studies. SOR was associated with statistical significance (p < .001), and multiple publications of the same study (p = .005). Our study demonstrated the high prevalence of SOR, highlighting the need to improve the quality of reporting of RCTs on NSPT studies.

Selective outcome reporting in root coverage randomized clinical trials

BACKGROUND

Outcome discrepancies between protocols and respective publications represent a concerning bias. The purpose of this study was to assess the prevalence of selective outcome reporting (SOR) in root coverage randomized clinical trials (RCTs).

METHODS

Published root coverage RCTs (July 2005 to March 2020) were included if a corresponding protocol could be identified in a public registry. Discrepancies between protocol and its correspondent publication(s) were compared regarding primary and secondary outcomes and other study characteristics. Associations between trial characteristics and SOR were evaluated.

RESULTS

Forty four studies (54 publications) were included. The majority of studies (77.3%) were retrospectively registered. SOR was frequent (40.9% of trials) and consisted of primary outcome downgrade (22.7%); secondary outcome upgrade (11.4%); new primary outcome introduced in publication (25%); protocol primary outcome omitted from publication (13.6%) and discrepancy in primary outcome timing (18.2%). SOR was unclear in 20.5% of studies and favoured statistical significance in 12 studies (27.3%). SOR was significantly associated with study significance (p < 0.001) and unclear outcome definition in the publication (p < 0.001). Only a third (32.8%) of primary outcomes were completely defined.

CONCLUSIONS

The present study identified high prevalence of SOR in root coverage RCTs.

Assessment of transparent and reproducible research practices in the psychiatry literature

BACKGROUND

Reproducibility is a cornerstone of scientific advancement; however, many published works may lack the core components needed for study reproducibility.

AIMS

In this study, we evaluate the state of transparency and reproducibility in the field of psychiatry using specific indicators as proxies for these practices.

METHODS

An increasing number of publications have investigated indicators of reproducibility, including research by Harwicke et al, from which we based the methodology for our observational, cross-sectional study. From a random 5-year sample of 300 publications in PubMed-indexed psychiatry journals, two researchers extracted data in a duplicate, blinded fashion using a piloted Google form. The publications were examined for indicators of reproducibility and transparency, which included availability of: materials, data, protocol, analysis script, open-access, conflict of interest, funding and online preregistration.

RESULTS

This study ultimately evaluated 296 randomly-selected publications with a 3.20 median impact factor. Only 107 were available online. Most primary authors originated from USA, UK and the Netherlands. The top three publication types were cohort studies, surveys and clinical trials. Regarding indicators of reproducibility, 17 publications gave access to necessary materials, four provided in-depth protocol and one contained raw data required to reproduce the outcomes. One publication offered its analysis script on request; four provided a protocol availability statement. Only 107 publications were publicly available: 13 were registered in online repositories and four, ten and eight publications included their hypothesis, methods and analysis, respectively. Conflict of interest was addressed by 177 and reported by 31 publications. Of 185 publications with a funding statement, 153 publications were funded and 32 were unfunded.

CONCLUSIONS

Currently, Psychiatry research has significant potential to improve adherence to reproducibility and transparency practices. Thus, this study presents a reference point for the state of reproducibility and transparency in Psychiatry literature. Future assessments are recommended to evaluate and encourage progress.

Selective outcome reporting in randomized clinical trials of dental implants

AIM

Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants.

MATERIALS AND METHODS

Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols.

RESULTS

A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04).

CONCLUSIONS

Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.