Selective outcome reporting in randomised controlled trials including participants with stroke or transient ischaemic attack: A systematic review

INTRODUCTION

The prevalence of outcome reporting bias (ORB, i.e. selective reporting according to the results observed) across primary outcomes in randomised controlled trials (RCTs) including participants with stroke or transient ischaemic attack (TIA) is unknown.

MATERIALS AND METHODS

We searched the Cochrane Database of Systematic Reviews on 3 February 2021 for reviews published 2008-2020 with at least one RCT of a therapeutic intervention, for participants with stroke or TIA, and a safety or efficacy outcome. We took a random sample of these RCTs and included those with a trial registry record or protocol published before reporting results. Two reviewers assessed discrepancies in outcome reporting across the trial registry record, protocol, statistical analysis plan, and publication for each RCT, using the classification system designed by the Outcome Reporting Bias in Trials group.

RESULTS

Of 600 RCTs, we identified a trial registry record in 120 (20%), a protocol in 28 (5%), and a statistical analysis plan in 5 (1%) with 123 (21%) distinct RCTs being eligible for assessment: 110 (89%, 95% CI 83-94) were at no risk, 7 (6%, 95% CI 3-11) RCTs were at low risk, and 6 (5%, 95% CI 2-10) were at high risk of ORB.

DISCUSSION

The prevalence of ORB in primary outcomes was low in stroke/TIA RCTs that were included in Cochrane reviews and had an identifiable trial registry record or protocol. Concerningly, we were unable to identify a trial registry record or protocol in most of our sample.

CONCLUSION

Work is needed to further reduce ORB in stroke/TIA RCTs and explore the generalisability of these findings to RCTs outside of Cochrane reviews or without a registry record or protocol, as well as to secondary outcomes.

Selective outcome reporting across psychopharmacotherapy randomized controlled trials

OBJECTIVE

Selective reporting impairs the valid interpretation of trials and leads to bias with regards to the clinical evidence. We aimed to examine factors associated with selective reporting in psychopharmacotherapy trials and thus enable solutions to prevent such selective reporting in the future.

METHODS

We retrieved all registry records of trials investigating medication for depressive, bipolar and psychotic disorders. Multivariate logistic regression was performed with selective reporting as outcome, and funding source, psychiatric disorder, year of study start date, participating centers, and anticipated sample size as explanatory variables, after testing for multicollinearity. Adjusted odds ratios (AOR) were calculated. Two-sided Fisher exact test was used to compare the proportions of newly added positive primary outcomes with the proportions of positive results in the overall group of primary outcomes.

RESULTS

Of 151 included trials (N = 94,303 participants), 21 (14%) showed irregularities between registered and published primary outcomes. Higher odds of such irregularities were associated with non-industry-funded RCTs (AOR 5.3; p = 0.014) and trials investigating major depressive disorder (AOR 12.7; p = 0.024) or schizophrenia (AOR 14.5; p = 0.016; Table 1).

CONCLUSION

We demonstrate discrepancies between trial registrations and publications across RCTs investigating debilitating psychiatric disorders, especially in non-industry funded RCTs.

Around ten percent of most recent Cochrane reviews included outcomes in their literature search strategy and were associated with potentially exaggerated results: A research-on-research study

OBJECTIVES

To assess the proportion of the recent Cochrane reviews that included outcomes in their literature search strategy, how often they acknowledged these limitations, and how qualitatively different the results of outcomes included and not included in the search strategy were.

DESIGN AND SETTING

We identified all the Cochrane reviews of the interventions published in 2020 that used a search strategy connecting outcome terms with "AND." Reviews were defined as acknowledging the limitations of searching for outcomes if they mentioned them in the discussion. We compared the characteristics of outcomes included and not included in the search strategy.

RESULTS

Of the 523 Cochrane reviews published in 2020, 51 (9.8%) included outcomes in their search strategy. Only one review acknowledged it as a limitation. Forty-seven (92%) assessed outcomes not included in the search strategy. Outcomes included in the search strategies tended to include a larger number of studies and show their effects in favor of the intervention.

CONCLUSIONS

Around ten percent of the recent Cochrane reviews included outcomes in their search, which may have resulted in more outcomes significantly in favor of the intervention. Reviewers should be more explicit in acknowledging the potential implications of searching for outcomes.

Trial registration and selective outcome reporting in Endodontic Research: Evidence over a 5-year period

AIM

To assess the prevalence of registration of clinical trials in endodontic research and to identify outcome reporting discrepancies between trial registration entries and respective final publications. Associations with publication characteristics, such as journal, year of publication, origin, number of centres and authors, funding and statistical significance of the findings, were also sought.

METHODOLOGY

All reports of endodontic randomized controlled trials (RCTs) published in 4 endodontic specialty and 4 general dental journals from 1 January 2016 to 31 December 2020 were identified. Trial registration frequency patterns were assessed for the included RCTs, whilst for the ones registered, outcome reporting discrepancies were recorded. Article characteristics such as year of publication, geographic region, number of centres, authors participating in the publication, funding, type of registration and others were identified. Descriptive statistics and univariable/multivariable logistic regression were performed to examine the effect of study characteristics on identified registration practices.

RESULTS

One hundred and fifty-five RCTs were included, with the majority published in specialty journals (121/155; 78.1%). A total of 42.6% of the identified RCTs was registered (66/155), mostly retrospectively (38/66; 57.6%). There was strong evidence that each additional year for more recent publication accounted for 1.42 times higher odds of being registered (adjusted Odds Ratio = 1.42; 95%CI: 1.11, 1.80; p = .004). More than 1/3 of registered RCTs presented outcome reporting discrepancies (24/66; 36.4%), whilst such inconsistencies were almost evenly distributed between primary and secondary outcomes.

CONCLUSIONS

Trial registration policies in endodontic research should be reviewed and active endorsement of prospective registration practices should be prioritized for better clarity and transparency of the disseminated research. Outcome reporting discrepancies shall thus be eliminated, offering increased credibility in research findings and eliminating bias in this respect.

Evaluation of selective outcome reporting and trial registration practices among addiction clinical trials

BACKGROUND AND AIMS

Selective outcome reporting occurs when trialists pre-specify primary and secondary outcomes during trial planning but alter the definitions in the published report. Here, we investigate selective outcome reporting in published addiction randomized controlled trials (RCTs) and evaluate whether particular funding sources are associated with an increased likelihood of selective outcome reporting.

DESIGN

We conducted a cross-sectional study of published addiction clinical trials. A PubMed search was performed to identify RCTs in addiction journals from 2013 to 2017. Included studies used a randomized design to address one of the following topics: (1) drug, alcohol and tobacco addiction prevention, (2) stabilization following excessive use of a substance, (3) relapse prevention or (4) recovery maintenance.

SETTING

Single-center, medical research institution.

PARTICIPANTS

Our sample included 162 RCTs that were prospectively registered with a clearly defined primary outcome.

MEASUREMENT

We extracted the following items from addiction RCTs: journal, funding source, trial registry number (if included), sample size, dates of subject enrollment, whether primary and secondary outcomes were denoted, all published outcomes, P-value for all outcomes and whether authors mentioned any deviations from the trial protocol as it related to RCT outcomes.

FINDINGS

In total, 47 of 162 RCTs (29.0%) had at least one major discrepancy between the trial registry and published RCT. Overall, these 47 RCTs included 54 major discrepancies. The most common major discrepancy was demotion of a primary registered outcome (19/54, 35.2%). The majority of RCTs (132/162, 81.5%) were funded from public sources. Additionally, 166 RCTs were excluded from our sample because registration could not be confirmed.

CONCLUSIONS

There is evidence suggestive of selective outcome reporting in addiction randomized controlled trials (RCTs). The most common major discrepancies pertained to the primary outcome.

Published intimate partner violence studies often differ from their trial registration records

INTRODUCTION

Registering study protocols in a trial registry is important for methodologic transparency and reducing selective reporting bias. The objective of this investigation was to determine whether published studies of intimate partner violence (IPV) that had been registered matched the registration record on key study design elements.

METHODS

We systematically searched three trial registries to identify registered IPV studies and the published literature for the associated publication. Two authors independently determined for each study whether key study elements in the registry matched those in the published paper.

RESULTS

We included 66 studies published between 2006 and 2017. Nearly half (29/66, 44%) were registered after study completion. Many (26/66, 39%) had discrepancies regarding the primary outcome, and nearly two-thirds (42/66, 64%) had discrepancies in secondary outcomes. Discrepancies in study design were less frequent (13/66, 20%). However, large changes in sample size (26/66, 39%) and discrepancies in funding source (28/66, 42%) were frequently observed.

CONCLUSIONS

Trial registries are important tools for research transparency and identifying and preventing outcome switching and selective outcome reporting bias. Published IPV studies often differ from their records in trial registries. Researchers should pay close attention to the accuracy of trial registry records.

Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study)

Data for individual trials included in systematic reviews may be available in multiple sources. For example, a single trial might be reported in 2 journal articles and 3 conference abstracts. Because of differences across sources, source selection can influence the results of systematic reviews. We used our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study, and evidence from previous studies, to develop practical guidance for using multiple data sources in systematic reviews. We recommend the following: (1) Specify which sources you will use. Before beginning a systematic review, consider which sources are likely to contain the most useful data. Try to identify all relevant reports and to extract information from the most reliable sources. (2) Link individual trials with multiple sources. Write to authors to determine which sources are likely related to the same trials. Use a modified Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flowchart to document both the selection of trials and the selection of sources. (3) Follow a prespecified protocol for extracting trial characteristics from multiple sources. Identify differences among sources, and contact study authors to resolve differences if possible. (4) Prespecify outcomes and results to examine in the review and meta-analysis. In your protocol, describe how you will handle multiple outcomes within each domain of interest. Look for outcomes using all eligible sources. (5) Identify which data sources were included in the review. Consider whether the results might have been influenced by data sources used. (6) To reduce bias, and to reduce research waste, share the data used in your review.

Selective outcome reporting in obesity clinical trials: a cross-sectional review

Selective outcome reporting is a form of bias resulting from discrepancies between outcomes presented in a trial's registration and the published report. We investigate this selective bias in obesity clinical trials. A PubMed search was conducted to identify randomized controlled trials (RCTs) published in four obesity journals from 2013 to 2015. Primary, secondary and tertiary outcomes were recorded for each trial and compared to pre-specified outcomes in each trial's registration. Of the 392 identified articles, 142 were included in the final analysis; 22 (15%) RCTs demonstrated major outcome discrepancies between registration and publication: No primary outcomes were demoted to a secondary or tertiary outcome; 14 (36.84%) primary outcomes were omitted; 14 (36.84%) primary outcomes were added: 5 (13.16%) secondary outcomes were upgraded to primary outcomes; and timing of assessment for a primary outcome changed 5 (13.16%) times. Out of the 63 prospectively registered studies, 53 had no discrepancies. A total of 76 of the studies (29.80%) were unregistered or did not have an associated registration number. Our results suggest that selective outcome reporting may be a concern in obesity clinical trials. As selective outcome reporting may distort clinical findings and limit outcomes in systematic reviews, we encourage trialists and journal editors to work towards solutions to mitigate this issue.

A test for reporting bias in trial networks: simulation and case studies

BACKGROUND

Networks of trials assessing several treatment options available for the same condition are increasingly considered. Randomized trial evidence may be missing because of reporting bias. We propose a test for reporting bias in trial networks.

METHODS

We test whether there is an excess of trials with statistically significant results across a network of trials. The observed number of trials with nominally statistically significant p-values across the network is compared with the expected number. The performance of the test (type I error rate and power) was assessed using simulation studies under different scenarios of selective reporting bias. Examples are provided for networks of antidepressant and antipsychotic trials, where reporting biases have been previously demonstrated by comparing published to Food and Drug Administration (FDA) data.

RESULTS

In simulations, the test maintained the type I error rate and was moderately powerful after adjustment for type I error rate, except when the between-trial variance was substantial. In all, a positive test result increased moderately or markedly the probability of reporting bias being present, while a negative test result was not very informative. In the two examples, the test gave a signal for an excess of statistically significant results in the network of published data but not in the network of FDA data.

CONCLUSION

The test could be useful to document an excess of significant findings in trial networks, providing a signal for potential publication bias or other selective analysis and outcome reporting biases.

Clinical trial registries are of minimal use for identifying selective outcome and analysis reporting

OBJECTIVE

This study aimed to examine selective outcome reporting (SOR) and selective analysis reporting (SAR) in randomized controlled trials (RCTs) and to explore the usefulness of trial registries for identifying SOR and SAR.

STUDY DESIGN AND SETTING

We selected one "index outcome" for each of three comparative effectiveness reviews (CERs) of pharmacotherapy and extracted data on this outcome from trial registries and from study publications.

RESULTS

Among 50 RCTs published since 2005 and reporting the index outcome, only 50% were listed in registries; 90% of RCTs were assessed as having SOR or SAR. The index outcome in the registry was different from that in the publication in 75% of trials in two CERs, and not specified at all in the third. Reported outcomes and analyses were not consistent between the publication's methods section and the results section in 33% and 46% of the two CERs where the index outcome was a benefit. There were no statistically significant predictors of SOR and SAR in our small sample where some predictors lacked variability.

CONCLUSION

The SOR and SAR were frequent in this pilot study, and the most common type of SOR was the publication of outcomes that were not pre-specified. Trial registries were of little use in identifying SOR and of no use in identifying SAR.