Two prognostic indicators of the publication rate of clinical studies were available during ethical review

Non-Publication Is Common among Phase 1, Single-Center, Not Prospectively Registered, or Early Terminated Clinical Drug Trials

The objective of this study was to investigate the occurrence and determinants of non-publication of clinical drug trials in the Netherlands.All clinical drug trials reviewed by the 28 Institutional Review Boards (IRBs) in the Netherlands in 2007 were followed-up from approval to publication. Candidate determinants were the sponsor, phase, applicant, centers, therapeutic effect expected, type of trial, approval status of the drug(s), drug type, participant category, oncology or other disease area, prospective registration, and early termination. The main outcome was publication as peer reviewed article. The percentage of trials that were published, crude and adjusted odds ratio (OR), and 95% confidence interval (CI) were used to quantify the associations between determinants and publication. In 2007, 622 clinical drug trials were reviewed by IRBs in the Netherlands. By the end of follow-up, 19 of these were rejected by the IRB, another 19 never started inclusion, and 10 were still running. Of the 574 trials remaining in the analysis, 334 (58%) were published as peer-reviewed article. The multivariable logistic regression model identified the following determinants with a robust, statistically significant association with publication: phase 2 (60% published; adjusted OR 2.6, 95% CI 1.1-5.9), phase 3 (73% published; adjusted OR 4.1, 95% CI 1.7-10.0), and trials not belonging to phase 1-4 (60% published; adjusted OR 3.2, 95% CI 1.5 to 6.5) compared to phase 1 trials (35% published); trials with a company or investigator as applicant (63% published) compared to trials with a Contract Research Organization (CRO) as applicant (50% published; adjusted OR 1.7; 95% CI 1.1-2.8); and multicenter trials also conducted in other EU countries (68% published; adjusted OR 2.2, 95% CI 1.1-4.4) or also outside the European Union (72% published; adjusted OR 2.0, 95% CI 1.0-4.0) compared to single-center trials (45% published). Trials that were not prospectively registered (48% published) had a lower likelihood of publication compared to prospectively registered trials (75% published; adjusted OR 0.5, 95% CI 0.3-0.8), as well as trials that were terminated early (33% published) compared to trials that were completed as planned (64% published; adjusted OR 0.2, 95% CI 0.1-0.3). The non-publication rate of clinical trials seems to have improved compared to previous inception cohorts, but is still far from optimal, in particular among phase 1, single-center, not prospectively registered, and early terminated trials.

Systematic review protocol assessing the processes for linking clinical trial registries and their published results

INTRODUCTION

Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias. Our aim is to survey and quantify the processes that have been used to identify links between clinical trial registries and published trial reports in studies that rely on these links to evaluate the completeness and accuracy of trial reporting.

METHODS AND ANALYSIS

We will identify studies that describe a process for identifying the links between a trial registry included in the WHO International Clinical Trial Registry Platform and published trial results, and use those links to evaluate the completeness and accuracy of trial reporting. Information extracted from the studies will include the purpose and application domain of the study, registries used or searched, processes by which the links were identified, the study period and proportions for which links were found. We will summarise what is known about the number and availability of links between clinical trial registries and published results, and examine how automatic linking, inference and inquiry processes have been used to identify links since the introduction of trial registries.

ETHICS AND DISSEMINATION

The systematic review is focused on the analysis of secondary data and does not require ethics approval. The results of the systematic review will be used to inform standard processes used to identify links to and from clinical trial registries in studies that evaluate the completeness and accuracy of clinical trial reports, as well as systematic reviews. Our findings will be disseminated by publishing the systematic review in a peer-reviewed journal, and by engaging with stakeholders from clinical trial registries and bibliographic databases.

A Scoping Review of Empirical Research Relating to Quality and Effectiveness of Research Ethics Review

Background

To date there is no established consensus of assessment criteria for evaluating research ethics review.

Methods

We conducted a scoping review of empirical research assessing ethics review processes in order to identify common elements assessed, research foci, and research gaps to aid in the development of assessment criteria. Electronic searches of Ovid Medline, PsychInfo, and the Cochrane DSR, ACP Journal Club, DARE, CCTR, CMR, HTA, and NHSEED, were conducted. After de-duplication, 4234 titles and abstracts were reviewed. Altogether 4036 articles were excluded following screening of titles, abstracts and full text. A total of 198 articles included for final data extraction.

Results

Few studies originated from outside North America and Europe. No study reported using an underlying theory or framework of quality/effectiveness to guide study design or analyses. We did not identify any studies that had involved a controlled trial - randomised or otherwise – of ethics review procedures or processes. Studies varied substantially with respect to outcomes assessed, although tended to focus on structure and timeliness of ethics review.

Discussion

Our findings indicate a lack of consensus on appropriate assessment criteria, exemplified by the varied study outcomes identified, but also a fragmented body of research. To date research has been largely quantitative, with little attention given to stakeholder experiences, and is largely cross sectional. A lack of longitudinal research to date precludes analyses of change or assessment of quality improvement in ethics review.

Occurrence and determinants of selective reporting of clinical drug trials: design of an inception cohort study

INTRODUCTION

Responsible conduct of research implies that results of clinical trials should be completely and adequately reported. This article describes the design of a cohort study that aims to investigate the occurrence and the determinants of selective reporting in an inception cohort of all clinical drug trials that were reviewed by the Dutch Institutional Review Boards (IRBs) in 2007. It also describes the characteristics of the study cohort.

METHODS AND ANALYSIS

In 2007, Dutch IRBs reviewed 622 clinical drug trials. For each trial, we assessed the stages of progress. We discriminated five intermediate stages and five definite stages. Intermediate stages of progress are: approved by an IRB; started inclusion; completed as planned; terminated early; published as article. The definite stages of progress are: rejected by an IRB; never started inclusion; not published as article; completely reported; selectively reported. We will use univariate and multivariate Cox regression models to identify trial characteristics associated with non-publication. We will identify seven trial-specific discrepancy items, including the objectives, inclusion and exclusion criteria, end points, sample size, additional analyses, type of population analysis and sponsor acknowledgement. The percentage of trials with discrepancies between the protocol and the publication will be scored. We will investigate the association between trial characteristics and the occurrence of discrepancies.

ETHICS AND DISSEMINATION

No IRB-approval is required for this study. Access to confidential research protocols was provided by the Central Committee on Research Involving Human Subjects. We plan to finish data collection in June 2015, and expect to complete data cleaning, analysis and manuscript preparation within the next 3 months. Hence, a first draft of an article containing the results is expected before the end of October 2015.

Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries

Background

The synthesis of published research in systematic reviews is essential when providing evidence to inform clinical and health policy decision-making. However, the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted (dissemination bias). To investigate the extent of dissemination bias we conducted a systematic review that determined the proportion of studies published as peer-reviewed journal articles and investigated factors associated with full publication in cohorts of studies (i) approved by research ethics committees (RECs) or (ii) included in trial registries.

Methods and Findings

Four bibliographic databases were searched for methodological research projects (MRPs) without limitations for publication year, language or study location. The searches were supplemented by handsearching the references of included MRPs. We estimated the proportion of studies published using prediction intervals (PI) and a random effects meta-analysis. Pooled odds ratios (OR) were used to express associations between study characteristics and journal publication. Seventeen MRPs (23 publications) evaluated cohorts of studies approved by RECs; the proportion of published studies had a PI between 22% and 72% and the weighted pooled proportion when combining estimates would be 46.2% (95% CI 40.2%–52.4%, I² = 94.4%). Twenty-two MRPs (22 publications) evaluated cohorts of studies included in trial registries; the PI of the proportion published ranged from 13% to 90% and the weighted pooled proportion would be 54.2% (95% CI 42.0%–65.9%, I² = 98.9%). REC-approved studies with statistically significant results (compared with those without statistically significant results) were more likely to be published (pooled OR 2.8; 95% CI 2.2–3.5). Phase-III trials were also more likely to be published than phase II trials (pooled OR 2.0; 95% CI 1.6–2.5). The probability of publication within two years after study completion ranged from 7% to 30%.

Conclusions

A substantial part of the studies approved by RECs or included in trial registries remains unpublished. Due to the large heterogeneity a prediction of the publication probability for a future study is very uncertain. Non-publication of research is not a random process, e.g., it is associated with the direction of study findings. Our findings suggest that the dissemination of research findings is biased.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review

BACKGROUND

The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.

METHODOLOGY/PRINCIPAL FINDINGS

In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

CONCLUSIONS

This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions

Context

Publication bias jeopardizes evidence-based medicine, mainly through biased literature syntheses. Publication bias may also affect laboratory animal research, but evidence is scarce.

Objectives

To assess the opinion of laboratory animal researchers on the magnitude, drivers, consequences and potential solutions for publication bias. And to explore the impact of size of the animals used, seniority of the respondent, working in a for-profit organization and type of research (fundamental, pre-clinical, or both) on those opinions.

Design

Internet-based survey.

Setting

All animal laboratories in The Netherlands.

Participants

Laboratory animal researchers.

Main Outcome Measure(s)

Median (interquartile ranges) strengths of beliefs on 5 and 10-point scales (1: totally unimportant to 5 or 10: extremely important).

Results

Overall, 454 researchers participated. They considered publication bias a problem in animal research (7 (5 to 8)) and thought that about 50% (32–70) of animal experiments are published. Employees (n = 21) of for-profit organizations estimated that 10% (5 to 50) are published. Lack of statistical significance (4 (4 to 5)), technical problems (4 (3 to 4)), supervisors (4 (3 to 5)) and peer reviewers (4 (3 to 5)) were considered important reasons for non-publication (all on 5-point scales). Respondents thought that mandatory publication of study protocols and results, or the reasons why no results were obtained, may increase scientific progress but expected increased bureaucracy. These opinions did not depend on size of the animal used, seniority of the respondent or type of research.

Conclusions

Non-publication of “negative” results appears to be prevalent in laboratory animal research. If statistical significance is indeed a main driver of publication, the collective literature on animal experimentation will be biased. This will impede the performance of valid literature syntheses. Effective, yet efficient systems should be explored to counteract selective reporting of laboratory animal research.

Ethical review from the inside: repertoires of evaluation in Research Ethics Committee meetings

Evaluating the practice of ethical review by Research Ethics Committees (REC) could help protect the interests of human participants and promote scientific progress. To facilitate such evaluations, we conducted an ethnographic study of how an REC reviews research proposals during its meetings. We observed 13 meetings of a Dutch REC and studied REC documents. We coded this material inductively and categorised these codes in two repertoires of evaluation: a repertoire of rules and a repertoire of production. In the repertoire of rules the REC applies rules, weighs scientific value and burdens to the participants and makes a final judgment on a research proposal in a meeting. In the repertoire of production, REC members check documents and forms and advise researchers on how to improve their proposals and can use informal communication. Based on these findings, we think that evaluations of the practice of ethical review should take into account the fact that RECs can use a repertoire of rules and a repertoire of production to evaluate research proposals. Combining these two repertoires can be a viable option so that the REC gives researchers advice on how to improve their proposals to prevent rejection of valuable research.

Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments

In 2006, Peters et al. identified 86 systematic reviews (SRs) of laboratory animal experiments (LAEs). They found 46 LAE meta-analyses (MAs), often of poor quality. Six of these 46 MAs tried to assess publication bias. Publication bias is the phenomenon of an experiment's results determining its likelihood of publication, often over-representing positive findings. As such, publication bias is the Achilles heel of any SR. Since researchers increasingly become aware of the fact that SRs directly support the 'three Rs', we expect the number of SRs of LAEs will sharply increase. Therefore, it is useful to see how publication bias is dealt with. Our objective was to identify all SRs and MAs of LAEs where the purpose was to inform human health published between July 2005 and 2010 with special attention to MAs' quality features and publication bias. We systematically searched Medline, Embase, Toxline and ScienceDirect from July 2005 to 2010, updating Peters' review. LAEs not directly informing human health or concerning fundamental biology were excluded. We found 2780 references of which 163 met the inclusion criteria: 158 SRs, of which 30 performed an MA, and five MAs without an SR. The number of SRs roughly doubled every three years since 1997. The number of MAs roughly doubled every five years since 1999. Compared with before July 2005, more MAs were preceded by SR and reported on (quality) features of included studies and heterogeneity. A statistically significant proportion of MAs considered publication bias (26/35) and tried to formally assess it (21/35).