Conflicts of Interest, Selective Inertia, and Research Malpractice in Randomized Clinical Trials: An Unholy Trinity

Methodological Aspects in Studies Based on Clinical Routine Data

Randomized controlled clinical trials are regarded as the gold standard for comparing different clinical interventions, but generally their conduct is operationally cumbersome, time-consuming, and expensive. Studies and investigations based on clinical routine data on the contrary utilize existing data acquired under real-life conditions and are increasingly popular among practitioners. In this paper, methodological aspects of studies based on clinical routine data are discussed. Important limitations and considerations as well as unique strengths of these types of studies are indicated and exemplarily demonstrated in a recent real-case study based on clinical routine data. In addition two simulation studies reveal the impact of bias in studies based on clinical routine data on the type I error rate and false decision rate in favor of the inferior intervention. It is concluded that correctly analyzing clinical routine data yields a valuable addition to clinical research; however, as a result of a lack of statistical foundation, internal validity, and comparability, generalizing results and inferring properties derived from clinical routine data to all patients of interest has to be considered with extreme caution.

FUNDING

Grünenthal GmbH.

Why statistical inference from clinical trials is likely to generate false and irreproducible results

One area of biomedical research where the replication crisis is most visible and consequential is clinical trials. Why do outcomes of so many clinical trials contradict each other? Why is the effectiveness of many drugs and other medical interventions so low? Why have prescription medications become the third leading cause of death in the US and Europe after cardiovascular diseases and cancer? In answering these questions, the main culprits identified so far have been various biases and conflicts of interest in planning, execution and analysis of clinical trials as well as reporting their outcomes. In this work, we take an in-depth look at statistical methodology used in planning clinical trials and analyzing trial data. We argue that this methodology is based on various questionable and empirically untestable assumptions, dubious approximations and arbitrary thresholds, and that it is deficient in many other respects. The most objectionable among these assumptions is that of distributional homogeneity of subjects' responses to medical interventions. We analyze this and other assumptions both theoretically and through clinical examples. Our main conclusion is that even a totally unbiased, perfectly randomized, reliably blinded, and faithfully executed clinical trial may still generate false and irreproducible results. We also formulate a few recommendations for the improvement of the design and statistical methodology of clinical trials informed by our analysis.

Self-declared stock ownership and association with positive trial outcome in randomized controlled trials with binary outcomes published in general medical journals: a cross-sectional study

Background

Describe the prevalence and types of conflicts of interest (COI) in published randomized controlled trials (RCTs) in general medical journals with a binary primary outcome and assess the association between conflicts of interest and favorable outcome.

Methods

Parallel-group RCTs with a binary primary outcome published in three general medical journals during 2013–2015 were identified. COI type, funding source, and outcome were extracted. Binomial logistic regression model was performed to assess association between COI and funding source with outcome.

Results

A total of 509 consecutive parallel-group RCTs were included in the study. COI was reported in 74% in mixed funded RCTs and in 99% in for-profit funded RCTs. Stock ownership was reported in none of the non-profit RCTs, in 7% of mixed funded RCTs, and in 50% of for-profit funded RCTs. Mixed-funded RCTs had employees from the funding company in 11% and for-profit RCTs in 76%. Multivariable logistic regression revealed that stock ownership in the funding company among any of the authors was associated with a favorable outcome (odds ratio = 3.53; 95% confidence interval = 1.59–7.86; p < 0.01).

Conclusion

COI in for-profit funded RCTs is extensive, because the factors related to COI are not fully independent, a multivariable analysis should be cautiously interpreted. However, after multivariable adjustment only stock ownership from the funding company among authors is associated with a favorable outcome.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-2108-z) contains supplementary material, which is available to authorized users.

Aware, Yet Ignorant: Exploring the Views of Early Career Researchers About Funding and Conflicts of Interests in Science

This study investigates the level of awareness about funding influences and potential conflicts of interests (COI) among early career researchers. The sample for this study included users of one or more of the 14 U.S. laboratories associated with the National Nanotechnology Infrastructure Network. To be eligible, respondents must have been either still completing graduate work or <5 years since graduation. In total, 713 early career researchers completed the web survey, with about half still in graduate school. Results indicate that although respondents were aware of potential funding and COI influences on their work, they remained largely ignorant of their role in addressing or managing these issues. Respondents often attributed the responsibility of addressing these issues to their supervisors. Respondents who had received some training around these issues, however, were more likely to assume more personal responsibility. Overall, this study points out that ignorance among early career researchers is less about awareness of funding and COI issues and more about taking personal responsibility for addressing these issues.

Risk of selection bias in randomized trials: further insight

The quality of randomization is an under-appreciated facet of trial design. The present piece represents an advance in our collective understanding of how allocation concealment and randomization relate to risk of selection bias in randomized trials, and other measures are also considered. Though the overwhelming majority of the advice given is timely and correct, it is more instructive to focus on the relatively narrow sliver of advice that is incorrect (namely, that trials should not stratify by site, and that unrestricted randomization is a solution to the problem of selection bias), so it is in this context that the comments here must be understood. In no way is this intended to be a rebuttal of the excellent work we have before us. Rather, it is a refinement.

Characterizing permuted block randomization as a big stick procedure

There are numerous approaches to randomizing patients to treatment groups in clinical trials. The most popular is permuted block randomization, and a newer and better class, which is gaining in popularity, is the so-called class of MTI procedures, which use a big stick to force the allocation sequence back towards balance when it reaches the MTI (maximally tolerated imbalance). Three prominent members of this class are the aptly named big stick procedure, Chen's procedure, and the maximal procedure. As we shall establish in this article, blocked randomization, though not typically cast as an MTI procedure, does in fact use the big stick as well. We shall argue that its weaknesses, which are well known, arise precisely from its improper use, bordering on outright abuse, of this big stick. Just as rocket powered golf clubs add power to a golf swing, so too does the big stick used by blocked randomization hit with too much power. In addition, the big stick is invoked when it need not be, thereby resulting in the excessive prediction for which permuted blocks are legendary. We bridge the gap between the MTI procedures and block randomization by identifying a new randomization procedure intermediate between the two, namely based on an excessively powerful big stick, but one that is used only when needed. We shall then argue that the MTI procedures are all superior to this intermediate procedure by virtue of using a restrained big stick, and that this intermediate procedure is superior to block randomization by virtue of restraint in when the big stick is invoked. The transitivity property then completes our argument.