Patient safety requires a new way to publish clinical trials

Marketing trials, marketing tricks - how to spot them and how to stop them

Background

Last this year in this journal, Barbour and colleagues reported a study of “marketing trials” in leading medical journals (Trials 2016;17:31). In this commentary I discuss their research, describe new analyses of the study cohort and consider measures to address marketing within academic medical literature.

Discussion

Barbour et al. sought to identify a subgroup of “marketing trials” within leading medical journals, but in reality, nearly all industry-financed trials serve marketing functions, and many exhibit marketing-related features, including biases, in their framing, methodology or reporting. I conducted new analyses of the cohort of Barbour et al., showing that most trials funded exclusively by drug manufacturers had direct involvement of the manufacturer in design, analysis and reporting, and features supportive of product seeding. However, these commercial enterprises were without exception presented to journal readers as academic-led projects, using attributional spin, which should itself be considered an important form of marketing bias. Barbour et al. correctly conclude that commercial bias in industry clinical trials articles often requires expertise to recognize, and in many cases cannot be identified from the published journal report. Several potential remedies are discussed, including independent clinical research, data sharing, improved reporting guidance, improved tools for assessing research quality, reforms to article attribution, submission checklists and new editorial standards.

Conclusion

Medicine’s journals have a responsibility to uphold rigorous scientific and reporting standards, require ready trials data access and ensure the commercial dimensions of research are brought prominently to their readers’ attention. Failure to meet these responsibilities constitutes an enduring threat to the integrity of biomedical literature.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-1827-5) contains supplementary material, which is available to authorized users.

Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-Analysis of US National Institutes of Health-Funded Trials

BACKGROUND

The efficacy of antidepressant medication has been shown empirically to be overestimated due to publication bias, but this has only been inferred statistically with regard to psychological treatment for depression. We assessed directly the extent of study publication bias in trials examining the efficacy of psychological treatment for depression.

METHODS AND FINDINGS

We identified US National Institutes of Health grants awarded to fund randomized clinical trials comparing psychological treatment to control conditions or other treatments in patients diagnosed with major depressive disorder for the period 1972-2008, and we determined whether those grants led to publications. For studies that were not published, data were requested from investigators and included in the meta-analyses. Thirteen (23.6%) of the 55 funded grants that began trials did not result in publications, and two others never started. Among comparisons to control conditions, adding unpublished studies (Hedges' g = 0.20; CI95% -0.11~0.51; k = 6) to published studies (g = 0.52; 0.37~0.68; k = 20) reduced the psychotherapy effect size point estimate (g = 0.39; 0.08~0.70) by 25%. Moreover, these findings may overestimate the "true" effect of psychological treatment for depression as outcome reporting bias could not be examined quantitatively.

CONCLUSION

The efficacy of psychological interventions for depression has been overestimated in the published literature, just as it has been for pharmacotherapy. Both are efficacious but not to the extent that the published literature would suggest. Funding agencies and journals should archive both original protocols and raw data from treatment trials to allow the detection and correction of outcome reporting bias. Clinicians, guidelines developers, and decision makers should be aware that the published literature overestimates the effects of the predominant treatments for depression.

Trust, Respect, and Reciprocity: Informing Culturally Appropriate Data-Sharing Practice in Vietnam

International science funders and publishers are driving a growing trend in data sharing. There is mounting pressure on researchers in low- and middle-income settings to conform to new sharing policies, despite minimal empirically grounded accounts of the ethical challenges of implementing the policies in these settings. This study used in-depth interviews and focus group discussions with 48 stakeholders in Vietnam to explore the experiences, attitudes, and expectations that inform ethical and effective approaches to sharing clinical research data. Distinct views on the role of trust, respect, and reciprocity were among those that emerged to inform culturally appropriate best practices. We conclude by discussing the challenges that authors of data-sharing policies should consider in this unique context.

Do simple screening statistical tools help to detect reporting bias?

BACKGROUND

As a result of reporting bias, or frauds, false or misunderstood findings may represent the majority of published research claims. This article provides simple methods that might help to appraise the quality of the reporting of randomized, controlled trials (RCT).

METHODS

This evaluation roadmap proposed herein relies on four steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported p values; data simulation using parametric bootstrap and explicit computation of the p values. Such an approach was illustrated using published data from a retracted RCT comparing a hydroxyethyl starch versus albumin-based priming for cardiopulmonary bypass.

RESULTS

Despite obvious nonnormal distributions, several variables are presented as if they were normally distributed. The set of 16 p values testing for differences in baseline characteristics across randomized groups did not follow a Uniform distribution on [0,1] (p = 0.045). The p values obtained by explicit computations were different from the results reported by the authors for the two following variables: urine output at 5 hours (calculated p value < 10-6, reported p ≥ 0.05); packed red blood cells (PRBC) during surgery (calculated p value = 0.08; reported p < 0.05). Finally, parametric bootstrap found p value > 0.05 in only 5 of the 10,000 simulated datasets concerning urine output 5 hours after surgery. Concerning PRBC transfused during surgery, parametric bootstrap showed that only the corresponding p value had less than a 50% chance to be inferior to 0.05 (3,920/10,000, p value < 0.05).

CONCLUSIONS

Such simple evaluation methods might offer some warning signals. However, it should be emphasized that such methods do not allow concluding to the presence of error or fraud but should rather be used to justify asking for an access to the raw data.

Preparing raw clinical data for publication: guidance for journal editors, authors, and peer reviewers

In recognition of the benefits of transparent reporting, many peer-reviewed journals require that their authors be prepared to share their raw, unprocessed data with other scientists and/or state the availability of raw data in published articles. But little information on how data should be prepared for publication - or sharing - has emerged. In clinical research patient privacy and consent for use of personal health information are key considerations, but agreed-upon definitions of what constitutes anonymised patient information do not appear to have been established. We aim to address this issue by providing practical guidance for those involved in the publication process, by proposing a minimum standard for de-identifying datasets for the purposes of publication in a peer-reviewed biomedical journal, or sharing with other researchers. Basic advice on file preparation is provided along with procedural guidance on prospective and retrospective publication of raw data, with an emphasis on randomised controlled trials.

In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal web sites.

Preparing raw clinical data for publication: guidance for journal editors, authors, and peer reviewers

Many peer reviewed journals now require authors to be prepared to share their raw, unprocessed data with other scientists or state the availability of raw data in published articles, but little information on how such data should be prepared for sharing has emerged. Iain Hrynaszkiewicz and colleagues propose a minimum standard for de-identifying datasets to ensure patient privacy when sharing clinical research data

A systematic review of the reporting of Data Monitoring Committees' roles, interim analysis and early termination in pediatric clinical trials

Background

Decisions about interim analysis and early stopping of clinical trials, as based on recommendations of Data Monitoring Committees (DMCs), have far reaching consequences for the scientific validity and clinical impact of a trial. Our aim was to evaluate the frequency and quality of the reporting on DMC composition and roles, interim analysis and early termination in pediatric trials.

Methods

We conducted a systematic review of randomized controlled clinical trials published from 2005 to 2007 in a sample of four general and four pediatric journals. We used full-text databases to identify trials which reported on DMCs, interim analysis or early termination, and included children or adolescents. Information was extracted on general trial characteristics, risk of bias, and a set of parameters regarding DMC composition and roles, interim analysis and early termination.

Results

110 of the 648 pediatric trials in this sample (17%) reported on DMC or interim analysis or early stopping, and were included; 68 from general and 42 from pediatric journals. The presence of DMCs was reported in 89 of the 110 included trials (81%); 62 papers, including 46 of the 89 that reported on DMCs (52%), also presented information about interim analysis. No paper adequately reported all DMC parameters, and nine (15%) reported all interim analysis details. Of 32 trials which terminated early, 22 (69%) did not report predefined stopping guidelines and 15 (47%) did not provide information on statistical monitoring methods.

Conclusions

Reporting on DMC composition and roles, on interim analysis results and on early termination of pediatric trials is incomplete and heterogeneous. We propose a minimal set of reporting parameters that will allow the reader to assess the validity of trial results.

Towards agreement on best practice for publishing raw clinical trial data

Many research-funding agencies now require open access to the results of research they have funded, and some also require that researchers make available the raw data generated from that research. Similarly, the journal Trials aims to address inadequate reporting in randomised controlled trials, and in order to fulfil this objective, the journal is working with the scientific and publishing communities to try to establish best practice for publishing raw data from clinical trials in peer-reviewed biomedical journals. Common issues encountered when considering raw data for publication include patient privacy - unless explicit consent for publication is obtained - and ownership, but agreed-upon policies for tackling these concerns do not appear to be addressed in the guidance or mandates currently established. Potential next steps for journal editors and publishers, ethics committees, research-funding agencies, and researchers are proposed, and alternatives to journal publication, such as restricted access repositories, are outlined.

What should be done to tackle ghostwriting in the medical literature?

BACKGROUND TO THE DEBATE

Ghostwriting occurs when someone makes substantial contributions to a manuscript without attribution or disclosure. It is considered bad publication practice in the medical sciences, and some argue it is scientific misconduct. At its extreme, medical ghostwriting involves pharmaceutical companies hiring professional writers to produce papers promoting their products but hiding those contributions and instead naming academic physicians or scientists as the authors. To improve transparency, many editors' associations and journals allow professional medical writers to contribute to the writing of papers without being listed as authors provided their role is acknowledged. This debate examines how best to tackle ghostwriting in the medical literature from the perspectives of a researcher, an editor, and the professional medical writer.

An update on the diagnosis of adrenal insufficiency and the use of corticotherapy in critical illness

OBJECTIVE

To examine recent literature regarding corticotherapy in critically ill patients suffering from sepsis, acute respiratory distress syndrome (ARDS), and severe community-acquired pneumonia (SCAP).

DATA SOURCES

Literature was identified through MEDLINE (1966-April 2007) using combinations of the key words hydrocortisone, adrenal insufficiency, acute respiratory distress syndrome, pneumonia, sepsis, and cortisol. Bibliographies of relevant articles were reviewed for additional citations. Presentations at recent critical care meetings were also incorporated.

STUDY SELECTION AND DATA EXTRACTION

Articles were chosen based upon their relevance to the topics covered.

DATA SYNTHESIS

Earlier studies using high-dose corticotherapy in the intensive care unit have shown treatment to be ineffective. Recent studies using extended courses of low-to-moderate doses of steroids have found favorable results; however, these results must be interpreted with caution due to limitations in the data. One trial of steroids in septic shock found a survival benefit in patients who failed to increase their baseline cortisol by greater than 9 microg/dL in response to adrenocorticotropic hormone, but these results were not reproduced in a subsequent Phase 3 trial. Recently, inaccuracies in measuring cortisol have been identified, making interpretation of cortisol concentrations difficult. A large-scale study failed to confirm a previously reported mortality benefit of corticotherapy in late ARDS, but preliminary data suggest a role for steroid treatment in early ARDS. Finally, a pilot study has found that hydrocortisone lowers morbidity and mortality in SCAP.

CONCLUSIONS

Corticotherapy may be beneficial to some patients with sepsis. The decision to administer steroids in sepsis cannot be based on biochemical markers of adrenal function; rather, treatment should be considered in septic patients with vasopressor refractory hypotension. Although preliminary evidence suggests a role for steroids in early ARDS and SCAP, there are not enough data to suggest routine administration of steroids in these conditions. Additional studies are needed to assess corticotherapy in the critically ill.

Statins, fibrates, and melanoma risk: a systematic review and meta-analysis

BACKGROUND

Large randomized, controlled clinical trials of lovastatin and gemfibrozil for heart disease prevention have reported statistically significantly lower melanoma incidences in persons receiving these medications. Results of in vitro animal model and human case-control studies also suggest that statins and fibrates may reduce the risk of melanoma.

METHODS

We performed a systematic review of trials that randomly assigned participants to receive statins or fibrates versus an alternative therapy for a minimum of 6 months. Trials were identified by searching five electronic databases and the reference lists of eligible publications. Unpublished data were solicited from trial investigators and pharmaceutical companies. A meta-analysis was performed using a fixed-effects model, and odds ratios (ORs) with 95% confidence intervals (CIs) were calculated to estimate pooled treatment effects. All statistical tests were two-sided.

RESULTS

We obtained data on incident melanomas from 20 of 36 qualifying randomized controlled trials (12 statin trials and eight fibrate trials), with a total of 70,820 participants. A total of 127 melanomas occurred among the 39,426 participants in the statin trials (59 among the 19,872 statin group participants and 68 among the 19,554 control group participants). A total of 27 melanomas occurred among the 31,394 participants enrolled in the fibrate trials (seven among the 12,324 fibrate group participants and 20 among the 19,070 control group participants). Overall, incidence of melanoma was not statistically significantly associated with the use of either statins (OR = 0.87, 95% CI = 0.61 to 1.23) or fibrates (OR = 0.45, 95% CI = 0.20 to 1.01). In a subgroup analysis by drug, only lovastatin use (in one trial) was statistically significantly associated with lower incidence of melanoma (OR = 0.52, 95% CI = 0.27 to 0.99).

CONCLUSIONS

These findings do not validate the possibility that statins or fibrates prevent melanoma.

Evolution and translation of research findings: from bench to where?

The credibility and replication of research findings evolve over time, as data accumulate. However, translation of postulated research promises to real-life biomedical applications is uncommon. In some fields of research, we may observe diminishing effects for the strength of research findings and rapid alternations of exaggerated claims and extreme contradictions--the "Proteus Phenomenon." While these phenomena are probably more prominent in the basic sciences, similar manifestations have been documented even in clinical trials and they may undermine the credibility of clinical research. Significance-chasing bias may be in part responsible, but the greatest threat may come from the poor relevance and scientific rationale and thus low pre-study odds of success of research efforts. Given that we currently have too many research findings, often with low credibility, replication and rigorous evaluation become as important as or even more important than discovery. Credibility, replication, and translation are all desirable properties of research findings, but are only modestly correlated. In this essay, I discuss some of the evidence (or lack thereof) for the process of evolution and translation of research findings, with emphasis on the biomedical sciences.