Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study

Dissemination of 2014 dual antiplatelet therapy (DAPT) trial results: a systematic review of scholarly and media attention over 7 months

OBJECTIVE

To explore how the results from the 2014 dual antiplatelet therapy (DAPT) trial were disseminated to the scientific community and online media.

DESIGN

A a systematic review of scholarly and public attention surrounding the DAPT study.

SETTINGS

Data were collected from the ISI Web of Knowledge, Google Scholar, PubMed Commons, EurekAlert, the DAPT study website (www.daptstudy.org) and the New England Journal of Medicine website (for scholarly attention) and Altmetric Explorer, Snap Bird, YouTube (for public attention) citing DAPT study results appearing from 16 November 2014 to 10 June 2015.

PARTICIPANTS

No participants were involved in this study.

MAIN OUTCOME MEASURE

Proportion of contents highlighting the increased risk of mortality and critical to the author's interpretation of the results.

RESULTS

We identified 425 items reported by seven sources; 164 (39%) disseminated the authors' interpretation via an electronic link or a reference, with no additional text. Among 81 items (19 %), the message favoured prolonged treatment and consequently overstated the article conclusions. Among 119 items (28 %), the text was uncertain about the benefit of prolonged treatment but was reported with no or inappropriate mention of increased risk of mortality. Only 34 items (8 %) were uncertain about the benefit of prolonged treatment and mentioned increased risk of mortality. In all, 27 items (6 %) did not favour prolonged treatment, and only 12 of these (3 %) clearly raised some concerns about the reporting of increased risk of death.

CONCLUSION

Dissemination of the DAPT study results to the scientific community and on different media sources rarely criticised the interpretation of the study results.

Sugar industry sponsorship of germ-free rodent studies linking sucrose to hyperlipidemia and cancer: An historical analysis of internal documents

In 1965, the Sugar Research Foundation (SRF) secretly funded a review in the New England Journal of Medicine that discounted evidence linking sucrose consumption to blood lipid levels and hence coronary heart disease (CHD). SRF subsequently funded animal research to evaluate sucrose's CHD risks. The objective of this study was to examine the planning, funding, and internal evaluation of an SRF-funded research project titled "Project 259: Dietary Carbohydrate and Blood Lipids in Germ-Free Rats," led by Dr. W.F.R. Pover at the University of Birmingham, Birmingham, United Kingdom, between 1967 and 1971. A narrative case study method was used to assess SRF Project 259 from 1967 to 1971 based on sugar industry internal documents. Project 259 found a statistically significant decrease in serum triglycerides in germ-free rats fed a high sugar diet compared to conventional rats fed a basic PRM diet (a pelleted diet containing cereal meals, soybean meals, whitefish meal, and dried yeast, fortified with a balanced vitamin supplement and trace element mixture). The results suggested to SRF that gut microbiota have a causal role in carbohydrate-induced hypertriglyceridemia. A study comparing conventional rats fed a high-sugar diet to those fed a high-starch diet suggested that sucrose consumption might be associated with elevated levels of beta-glucuronidase, an enzyme previously associated with bladder cancer in humans. SRF terminated Project 259 without publishing the results. The sugar industry did not disclose evidence of harm from animal studies that would have (1) strengthened the case that the CHD risk of sucrose is greater than starch and (2) caused sucrose to be scrutinized as a potential carcinogen. The influence of the gut microbiota in the differential effects of sucrose and starch on blood lipids, as well as the influence of carbohydrate quality on beta-glucuronidase and cancer activity, deserve further scrutiny.

Is Article Methodological Quality Associated With Conflicts of Interest?: An Analysis of the Plastic Surgery Literature

PURPOSE

Conflicts of interest (COI) are an emerging area of discussion within the field of plastic surgery. Recently, several reports have found that research studies that disclose COI are associated with publication of positive outcomes. We hypothesize that this association is driven by higher-quality studies receiving industry funding. This study aimed to investigate the association between industry support and study methodological quality.

METHODS

We reviewed all entries in Plastic and Reconstructive Surgery, Annals of Plastic Surgery, and Journal of Plastic, Reconstructive, and Aesthetic Surgery within a 1-year period encompassing 2013. All clinical research articles were analyzed. Studies were evaluated blindly for methodology quality based on a validated scoring system. An ordinal logistic regression model was used to examine the association between methodology score and COI.

RESULTS

A total of 1474 articles were reviewed, of which 483 met our inclusion criteria. These articles underwent methodological quality scoring. Conflicts of interest were reported in 28 (5.8%) of these articles. After adjusting for article characteristics in the ordinal logistic regression analysis, there was no significant association between articles with COI and higher methodological scores (P = 0.7636).

CONCLUSIONS

Plastic surgery studies that disclose COI are not associated with higher methodological quality when compared with studies that do not disclose COI. These findings suggest that although the presence of COI is associated with positive findings, the association is not shown to be driven by higher-quality studies.

'Spin' in published biomedical literature: A methodological systematic review

In the scientific literature, spin refers to reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favourable light. The presence of spin in biomedical research can negatively impact the development of further studies, clinical practice, and health policies. This systematic review aims to explore the nature and prevalence of spin in the biomedical literature. We searched MEDLINE, PreMEDLINE, Embase, Scopus, and hand searched reference lists for all reports that included the measurement of spin in the biomedical literature for at least 1 outcome. Two independent coders extracted data on the characteristics of reports and their included studies and all spin-related outcomes. Results were grouped inductively into themes by spin-related outcome and are presented as a narrative synthesis. We used meta-analyses to analyse the association of spin with industry sponsorship of research. We included 35 reports, which investigated spin in clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. The nature of spin varied according to study design. The highest (but also greatest) variability in the prevalence of spin was present in trials. Some of the common practices used to spin results included detracting from statistically nonsignificant results and inappropriately using causal language. Source of funding was hypothesised by a few authors to be a factor associated with spin; however, results were inconclusive, possibly due to the heterogeneity of the included papers. Further research is needed to assess the impact of spin on readers’ decision-making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research in order to ensure accurate interpretation and dissemination of research.

In the scientific literature, spin refers to reporting practices that distort the interpretation of results and mislead readers so that results are viewed in a more favourable light. The presence of spin in biomedical research can negatively impact the development of further studies, clinical practice, and health policies. We conducted a systematic review to explore the nature and prevalence of spin in the biomedical literature. We included 35 reports, which investigated spin in clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. The nature of spin varied according to study design. The highest (but also greatest) variability in the prevalence of spin was present in trials. Some of the common practices used to spin results included detracting from statistically nonsignificant results and inappropriately using causal language. Source of funding was hypothesised by a few authors to be a factor associated with spin; however, results were inconclusive, possibly due to the heterogeneity of the included papers. Further research is needed to assess the impact of spin on readers’ decision-making. Editors and peer reviewers should be familiar with the prevalence and manifestations of spin in their area of research in order to ensure accurate interpretation and dissemination of research.

Factors associated with online media attention to research: a cohort study of articles evaluating cancer treatments

Background

New metrics have been developed to assess the impact of research and provide an indication of online media attention and data dissemination. We aimed to describe online media attention of articles evaluating cancer treatments and identify the factors associated with high online media attention.

Methods

We systematically searched MEDLINE via PubMed on March 1, 2015 for articles published during the first 6 months of 2014 in oncology and medical journals with a diverse range of impact factors, from 3.9 to 54.4, and selected a sample of articles evaluating a cancer treatment regardless of study design. Altmetric Explorer was used to identify online media attention of selected articles. The primary outcome was media attention an article received online as measured by Altmetric score (i.e., number of mentions in online news outlets, science blogs and social media). Regression analysis was performed to investigate the factors associated with high media attention, and regression coefficients represent the logarithm of ratio of mean (RoM) values of Altmetric score per unit change in the covariate.

Results

Among 792 articles, 218 (27.5%) received no online media attention (Altmetric score = 0). The median [Q1–Q3] Altmetric score was 2.0 [0.0–8.0], range 0.0–428.0. On multivariate analysis, factors associated with high Altmetric score were presence of a press release (RoM = 10.14, 95%CI [4.91–20.96]), open access to the article (RoM = 1.48, 95%CI [1.02–2.16]), and journal impact factor (RoM = 1.10, 95%CI [1.07–1.12]. As compared with observational studies, systematic reviews were not associated with high Altmetric score (RoM = 1.46, 95%CI [0.74–2.86]; P = 0.27), nor were RCTs (RoM = 0.65, 95%CI [0.41–1.02]; P = 0.059) and phase I/II non-RCTs (RoM = 0.58, 95%CI [0.33–1.05]; P = 0.07). The articles with abstract conclusions favouring study treatments were not associated with high Altmetric score (RoM = 0.97, 95%CI [0.60–1.58]; P = 0.91).

Conclusions

Most important factors associated with high online media attention were the presence of a press release and the journal impact factor. There was no evidence that study design with high level of evidence and type of abstract conclusion were associated with high online media attention.

Electronic supplementary material

The online version of this article (doi:10.1186/s41073-017-0033-z) contains supplementary material, which is available to authorized users.

Conclusions in systematic reviews of mammography for breast cancer screening and associations with review design and author characteristics

BACKGROUND

Debates about the benefits and harms of mammography continue despite the accumulation of evidence. We sought to quantify the disagreement across systematic reviews of mammography and determine whether author or design characteristics were associated with conclusions that were favourable to the use of mammography for routine breast cancer screening.

METHODS

We identified systematic reviews of mammography published between January 2000 and November 2015, and extracted information about the selection of evidence, age groups, the use of meta-analysis, and authors' professions and financial competing interest disclosures. Conclusions about specific age groups were graded as favourable if they stated that there were meaningful benefits, that benefits of mammography outweighed harms, or that harms were inconsequential. The main outcome measures were the proportions of favourable conclusions relative to review design and author characteristics.

RESULTS

From 59 conclusions identified in 50 reviews, 42% (25/59) were graded as favourable by two investigators. Among the conclusions produced by clinicians, 63% (12/19) were graded as favourable compared to 32% (13/40) from other authors. In the 50-69 age group where the largest proportion of systematic reviews were focused, conclusions drawn by authors without financial competing interests (odds ratio 0.06; 95% CI 0.07-0.56) and non-clinicians (odds ratio 0.11; 95% CI 0.01-0.84) were less likely to be graded as favourable. There was no trend in the proportion of favourable conclusions over the period, and we found no significant association between review design characteristics and favourable conclusions.

CONCLUSIONS

Differences in the conclusions of systematic reviews of the evidence for mammography have persisted for 15 years. We found no strong evidence that design characteristics were associated with greater support for the benefits of mammography in routine breast cancer screening. Instead, the results suggested that the specific expertise and competing interests of the authors influenced the conclusions of systematic reviews.

Technology Transfer: From the Research Bench to Commercialization: Part 2: The Commercialization Process

Technology transfer (TT) encompasses a variety of activities that move academic discoveries into the public sector. Part 1 of this 2-part series explored steps in acquisition of intellectual property (IP) rights (e.g., patents and copyrights). Part 2 focuses on processes of commercialization, including the technology transfer office, project development toward commercialization, and licensing either through the establishment of startup companies (venture capital-backed or otherwise) or directly to industry. In private industry, TT often occurs through the sale of IP, products, or services, but in universities, the majority of TT occurs through the licensing of IP.

Citation Rate Predictors in the Plastic Surgery Literature

BACKGROUND

The purpose of this study is to determine and characterize the scientific and nonscientific factors that influence the rate of article citation in the field of plastic surgery.

DESIGN

Cross-sectional study.

SETTING

We reviewed all entries in Annals of Plastic Surgery and Journal of Plastic, Reconstructive, and Aesthetic Surgery from January 1, 2007 to December 31, 2007; and Plastic and Reconstructive Surgery from January 1, 2007 to December 31, 2008. All scientific articles were analyzed and several article characteristics were extracted. The number of citations at 5 years was collected as the outcome variable. A multivariable analysis was performed to determine which variables were associated with higher citations rates.

RESULTS

A total of 2456 articles were identified of which only 908 fulfilled the inclusion criteria. Most studies were publications in the fields of reconstructive (26.3%) or pediatric/craniofacial (17.6%) surgery. The median number of citations 5 years from publication was 8. In the multivariable analysis, factors associated with higher citations rates were subspecialty field (p = 0.0003), disclosed conflict of interest (p = 0.04), number of authors (p = 0.04), and journal (p = 0.02).

CONCLUSION

We have found that higher level of evidence (or other study methodology factors) is not associated with higher citation rates. Instead, conflict of interest, subspecialty topic, journal, and number of authors are strong predictors of high citation rates in plastic surgery.

Systematic Review: A Method at Risk for Being Corrupted

The production of systematic reviews is increasing, but their credibility is under threat. Although systematic reviews are an important tool for policymaking, their influence can be weakened by methodological problems and poor policy relevance. Using Cochrane as an example, I address standards for systematic reviews, the influence of special interests on these reviews, and ways to increase their relevance for policymakers.

Evidence and evidence gaps in tinnitus therapy

A nearly endless number of procedures has been tried and in particular sold for the treatment of tinnitus, unfortunately they have not been evaluated appropriately in an evidence-based way. A causal therapy, omitting the tinnitus still does not exist, actually it cannot exist because of the various mechanisms of its origin. However or perhaps because of that, medical interventions appear and reappear like fashion trends that can never be proven by stable and reliable treatment success. This contribution will discuss and acknowledge all current therapeutic procedures and the existing or non-existing evidence will be assessed. Beside external evidence, the term of evidence also encompasses the internal evidence, i.e. the experience of the treating physician and the patient's needs shall be included. While there is no evidence for nearly all direct procedures that intend modulating or stimulating either the cochlea or specific cervical regions such as the auditory cortex, there are therapeutic procedures that are acknowledged in clinical practice and have achieved at least a certain degree of evidence and generate measurable effect sizes. Those are in particular habituation therapy and psychotherapeutic measures, especially if they are combined with concrete measures for improved audio perception (hearing aids, CI, hearing therapies).

Epidemiological characteristics and methodological quality of meta-analyses on diabetes mellitus treatment: a systematic review

OBJECTIVE

Well-conducted meta-analyses (MAs) are considered as one of the best sources of clinical evidence for treatment decision. MA with methodological flaws may introduce bias and mislead evidence users. The aim of this study is to investigate the characteristics and methodological quality of MAs on diabetes mellitus (DM) treatments.

DESIGN

Systematic review.

METHODS

Cochrane Database of Systematic Review and Database of Abstract of Reviews of Effects were searched for relevant MAs. Assessing methodological quality of systematic reviews (AMSTAR) tool was used to evaluate the methodological quality of included MAs. Logistic regression analysis was used to identify association between characteristics of MA and AMSTAR results.

RESULTS

A total of 252 MAs including 4999 primary studies and 13,577,025 patients were included. Over half of the MAs (65.1%) only included type 2 DM patients and 160 MAs (63.5%) focused on pharmacological treatments. About 89.7% MAs performed comprehensive literature search and 89.3% provided characteristics of included studies. Included MAs generally had poor performance on the remaining AMSTAR items, especially in assessing publication bias (39.3%), providing lists of studies (19.0%) and declaring source of support comprehensively (7.5%). Only 62.7% MAs mentioned about harm of interventions. MAs with corresponding author from Asia performed less well in providing MA protocol than those from Europe.

CONCLUSIONS

Methodological quality of MA on DM treatments was unsatisfactory. There is considerable room for improvement, especially in assessing publication bias, providing lists of studies and declaring source of support comprehensively. Also, there is an urgent need for MA authors to report treatment harm comprehensively.

Relationship between Research Outcomes and Risk of Bias, Study Sponsorship, and Author Financial Conflicts of Interest in Reviews of the Effects of Artificially Sweetened Beverages on Weight Outcomes: A Systematic Review of Reviews

BACKGROUND

Artificially sweetened beverage consumption has steadily increased in the last 40 years. Several reviews examining the effects of artificially sweetened beverages on weight outcomes have discrepancies in their results and conclusions.

OBJECTIVES

To determine whether risk of bias, results, and conclusions of reviews of effects of artificially sweetened beverage consumption on weight outcomes differ depending on review sponsorship and authors' financial conflicts of interest.

METHODS

We performed a systematic review of reviews of the effects of artificially sweetened beverages on weight. Two assessors independently screened articles for inclusion, extracted data, and assessed risks of bias. We compared risk of bias, results and conclusions of reviews by different industry sponsors, authors' financial conflict of interest and journal sponsor. We also report the concordance between review results and conclusions.

RESULTS

Artificial sweetener industry sponsored reviews were more likely to have favorable results (3/4) than non-industry sponsored reviews (1/23), RR: 17.25 (95% CI: 2.34 to 127.29), as well as favorable conclusions (4/4 vs. 15/23), RR: 1.52 (95% CI: 1.14 to 2.06). All reviews funded by competitor industries reported unfavorable conclusions (4/4). In 42% of the reviews (13/31), authors' financial conflicts of interest were not disclosed. Reviews performed by authors that had a financial conflict of interest with the food industry (disclosed in the article or not) were more likely to have favorable conclusions (18/22) than reviews performed by authors without conflicts of interest (4/9), RR: 7.36 (95% CI: 1.15 to 47.22). Risk of bias was similar and high in most of the reviews.

CONCLUSIONS

Review sponsorship and authors' financial conflicts of interest introduced bias affecting the outcomes of reviews of artificially sweetened beverage effects on weight that could not be explained by other sources of bias.

Association Between Financial Conflicts of Interests and Supportive Opinions for Erectile Dysfunction Treatment

A conflict of interest (COI) is a situation in which a person has competing loyalties or interests that make it difficult to fulfil his or her duties impartially. Conflict of interest is not categorically improper in itself but requires proper management. A SCOPUS literature search was performed for publications on the efficacy/safety of Phospho-Di-Esterase Inhibitors (PDEIs) for treating erectile dysfunction. A categorization tool (CoOpCaT) was used to review and classify the publications as supportive/not-supportive for the discussed active ingredient and reporting or not reporting a COI for that specific drug or for the remaining PDEIs (i.e. competitors). Multivariable binary logistic regression was performed. In the 419 selected records the prevalence of supportive opinions was higher when a COI for the index label was declared. The CoOpCaT showed good internal consistency, discriminative validity and intra/inter-rater agreement. The strongest predictor for a supportive opinion was the total number of financial COIs for the index label. A mild protective effect of the total number of financial COIs for any competitor label was noted. Financial COIs have frequently been associated with bias, and the measures currently adopted to restrain it lack effectiveness. Some evidence for monitoring and/or compensating this bias is reported here, but the ultimate solution remains distant.

The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses

POLICY POINTS

Currently, there is massive production of unnecessary, misleading, and conflicted systematic reviews and meta-analyses. Instead of promoting evidence-based medicine and health care, these instruments often serve mostly as easily produced publishable units or marketing tools. Suboptimal systematic reviews and meta-analyses can be harmful given the major prestige and influence these types of studies have acquired. The publication of systematic reviews and meta-analyses should be realigned to remove biases and vested interests and to integrate them better with the primary production of evidence.

CONTEXT

Currently, most systematic reviews and meta-analyses are done retrospectively with fragmented published information. This article aims to explore the growth of published systematic reviews and meta-analyses and to estimate how often they are redundant, misleading, or serving conflicted interests.

METHODS

Data included information from PubMed surveys and from empirical evaluations of meta-analyses.

FINDINGS

Publication of systematic reviews and meta-analyses has increased rapidly. In the period January 1, 1986, to December 4, 2015, PubMed tags 266,782 items as "systematic reviews" and 58,611 as "meta-analyses." Annual publications between 1991 and 2014 increased 2,728% for systematic reviews and 2,635% for meta-analyses versus only 153% for all PubMed-indexed items. Currently, probably more systematic reviews of trials than new randomized trials are published annually. Most topics addressed by meta-analyses of randomized trials have overlapping, redundant meta-analyses; same-topic meta-analyses may exceed 20 sometimes. Some fields produce massive numbers of meta-analyses; for example, 185 meta-analyses of antidepressants for depression were published between 2007 and 2014. These meta-analyses are often produced either by industry employees or by authors with industry ties and results are aligned with sponsor interests. China has rapidly become the most prolific producer of English-language, PubMed-indexed meta-analyses. The most massive presence of Chinese meta-analyses is on genetic associations (63% of global production in 2014), where almost all results are misleading since they combine fragmented information from mostly abandoned era of candidate genes. Furthermore, many contracting companies working on evidence synthesis receive industry contracts to produce meta-analyses, many of which probably remain unpublished. Many other meta-analyses have serious flaws. Of the remaining, most have weak or insufficient evidence to inform decision making. Few systematic reviews and meta-analyses are both non-misleading and useful.

CONCLUSIONS

The production of systematic reviews and meta-analyses has reached epidemic proportions. Possibly, the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted.

Conflict of interest disclosure in biomedical research: A review of current practices, biases, and the role of public registries in improving transparency

Conflicts of interest held by researchers remain a focus of attention in clinical research. Biases related to these relationships have the potential to directly impact the quality of healthcare by influencing decision-making, yet conflicts of interest remain underreported, inconsistently described, and difficult to access. Initiatives aimed at improving the disclosure of researcher conflicts of interest are still in their infancy but represent a vital reform that must be addressed before potential biases associated with conflicts of interest can be mitigated and trust in the impartiality of clinical evidence restored. In this review, we examine the prevalence of conflicts of interest, evidence of the effects that disclosed and undisclosed conflicts of interest have had on the reporting of clinical evidence, and the emerging approaches for improving the completeness and consistency of disclosures. Through this review of emerging technologies, we recognize a growing interest in publicly accessible registries for researcher conflicts of interest and propose five desiderata aimed at maximizing the value of such registries: mandates for ensuring that researchers keep their records up to date; transparent records that are made available to the public; interoperability to allow researchers, bibliographic databases, and institutions to interact with the registry; a consistent taxonomy for describing different classes of conflicts of interest; and the ability to automatically generate conflicts of interest statements for use in published articles.

Funding of Gambling Research: Ethical Issues, Potential Benefit and Guidelines

There has been an unprecedented growth of legalized gambling opportunities in Canada over the past two decades, partly to generate revenues without raising taxes. Unfortunately, for 2-3% of the Canadian population, gambling can become disordered (i.e. develop into a gambling addiction). To help attenuate the harms and prevalence of disordered gambling, all provincial governments earmark a portion of gambling revenues for the prevention, treatment and research into disordered gambling. However, the field of gambling studies has recently come under criticism in the way research is conducted. At the forefront of the criticism is the issue of accepting funding from the gambling industry. We provide an overview of the ethical considerations, potential ethical issues, and the possible benefits of accepting such funding. The aim of the present paper is not to argue for or against accepting industry funding, but rather to delineate the potential ethical issues and benefits related to that acceptance. More importantly, we provide a summary of best practice ethical guidelines, and recommendations to guide in the ethical decision making process in accepting or declining funding from gambling industry. To this end, we use the Canadian Code of Ethics for Psychologists as a framework in which to situate our guidelines and recommendations. Given that Canadian researchers have a long history and continue to contribute valuable knowledge in the field of gambling studies, it is of important for gambling researchers to be aware of the ethical considerations and issues related to funding from gambling industry.

Methodologic Quality of Systematic Reviews Published in the Plastic and Reconstructive Surgery Literature: A Systematic Review

BACKGROUND

Well-conducted systematic reviews have a critical role in informing evidence-based decision-making in plastic surgery. The authors' objective was to assess the methodologic quality of systematic reviews in the plastic surgery literature.

METHODS

The authors systematically assessed all systematic reviews in 10 high-impact plastic surgery journals using MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials, and the Cochrane Database of Systematic Reviews from 2003 to 2013. These were evaluated for methodologic quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR), a validated 11-point instrument.

RESULTS

After removal of duplicates and screening titles and abstracts, 190 systematic reviews met eligibility criteria. The majority of systematic reviews were published in Plastic and Reconstructive Surgery (n = 88). The most common domain covered was reconstruction (17.9 percent). Using AMSTAR, the median score was 4 (interquartile range, 2.25 to 6.00) on a scale of 1 to 11. No increase in AMSTAR score was observed with time (p = 0.18). Almost half of all systematic reviews (48.4 percent) included at least two independent data extractors, and less than one-third of them (15.3 percent) searched unpublished studies or provided a list of both included and excluded studies (14.8 percent). The methodologic quality of included primary studies was evaluated in 35.3 percent.

CONCLUSIONS

Systematic reviews in plastic surgery demonstrated inadequate adherence to methodologic standards of quality, which raises concerns about validity. There has been an increase in the number of systematic reviews published in plastic surgery over the past decade, yet there has been no significant improvement observed in methodologic quality.

Effectiveness of acupuncture and related therapies for palliative care of cancer: overview of systematic reviews

Acupuncture and related therapies such as moxibustion and transcutaneous electrical nerve stimulation are often used to manage cancer-related symptoms, but their effectiveness and safety are controversial. We conducted this overview to summarise the evidence on acupuncture for palliative care of cancer. Our systematic review synthesised the results from clinical trials of patients with any type of cancer. The methodological quality of the 23 systematic reviews in this overview, assessed using the Methodological Quality of Systematic Reviews Instrument, was found to be satisfactory. There is evidence for the therapeutic effects of acupuncture for the management of cancer-related fatigue, chemotherapy-induced nausea and vomiting and leucopenia in patients with cancer. There is conflicting evidence regarding the treatment of cancer-related pain, hot flashes and hiccups, and improving patients' quality of life. The available evidence is currently insufficient to support or refute the potential of acupuncture and related therapies in the management of xerostomia, dyspnea and lymphedema and in the improvement of psychological well-being. No serious adverse effects were reported in any study. Because acupuncture appears to be relatively safe, it could be considered as a complementary form of palliative care for cancer, especially for clinical problems for which conventional care options are limited.

The Relationship between Risk of Bias Criteria, Research Outcomes, and Study Sponsorship in a Cohort of Preclinical Thiazolidinedione Animal Studies: A Meta-Analysis

Introduction

There is little evidence regarding the influence of conflicts of interest on preclinical research. This study examines whether industry sponsorship is associated with increased risks of bias and/or effect sizes of outcomes in published preclinical thiazolidinedione (TZD) studies.

Methods

We identified preclinical TZD studies published between January 1, 1965, and November 14, 2012. Coders independently extracted information on study design criteria aimed at reducing bias, results for all relevant outcomes, sponsorship source and investigator financial ties from the 112 studies meeting the inclusion criteria. The average standardized mean difference (SMD) across studies was calculated for plasma glucose (efficacy outcome) and weight gain (harm outcome). In subgroup analyses, TZD outcomes were assessed by sponsorship source and risk of bias criteria.

Results

Seven studies were funded by industry alone, 17 studies funded by both industry and non‐industry, 49 studies funded by non‐industry alone and 39 studies had no disclosures. None of the studies used sample size calculations, intention‐to‐treat analyses, blinding of investigators or concealment of allocation. Most studies reported favourable results (88 of 112) and conclusions (95 of 112) supporting TZD use. Efficacy estimates were significantly larger in six studies sponsored by industry alone (−3.41; 95% CI −5.21, −1.53; I² = 93%) versus 42 studies sponsored by non‐industry sources (−0.97; 95% CI −1.37, −0.56; I² = 81%; p‐value = 0.01). Harms estimates were significantly larger in four studies sponsored by industry alone (5.00; 95% CI 1.22, 8.77; I² = 93%) versus 38 studies sponsored by non‐industry sources (0.30; 95% CI −0.08, 0.68; I² = 79%; p‐value = 0.02). TZD efficacy and harms did not differ by disclosure of financial COIs or risks of bias.

Conclusions

Industry‐sponsored TZD animal studies have exaggerated efficacy and harms outcomes compared with studies funded by non‐industry sources. There was poor reporting of COIs.

Methodological quality of meta-analyses on treatments for chronic obstructive pulmonary disease: a cross-sectional study using the AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool

BACKGROUND

Meta-analysis (MA) of randomised trials is considered to be one of the best approaches for summarising high-quality evidence on the efficacy and safety of treatments. However, methodological flaws in MAs can reduce the validity of conclusions, subsequently impairing the quality of decision making.

AIMS

To assess the methodological quality of MAs on COPD treatments.

METHODS

A cross-sectional study on MAs of COPD trials. MAs published during 2000-2013 were sampled from the Cochrane Database of Systematic Reviews and Database of Abstracts of Reviews of Effect. Methodological quality was assessed using the validated AMSTAR (Assessing the Methodological Quality of Systematic Reviews) tool.

RESULTS

Seventy-nine MAs were sampled. Only 18% considered the scientific quality of primary studies when formulating conclusions and 49% used appropriate meta-analytic methods to combine findings. The problems were particularly acute among MAs on pharmacological treatments. In 48% of MAs the authors did not report conflict of interest. Fifty-eight percent reported harmful effects of treatment. Publication bias was not assessed in 65% of MAs, and only 10% had searched non-English databases.

CONCLUSIONS

The methodological quality of the included MAs was disappointing. Consideration of scientific quality when formulating conclusions should be made explicit. Future MAs should improve on reporting conflict of interest and harm, assessment of publication bias, prevention of language bias and use of appropriate meta-analytic methods.

Impact of adding a limitations section to abstracts of systematic reviews on readers' interpretation: a randomized controlled trial

BACKGROUND

To allow an accurate evaluation of abstracts of systematic reviews, the PRISMA Statement recommends that the limitations of the evidence (e.g., risk of bias, publication bias, inconsistency, imprecision) should be described in the abstract. We aimed to evaluate the impact of adding such limitations sections on reader's interpretation.

METHOD

We performed a two-arm parallel group randomized controlled trial (RCT) using a sample of 30 abstracts of systematic reviews evaluating the effects of healthcare intervention with conclusions favoring the beneficial effect of the experimental treatments. Two formats of these abstracts were derived: one reported without and one with a standardized limitations section written according to the PRISMA statement for abstracts. The primary outcome was readers' confidence in the results of the systematic review as stated in the abstract assessed by a Likert scale from 0, not at all confident, to 10, very confident. In total, 300 participants (corresponding authors of RCT reports indexed in PubMed) were randomized by a web-based randomization procedure to interpret one abstract with a limitations section (n = 150) or without a limitations section (n = 150). Participants were blinded to the study hypothesis.

RESULTS

Adding a limitations section did not modify readers' interpretation of findings in terms of confidence in the results (mean difference [95% confidence interval] 0.19 [-0.37-0.74], p = 0.50), confidence in the validity of the conclusions (0.07 [-0.49-0.62], p = 0.80), or benefit of the experimental intervention (0.12 [-0.42-0.44], p = 0.65).This study is limited because the participants were expert-readers and are not representative of all systematic review readers.

CONCLUSION

Adding a limitations section to abstracts of systematic reviews did not affect readers' interpretation of the abstract results. Other studies are needed to confirm the results and explore the impact of a limitations section on a less expert panel of participants.

TRIAL REGISTRATION

ClinicalTrial.gov (NCT01848782).

Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles

BACKGROUND

Strong opinions for or against the use of systematic reviews to inform policymaking have been published in the medical literature. The purpose of this paper was to examine whether funding sources and author financial conflicts of interest were associated with whether an opinion article was supportive or critical of the use of systematic reviews for policymaking. We examined the nature of the arguments within each article, the types of disclosures present, and whether these articles are being cited in the academic literature.

METHODS

We searched for articles that expressed opinions about the use of systematic reviews for policymaking. We included articles that presented opinions about the use of systematic reviews for policymaking and categorized each article as supportive or critical of such use. We extracted all arguments regarding the use of systematic reviews from each article and inductively coded each as internal or external validity argument, categorized disclosed funding sources, conflicts of interest, and article types, and systematically searched for undisclosed financial ties. We counted the number of times each article has been cited in the "Web of Science." We report descriptive statistics.

RESULTS

Articles that were critical of the use of systematic reviews (n=25) for policymaking had disclosed or undisclosed industry ties 2.3 times more often than articles that were supportive of the use (n=34). We found that editorials, comments, letters, and perspectives lacked published disclosures nearly twice as often (60% v. 33%) as other types of articles. We also found that editorials, comments, letters, and perspectives were less frequently cited in the academic literature than other article types (median number of citations=5 v. 19).

CONCLUSIONS

It is important to consider whether an article has industry ties when evaluating the strength of the argument for or against the use of systematic reviews for policymaking. We found that journal conflict of interest disclosures are often inadequate, particularly for editorials, comments, letters, and perspectives and that these articles are being cited as evidence in the academic literature. Our results further suggest the need for more consistent and complete disclosure for all article types.

How good is "evidence" from clinical studies of drug effects and why might such evidence fail in the prediction of the clinical utility of drugs?

Promising evidence from clinical studies of drug effects does not always translate to improvements in patient outcomes. In this review, we discuss why early evidence is often ill suited to the task of predicting the clinical utility of drugs. The current gap between initially described drug effects and their subsequent clinical utility results from deficits in the design, conduct, analysis, reporting, and synthesis of clinical studies-often creating conditions that generate favorable, but ultimately incorrect, conclusions regarding drug effects. There are potential solutions that could improve the relevance of clinical evidence in predicting the real-world effectiveness of drugs. What is needed is a new emphasis on clinical utility, with nonconflicted entities playing a greater role in the generation, synthesis, and interpretation of clinical evidence. Clinical studies should adopt strong design features, reflect clinical practice, and evaluate outcomes and comparisons that are meaningful to patients. Transformative changes to the research agenda may generate more meaningful and accurate evidence on drug effects to guide clinical decision making.

Bias Neglect: A Blind Spot in the Evaluation of Scientific Results

Experimenter bias occurs when scientists' hypotheses influence their results, even if involuntarily. Meta-analyses have suggested that in some domains, such as psychology, up to a third of the studies could be unreliable due to such biases. A series of experiments demonstrates that while people are aware of the possibility that scientists can be more biased when the conclusions of their experiments fit their initial hypotheses, they robustly fail to appreciate that they should also be more sceptical of such results. This is true even when participants read descriptions of studies that have been shown to be biased. Moreover, participants take other sources of bias—such as financial incentives—into account, showing that this bias neglect may be specific to theory-driven hypothesis testing. In combination with a common style of scientific reporting, bias neglect could lead the public to accept premature conclusions.

Nonindustry-sponsored preclinical studies on statins yield greater efficacy estimates than industry-sponsored studies: a meta-analysis

This study by Lisa Bero and colleagues uses published preclinical statin research to show that nonindustry-funded animal studies yield more efficacious drug results than do industry-funded ones.

Industry-sponsored clinical drug studies are associated with publication of outcomes that favor the sponsor, even when controlling for potential bias in the methods used. However, the influence of sponsorship bias has not been examined in preclinical animal studies. We performed a meta-analysis of preclinical statin studies to determine whether industry sponsorship is associated with either increased effect sizes of efficacy outcomes and/or risks of bias in a cohort of published preclinical statin studies. We searched Medline (January 1966–April 2012) and identified 63 studies evaluating the effects of statins on atherosclerosis outcomes in animals. Two coders independently extracted study design criteria aimed at reducing bias, results for all relevant outcomes, sponsorship source, and investigator financial ties. The I² statistic was used to examine heterogeneity. We calculated the standardized mean difference (SMD) for each outcome and pooled data across studies to estimate the pooled average SMD using random effects models. In a priori subgroup analyses, we assessed statin efficacy by outcome measured, sponsorship source, presence or absence of financial conflict information, use of an optimal time window for outcome assessment, accounting for all animals, inclusion criteria, blinding, and randomization. The effect of statins was significantly larger for studies sponsored by nonindustry sources (−1.99; 95% CI −2.68, −1.31) versus studies sponsored by industry (−0.73; 95% CI −1.00, −0.47) (p value<0.001). Statin efficacy did not differ by disclosure of financial conflict information, use of an optimal time window for outcome assessment, accounting for all animals, inclusion criteria, blinding, and randomization. Possible reasons for the differences between nonindustry- and industry-sponsored studies, such as selective reporting of outcomes, require further study.

Industry-sponsored clinical drug studies are associated with publication of outcomes that favor the sponsor, even when controlling for potential bias in the methods used. However, the influence of sponsorship bias has not been examined in preclinical animal studies. We performed a meta-analysis to identify whether industry sponsorship is associated with increased risks of bias or effect sizes of outcomes in a cohort of published preclinical studies of the effects of statins on outcomes related to atherosclerosis. We found that in contrast to clinical studies, the effect of statins was significantly larger for studies sponsored by nonindustry sources versus studies sponsored by industry. Furthermore, statin efficacy did not differ with respect to disclosure of financial conflict information, use of an optimal time window for outcome assessment, accounting for all animals, inclusion criteria, blinding, and randomization. Possible reasons for the differences between nonindustry- and industry-sponsored studies, such as selective outcome reporting, require further study. Overall, our findings provide empirical evidence regarding the impact of funding and other methodological criteria on research outcomes.

Reducing waste from incomplete or unusable reports of biomedical research

Research publication can both communicate and miscommunicate. Unless research is adequately reported, the time and resources invested in the conduct of research is wasted. Reporting guidelines such as CONSORT, STARD, PRISMA, and ARRIVE aim to improve the quality of research reports, but all are much less adopted and adhered to than they should be. Adequate reports of research should clearly describe which questions were addressed and why, what was done, what was shown, and what the findings mean. However, substantial failures occur in each of these elements. For example, studies of published trial reports showed that the poor description of interventions meant that 40-89% were non-replicable; comparisons of protocols with publications showed that most studies had at least one primary outcome changed, introduced, or omitted; and investigators of new trials rarely set their findings in the context of a systematic review, and cited a very small and biased selection of previous relevant trials. Although best documented in reports of controlled trials, inadequate reporting occurs in all types of studies-animal and other preclinical studies, diagnostic studies, epidemiological studies, clinical prediction research, surveys, and qualitative studies. In this report, and in the Series more generally, we point to a waste at all stages in medical research. Although a more nuanced understanding of the complex systems involved in the conduct, writing, and publication of research is desirable, some immediate action can be taken to improve the reporting of research. Evidence for some recommendations is clear: change the current system of research rewards and regulations to encourage better and more complete reporting, and fund the development and maintenance of infrastructure to support better reporting, linkage, and archiving of all elements of research. However, the high amount of waste also warrants future investment in the monitoring of and research into reporting of research, and active implementation of the findings to ensure that research reports better address the needs of the range of research users.

Increasing value and reducing waste in research design, conduct, and analysis

Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

The cycle of bias in health research: a framework and toolbox for critical appraisal training

Recognizing bias in health research is crucial for evidence-based decision making. We worked with eight community groups to develop materials for nine modular, individualized critical appraisal workshops we conducted with 102 consumers (four workshops), 43 healthcare providers (three workshops), and 33 journalists (two workshops) in California. We presented workshops using a "cycle of bias" framework, and developed a toolbox of presentations, problem-based small group sessions, and skill-building materials to improve participants' ability to evaluate research for financial and other conflicts of interest, bias, validity, and applicability. Participant feedback indicated that the adaptability of the toolbox and our focus on bias were critical elements in the success of our workshops.

All that glitters isn't gold: a survey on acknowledgment of limitations in biomedical studies

BACKGROUND

Acknowledgment of all serious limitations to research evidence is important for patient care and scientific progress. Formal research on how biomedical authors acknowledge limitations is scarce.

OBJECTIVES

To assess the extent to which limitations are acknowledged in biomedical publications explicitly, and implicitly by investigating the use of phrases that express uncertainty, so-called hedges; to assess the association between industry support and the extent of hedging.

DESIGN

We analyzed reporting of limitations and use of hedges in 300 biomedical publications published in 30 high and medium -ranked journals in 2007. Hedges were assessed using linguistic software that assigned weights between 1 and 5 to each expression of uncertainty.

RESULTS

Twenty-seven percent of publications (81/300) did not mention any limitations, while 73% acknowledged a median of 3 (range 1-8) limitations. Five percent mentioned a limitation in the abstract. After controlling for confounders, publications on industry-supported studies used significantly fewer hedges than publications not so supported (p = 0.028).

LIMITATIONS

Detection and classification of limitations was--to some extent--subjective. The weighting scheme used by the hedging detection software has subjective elements.

CONCLUSIONS

Reporting of limitations in biomedical publications is probably very incomplete. Transparent reporting of limitations may protect clinicians and guideline committees against overly confident beliefs and decisions and support scientific progress through better design, conduct or analysis of new studies.

Conflict of Interest in Research—The Clinician Scientist’s Perspective

Conflict of interest (COI) in research represents situations that pose risks of undue influence on scientific objectivity and judgment because of secondary interests. This is complex but is inherent to biomedical research. The role of a clinician scientist can be conflicted when scientific objectivity is perceived to compete with scientific success (publications, grants), partiality to patients (clinical trials), obligations to colleagues (allowing poor scholarship to pass), research sponsors (industry), and financial gains (patents, royalties). While there are many ways which COIs can occur in research, COI mitigations remain reliable. Collaborations between investigators and industry are valuable to the development of novel therapies and undue discouragement of these relationships may inadvertently harm the advancement of healthcare. As a result, proper management of COI is fundamental and crucial to the maintenance of long-term, mutually beneficial relationships between industry and academia. The nature of COI in research and methods of mitigation are discussed from the perspective of a clinician scientist. Key words: Academia, Disclosure, Industry

"Spin" in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes

BACKGROUND

Spin in the reporting of randomized controlled trials, where authors report research in a way that potentially misrepresents results and mislead readers, has been demonstrated in the broader medical literature. We investigated spin in wound care trials with (a) no statistically significant result for the primary outcome and (b) no clearly specified primary outcome.

METHODS

We searched the Cochrane Wounds Group Specialised Register of Trials for randomized controlled trials (RCTs). Eligible studies were: Parallel-group RCTs of interventions for foot, leg or pressure ulcers published in 2004 to 2009 (inclusive) with either a clearly identified primary outcome for which there was a statistically non-significant result (Cohort A) or studies that had no clear primary outcome (Cohort B).We extracted general study details. For both Cohorts A and B we then assessed for the presence of spin. For Cohort A we used a pre-defined process to assess reports for spin. For Cohort B we aimed to assess spin by recording the number of positive treatment effect claims made. We also compared the number of statistically significant and non-significant results reported in the main text and the abstract looking specifically for spin in the form of selective outcome reporting.

RESULTS

Of the 71 eligible studies, 28 were eligible for Cohort A; of these, 71% (20/28) contained spin. Cohort B contained 43 studies; of these, 86% (37/43) had abstracts that claimed a favorable treatment claim. Whilst 74% (32/43) of main text results in Cohort B included at least one statistically non-significant result, this was not reflected in the abstract where only 28% contained (12/43) at least one statistically non-significant result.

CONCLUSIONS

Spin is a frequent phenomenon in reports of RCTs of wound treatments. Studies without statistically significant results for the primary outcome used spin in 71% of cases. Furthermore, 33% (43/132) of reports of wound RCTs did not specify a primary outcome and there was evidence of spin and selective outcome reporting in the abstracts of these. Readers should be wary of only reading the abstracts of reports of RCTs of wound treatments since they are frequently misleading regarding treatment effects.

The Drug Facts Box: Improving the communication of prescription drug information

Communication about prescription drugs ought to be a paragon of public science communication. Unfortunately, it is not. Consumers see $4 billion of direct-to-consumer advertising annually, which typically fails to present data about how well drugs work. The professional label--the Food and Drug Administration's (FDA) mechanism to get physicians information needed for appropriate prescribing--may also fail to present benefit data. FDA labeling guidance, in fact, suggests that industry omit benefit data for new drugs in an existing class and for drugs approved on the basis of unfamiliar outcomes (such as depression rating scales). The medical literature is also problematic: there is selective reporting of favorable trials, favorable outcomes within trials, and "spinning" unfavorable results to maximize benefit and minimize harm. In contrast, publicly available FDA reviews always include the phase 3 trial data on benefit and harm, which are the basis of drug approval. However, these reviews are practically inaccessible: lengthy, poorly organized, and weakly summarized. To improve accessibility, we developed the Drug Facts Box: a one-page summary of benefit and harm data for each indication of a drug. A series of studies--including national randomized trials--demonstrates that most consumers understand the Drug Facts Box and that it improves decision-making. Despite calls from their own Risk Communication Advisory Committee and Congress (in the Affordable Care Act) to consider implementing boxes, the FDA announced it needs at least 3-5 y more to make a decision. Given its potential public health impact, physicians and the public should not have to wait that long for better drug information.

Undue industry influences that distort healthcare research, strategy, expenditure and practice: a review

BACKGROUND

Expenditure on industry products (mostly drugs and devices) has spiraled over the last 15 years and accounts for substantial part of healthcare expenditure. The enormous financial interests involved in the development and marketing of drugs and devices may have given excessive power to these industries to influence medical research, policy, and practice.

MATERIAL AND METHODS

Review of the literature and analysis of the multiple pathways through which the industry has directly or indirectly infiltrated the broader healthcare systems. We present the analysis of the industry influences at the following levels: (i) evidence base production, (ii) evidence synthesis, (iii) understanding of safety and harms issues, (iv) cost-effectiveness evaluation, (v) clinical practice guidelines formation, (vi) healthcare professional education, (vii) healthcare practice, (viii) healthcare consumer's decisions.

RESULTS

We located abundance of consistent evidence demonstrating that the industry has created means to intervene in all steps of the processes that determine healthcare research, strategy, expenditure, practice and education. As a result of these interferences, the benefits of drugs and other products are often exaggerated and their potential harms are downplayed, and clinical guidelines, medical practice, and healthcare expenditure decisions are biased.

CONCLUSION

To serve its interests, the industry masterfully influences evidence base production, evidence synthesis, understanding of harms issues, cost-effectiveness evaluations, clinical practice guidelines and healthcare professional education and also exerts direct influences on professional decisions and health consumers. There is an urgent need for regulation and other action towards redefining the mission of medicine towards a more objective and patient-, population- and society-benefit direction that is free from conflict of interests.

Most meta-analyses of drug interventions have narrow scopes and many focus on specific agents

OBJECTIVE

To assess the extent to which meta-analysis publications of drugs and biologics focus on specific named agents or even only a single agent, and identify characteristics associated with such focus.

STUDY DESIGN AND SETTING

We evaluated 499 articles with meta-analyses published in 2010 and estimated how many did not cover all the available comparisons of tested interventions for a given condition (not all-inclusive); focused on specific named agent(s), or focused strictly on comparisons of only one specific active agent vs. placebo/no treatment or different doses/schedules.

RESULTS

Of 499 eligible articles, 403 (80.8%) were not all-inclusive, 214 (42.9%) covered only specific named agent(s), and 74 (14.8%) examined only comparisons with one active agent vs. placebo/no treatment or different doses/schedules. Only 39 articles (7.8%) covered all possible indications for the examined agent(s). After adjusting for type of treatment/field, focus on specific named agent(s) was associated with publication in journal venues (odds ratio [OR]: 1.95; 95% confidence interval [CI]: 1.17-3.26) vs. Cochrane, industry sponsoring (OR: 3.94; 95% CI: 1.66-10.66), and individual patient data analyses (OR: 6.59; 95% CI: 2.24-19.39). Individual patient data analyses primarily (29/34) focused on specific named agent(s).

CONCLUSION

The scope of meta-analysis publications frequently is narrow and shaped to serve particular agents.

[Consistency in the analysis and reporting of PEPs in oncology randomized controlled trials from registration to publication: a systematic review]

To improve the quality of reporting of randomized clinical trials (RCTs), international registries for RCTs and guidelines for primary endpoint (PEP) analysis were established. The objectives of this systematic review were to evaluate concordance of PEP between publication and the corresponding registry and to assess the intrapublication consistency in PEP reporting. All adult oncology RCTs in solid tumors published in 10 journals between 2005 and 2009 were reviewed. Registration information was extracted from international trial registries. A total 366 RCTs were identified. Trial registration was found for 215 trials and the rate increased from 43% in 2005 to 82% in 2009 (P < 0.001). There were 134 RCTs with clearly defined PEPs in registry, with the rate increasing from 15 to 67% (P < 0.001). PEP differs between registration and final publication in 14% trials with clearly defined PEPs. Reporting issues in methodology were found in 15% RCTs, mainly due to inadequate reporting of PEP or of sample size calculation. Problems with the interpretation of trial results were found in 22% publications, mostly due to negative superiority studies being interpreted as showing equivalence. The rates of trial registration and of trials with clearly defined PEP have improved over time, however 14% of these trials reported a different PEP in the final publication. Intrapublication inconsistencies in PEP reporting are frequent. Our findings highlight the need for investigators, peer reviewers and readers for increased awareness and scrutiny of reporting outcomes of oncology RCTs.