A comparison of the scientific quality of publicly and privately funded randomized controlled drug trials

Are European clinical trial funders policies on clinical trial registration and reporting improving? A cross-sectional study

Objectives

Assess the extent to which the clinical trial registration and reporting policies of 25 of the world's largest public and philanthropic medical research funders meet best practice benchmarks as stipulated by the 2017 WHO Joint Statement, and document changes in the policies and monitoring systems of 19 European funders over the past year.

Design Setting Participants

Cross-sectional study, based on assessments of each funder's publicly available documentation plus validation of results by funders. Our cohort includes 25 of the largest medical research funders in Europe, Oceania, South Asia, and Canada.

Interventions

Scoring all 25 funders using an 11-item assessment tool based on WHO best practice benchmarks, grouped into three primary categories: trial registries, academic publication, and monitoring, plus validation of results by funders.

Main outcome measures

How many of the 11 WHO best practice items each of the 25 funders has put into place, and changes in the performance of 19 previously assessed funders over the preceding year.

Results

The 25 funders we assessed had put into place an average of 5/11 (49%) WHO best practices. Only 6/25 funders (24%) took the PI's past reporting record into account during grant application reviews. Funders' performance varied widely from 0/11 to 11/11 WHO best practices adopted. Of the 19 funders for which 2021(2) baseline data was available, 10/19 (53%) had strengthened their policies over the preceding year.

Conclusions

Most medical research funders need to do more to curb research waste and publication bias by strengthening their clinical trial policies.

Industry sponsorship and research outcome

BACKGROUND

Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical-industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. A similar association between sponsorship and outcomes have been found for device studies, but the body of evidence is not as strong as for sponsorship of drug studies. This review is an update of a previous Cochrane review and includes empirical studies on the association between sponsorship and research outcome.

OBJECTIVES

To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

SEARCH METHODS

In this update we searched MEDLINE (2010 to February 2015), Embase (2010 to February 2015), the Cochrane Methodology Register (2015, Issue 2) and Web of Science (June 2015). In addition, we searched reference lists of included papers, previous systematic reviews and author files.

SELECTION CRITERIA

Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions.

DATA COLLECTION AND ANALYSIS

Two assessors screened abstracts and identified and included relevant papers. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals (CIs)).

MAIN RESULTS

Twenty-seven new papers were included in this update and in total the review contains 75 included papers. Industry sponsored studies more often had favorable efficacy results, RR: 1.27 (95% CI: 1.17 to 1.37) (25 papers) (moderate quality evidence), similar harms results RR: 1.37 (95% CI: 0.64 to 2.93) (four papers) (very low quality evidence) and more often favorable conclusions RR: 1.34 (95% CI: 1.19 to 1.51) (29 papers) (low quality evidence) compared with non-industry sponsored studies. Nineteen papers reported on sponsorship and efficacy effect size, but could not be pooled due to differences in their reporting of data and the results were heterogeneous. We did not find a difference between drug and device studies in the association between sponsorship and conclusions (test for interaction, P = 0.98) (four papers). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment, follow-up and selective outcome reporting. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.25 (95% CI: 1.05 to 1.50) (13 papers), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.83 (95% CI: 0.70 to 0.98) (six papers).

AUTHORS' CONCLUSIONS

Sponsorship of drug and device studies by the manufacturing company leads to more favorable efficacy results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.

Wounds research for patient benefit: a 5-year programme of research

Background

Complex wounds are those that heal by secondary intention and include lower-limb ulcers, pressure ulcers and some surgical wounds. The care of people with complex wounds is costly, with care mainly being delivered by community nurses. There is a lack of current, high-quality data regarding the numbers and types of people affected, care received and outcomes achieved.

Objectives

To (1) assess how high-quality data about complex wounds can be captured effectively for use in both service planning and research while ensuring integration with current clinical data collection systems and minimal impact on staff time; (2) investigate whether or not a clinical register of people with complex wounds could give valid estimates of treatment effects, thus reducing dependence on large-scale randomised controlled trials (RCTs); (3) identify the most important research questions and outcomes for people with complex wounds from the perspectives of patients, carers and health-care professionals; (4) evaluate the potential contributions to decision-making of individual patient data meta-analysis and mixed treatment comparison meta-analysis; and (5) complete and update systematic reviews in topic areas of high priority.

Methods

To meet objectives 1 and 2 we conducted a prevalence survey and developed and piloted a longitudinal disease register. A consultative, deliberative method and in-depth interviews were undertaken to address objective 3. To address objectives 4 and 5 we conducted systematic reviews including mixed treatment comparison meta-analysis.

Results

From the prevalence survey we estimated the point prevalence of all complex wounds to be 1.47 per 1000 people (95% confidence interval 1.38 to 1.56 per 1000 people). Pressure ulcers and venous leg ulcers were the most common type of complex wound. A total of 195 people with a complex wound were recruited to a complex wounds register pilot. We established the feasibility of correctly identifying, extracting and transferring routine NHS data into the register; however, participant recruitment, data collection and tracking individual wounds in people with multiple wounds were challenging. Most patients and health professionals regarded healing of the wound as the primary treatment goal. Patients were greatly troubled by the social consequences of having a complex wound. Complex wounds are frequently a consequence of, and are themselves, a long-term condition but treatment is usually focused on healing the wound. Consultative, deliberative research agenda setting on pressure ulcer prevention and treatment with patients, carers and clinicians yielded 960 treatment uncertainties and a top 12 list of research priorities. Of 167 RCTs of complex wound treatments in a systematic review of study quality, 41% did not specify a primary outcome and the overall quality of the conduct and reporting of the research was poor. Mixed-treatment comparison meta-analysis in areas of high priority identified that matrix hydrocolloid dressings had the highest probability (70%) of being the most effective dressing for diabetic foot ulcers, whereas a hyaluronan fleece dressing had the highest probability (35%) of being the most effective dressing for venous ulcers; however, the quality of this evidence was low and uncertainty is high.

Conclusions

Complex wounds are common and costly with a poor evidence base for many frequent clinical decisions. There is little routine clinical data collection in community nursing. A prospective complex wounds register has the potential to both assist clinical decision-making and provide important research evidence but would be challenging to implement without investment in information technology in NHS community services. Future work should focus on developing insights into typical wound healing trajectories, identifying factors that are prognostic for healing and assessing the cost-effectiveness of selected wound treatments.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Peer review comments on drug trials submitted to medical journals differ depending on sponsorship, results and acceptance: a retrospective cohort study

OBJECTIVE

During peer review, submitted manuscripts are scrutinised by independent experts to assist journal editors in their decision-making and to help improve the quality of articles. In this retrospective cohort study, peer review comments for drug trials submitted to medical journals were analysed to investigate whether there is a relation between the content of these comments and sponsorship, direction of results and decision about acceptance.

DESIGN/SETTING

Descriptive content analysis of reviewer comments made on manuscripts on drug trials submitted to eight medical journals (January 2010-April 2012). For each manuscript, the number of reviewers, decision about acceptance, sponsorship and direction of results were extracted. Reviewer comments were classified using a predefined checklist.

RESULTS

Reviewer reports for 246 manuscripts were assessed. Industry-sponsored trials were more likely to receive comments about lack of novelty (8.9%) than industry-supported (2.5%) and non-industry trials (6.1%, overall p=0.038). Non-industry trials more often received comments about poor experimental design (69.7%) than industry-supported (58.8%) and industry-sponsored trials (52.9%, overall p=0.019). Non-industry trials were also more likely to receive comments regarding inappropriate statistical analyses (28.4%) than industry-supported (23.5%) and industry-sponsored trials (15.1%, overall p=0.006). Manuscripts with negative results were more likely to receive comments about inappropriate conclusions (29.3%) than those with positive results (18.9%, p=0.010). Rejected manuscripts had more often received comments on the research question not being clinically relevant (7.8%) than accepted manuscripts (1.6%, p=0.002), and also on lack of novelty (8.3% vs 2.6%, p=0.008) and poor experimental design (68.6% vs 50.5%, p<0.001).

CONCLUSIONS

Reviewers identified fewer shortcomings regarding design and statistical analyses in industry-related trials, but commented more often on a lack of novelty in industry-sponsored trials. Negative trial results did not significantly influence the nature of comments other than appropriateness of the conclusion. Manuscript acceptance was primarily related to the research question and methodological robustness of studies.

Shortcomings of protocols of drug trials in relation to sponsorship as identified by Research Ethics Committees: analysis of comments raised during ethical review

BACKGROUND

Submission of study protocols to research ethics committees (RECs) constitutes one of the earliest stages at which planned trials are documented in detail. Previous studies have investigated the amendments requested from researchers by RECs, but the type of issues raised during REC review have not been compared by sponsor type. The objective of this study was to identify recurring shortcomings in protocols of drug trials based on REC comments and to assess whether these were more common among industry-sponsored or non-industry trials.

METHODS

Retrospective analysis of 226 protocols of drug trials approved in 2010-2011 by three RECs affiliated to academic medical centres in The Netherlands. For each protocol, information on sponsorship, number of participating centres, participating countries, study phase, registration status of the study drug, and type and number of subjects was retrieved. REC comments were extracted from decision letters sent to investigators after review and were classified using a predefined checklist that was based on legislation and guidelines on clinical drug research and previous literature.

RESULTS

Most protocols received comments regarding participant information and consent forms (n = 182, 80.5%), methodology and statistical analyses (n = 160, 70.8%), and supporting documentation, including trial agreements and certificates of insurance (n = 154, 68.1%). Of the submitted protocols, 122 (54.0%) were non-industry and 104 (46.0%) were industry-sponsored trials. Non-industry trials more often received comments on subject selection (n = 44, 36.1%) than industry-sponsored trials (n = 18, 17.3%; RR, 1.58; 95% CI, 1.01 to 2.47), and on methodology and statistical analyses (n = 95, 77.9% versus n = 65, 62.5%, respectively; RR, 1.18; 95% CI, 1.01 to 1.37). Non-industry trials less often received comments on supporting documentation (n = 72, 59.0%) than industry-sponsored trials (n = 82, 78.8%; RR, 0.83; 95% CI, 0.72 to 0.95).

CONCLUSIONS

RECs identified important ethical and methodological shortcomings in protocols of both industry-sponsored and non-industry drug trials. Investigators, especially of non-industry trials, should better prepare their research protocols in order to facilitate the ethical review process.

Funding source and the quality of reports of chronic wounds trials: 2004 to 2011

BACKGROUND

Critical commentaries suggest that wound care randomised controlled trials (RCTs) are often poorly reported with many methodological flaws. Furthermore, interventions in chronic wounds, rather than being drugs, are often medical devices for which there are no requirements for RCTs to bring products to market. RCTs in wounds trials therefore potentially represent a form of marketing. This study presents a methodological overview of chronic wound trials published between 2004 and 2011 and investigates the influence of industry funding on methodological quality.

METHODS

A systematic search for RCTs for the treatment of chronic wounds published in the English language between 2004 and 2011 (inclusive) in the Cochrane Wounds Group Specialised Register of Trials was carried out.Data were extracted on aspects of trial design, conduct and quality including sample size, duration of follow-up, specification of a primary outcome, use of surrogate outcomes, and risks of bias. In addition, the prevalence of industry funding was assessed and its influence on the above aspects of trial design, conduct and quality was assessed.

RESULTS

A total of 167 RCTs met our inclusion criteria. We found chronic wound trials often have short durations of follow-up (median 12 weeks), small sample sizes (median 63), fail to define a primary outcome in 41% of cases, and those that do define a primary outcome, use surrogate measures of healing in 40% of cases. Only 40% of trials used appropriate methods of randomisation, 25% concealed allocation and 34% blinded outcome assessors. Of the included trials, 41% were wholly or partially funded by industry, 33% declared non-commercial funding and 26% did not report a funding source. Industry funding was not statistically significantly associated with any measure of methodological quality, though this analysis was probably underpowered.

CONCLUSIONS

This overview confirms concerns raised about the methodological quality of RCTs in wound care and illustrates that greater efforts must be made to follow international standards for conducting and reporting RCTs. There is currently minimal evidence of an influence of industry funding on methodological quality although analyses had limited power and funding source was not reported for a quarter of studies.

Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis

OBJECTIVE

To compare methodological characteristics of randomized controlled trials (RCTs) published in higher vs. lower impact Core Clinical Journals.

STUDY DESIGN AND SETTING

We searched MEDLINE for RCTs published in 2007 in Core Clinical Journals. We randomly sampled 1,140 study reports in a 1:1 ratio in higher (five general medicine journals with the highest total citations in 2007) and lower impact journals.

RESULTS

Four hundred sixty-nine RCTs proved eligible: 219 in higher and 250 in lower impact journals. RCTs in higher vs. lower impact journals had larger sample sizes (median, 285 vs. 39), were more likely to receive industry funding (53% vs. 28%), declare concealment of allocation (66% vs. 36%), declare blinding of health care providers (53% vs. 41%) and outcome adjudicators (72% vs. 54%), report a patient-important primary outcome (69% vs. 50%), report subgroup analyses (64% vs. 26%), prespecify subgroup hypotheses (42% vs. 20%), and report a test for interaction (54% vs. 27%); P < 0.05 for all differences.

CONCLUSION

RCTs published in higher impact journals were more likely to report methodological safeguards against bias and patient-important outcomes than those published in lower impact journals. However, sufficient limitations remain such that publication in a higher impact journal does not ensure low risk of bias.

Characteristics of randomized trials published in Latin America and the Caribbean according to funding source

Introduction

Few studies have assessed the nature and quality of randomized controlled trials (RCTs) in Latin America and the Caribbean (LAC).

Methods and Findings

The aims of this systematic review are to evaluate the characteristics (including the risk of bias assessment) of RCT conducted in LAC according to funding source. A review of RCTs published in 2010 in which the author's affiliation was from LAC was performed in PubMed and LILACS. Two reviewers independently extracted data and assessed the risk of bias. The primary outcomes were risk of bias assessment and funding source. A total of 1,695 references were found in PubMed and LILACS databases, of which 526 were RCTs (N = 73.513 participants). English was the dominant publication language (93%) and most of the RCTs were published in non-LAC journals (84.2%). Only five of the 19 identified countries accounted for nearly 95% of all RCTs conducted in the region (Brazil 70.9%, Mexico 10.1%, Argentina 5.9%, Colombia 3.8%, and Chile 3.4%). Few RCTs covered priority areas related with Millennium Development Goals like maternal health (6.7%) or high priority infectious diseases (3.8%). Regarding children, 3.6% and 0.4% RCT evaluated nutrition and diarrhea interventions respectively but none pneumonia. As a comparison, aesthetic and sport related interventions account for 4.6% of all trials. A random sample of RCTs (n = 358) was assessed for funding source: exclusively public (33.8%); private (e.g. pharmaceutical company) (15.3%); other (e.g. mixed, NGO) (15.1%); no funding (35.8%). Overall assessments for risk of bias showed no statistically significant differences between RCTs and type of funding source. Statistically significant differences favoring private and others type of funding was found when assessing trial registration and conflict of interest reporting.

Conclusion

Findings of this study could be used to provide more direction for future research to facilitate innovation, improve health outcomes or address priority health problems.

Industry sponsorship and research outcome

BACKGROUND

Clinical research affecting how doctors practice medicine is increasingly sponsored by companies that make drugs and medical devices. Previous systematic reviews have found that pharmaceutical industry sponsored studies are more often favorable to the sponsor's product compared with studies with other sources of sponsorship. This review is an update using more stringent methodology and also investigating sponsorship of device studies.

OBJECTIVES

To investigate whether industry sponsored drug and device studies have more favorable outcomes and differ in risk of bias, compared with studies having other sources of sponsorship.

SEARCH METHODS

We searched MEDLINE (1948 to September 2010), EMBASE (1980 to September 2010), the Cochrane Methodology Register (Issue 4, 2010) and Web of Science (August 2011). In addition, we searched reference lists of included papers, previous systematic reviews and author files.

SELECTION CRITERIA

Cross-sectional studies, cohort studies, systematic reviews and meta-analyses that quantitatively compared primary research studies of drugs or medical devices sponsored by industry with studies with other sources of sponsorship. We had no language restrictions.

DATA COLLECTION AND ANALYSIS

Two assessors identified potentially relevant papers, and a decision about final inclusion was made by all authors. Two assessors extracted data, and we contacted authors of included papers for additional unpublished data. Outcomes included favorable results, favorable conclusions, effect size, risk of bias and whether the conclusions agreed with the study results. Two assessors assessed risk of bias of included papers. We calculated pooled risk ratios (RR) for dichotomous data (with 95% confidence intervals).

MAIN RESULTS

Forty-eight papers were included. Industry sponsored studies more often had favorable efficacy results, risk ratio (RR): 1.24 (95% confidence interval (CI): 1.14 to 1.35), harms results RR: 1.87 (95% CI: 1.54 to 2.27) and conclusions RR: 1.31 (95% CI: 1.20 to 1.44) compared with non-industry sponsored studies. Ten papers reported on sponsorship and effect size, but could not be pooled due to differences in their reporting of data. The results were heterogeneous; five papers found larger effect sizes in industry sponsored studies compared with non-industry sponsored studies and five papers did not find a difference in effect size. Only two papers (including 120 device studies) reported separate data for devices and we did not find a difference between drug and device studies on the association between sponsorship and conclusions (test for interaction, P = 0.23). Comparing industry and non-industry sponsored studies, we did not find a difference in risk of bias from sequence generation, allocation concealment and follow-up. However, industry sponsored studies more often had low risk of bias from blinding, RR: 1.32 (95% CI: 1.05 to 1.65), compared with non-industry sponsored studies. In industry sponsored studies, there was less agreement between the results and the conclusions than in non-industry sponsored studies, RR: 0.84 (95% CI: 0.70 to 1.01).

AUTHORS' CONCLUSIONS

Sponsorship of drug and device studies by the manufacturing company leads to more favorable results and conclusions than sponsorship by other sources. Our analyses suggest the existence of an industry bias that cannot be explained by standard 'Risk of bias' assessments.

Scientific bias arising from sampling, selective recruitment, and attrition: the case for improved reporting

In human research, the ability to generalize study findings is incumbent not only on an accurate understanding of the study protocol and measures but also on a clear understanding of the study population. Differential recruitment and attrition has the potential of introducing bias and threatening the generalizability of study findings; yet, relatively few scientific publications report data on sampling, subject exclusion, and dropout. A 4-month census sampling (September-December 2009) of research articles and short communications in this journal (n=116) was no exception. Among articles in which such data were appropriate to report, only 44% documented response rates, 53% described subjects who were excluded, and 10% performed analyses on enrollee versus nonenrollee differences; moreover, of the 17 longitudinal or intervention studies evaluated, only 3 of 17 reported dropout rates, and of those, only 2 of 3 reported reasons for dropout or an analysis that compared the characteristics of dropouts with those of completers. Given Cancer Epidemiology, Biomarkers and Prevention's mission to enhance the dissemination of unbiased scientific findings, we propose that guidelines regarding sample description, as defined by CONSORT, STROBE, or STREGA, be adopted by our journal for both observational and interventional studies that accurately describe the study population from the point of contact.