An Analysis of Sponsors/Collaborators of 69,160 Drug Trials Registered with ClinicalTrials.gov

Significant Differences and Experimental Designs Do Not Necessarily Imply Clinical Relevance: Effect Sizes and Causality Claims in Antidepressant Treatments

Clinical trials are the backbone of medical scientific research. However, this experimental strategy has some drawbacks. We focused on two issues: (a) The internal validity ensured by clinical trial procedures does not necessarily allow for generalization of efficacy results to causal claims about effectiveness in the population. (b) Statistical significance does not imply clinical or practical significance; p-values should be supplemented with effect size (ES) estimators and an interpretation of the magnitude of the effects found. We conducted a systematic review (from 2000 to 2020) on Scopus, PubMed, and four ProQuest databases, including PsycINFO. We searched for experimental studies with significant effects of pharmacological treatments on depressive symptoms, measured with a specific scale for depression. We assessed the claims of effectiveness, and reporting and interpreting of effect sizes in a small, unbiased sample of clinical trials (n = 10). Only 30% of the studies acknowledged that efficacy does not necessarily translate to effectiveness. Only 20% reported ES indices, and only 40% interpreted the magnitude of their findings. We encourage reflection on the applicability of results derived from clinical trials about the efficacy of antidepressant treatments, which often influence daily clinical decision-making. Comparing experimental results of antidepressants with supplementary observational studies can provide clinicians with greater flexibility in prescribing medication based on patient characteristics. Furthermore, the ES of a treatment should be considered, as treatments with a small effect may be worthwhile in certain circumstances, while treatments with a large effect may be justified despite additional costs or complications. Therefore, researchers are encouraged to report and interpret ES and explicitly discuss the suitability of their sample for the clinical population to which the antidepressant treatment will be applied.

The Inclusion of Economic Endpoints as Outcomes in Clinical Trials Reported to ClinicalTrials.gov

BACKGROUND

As medication expenditures rise, payers are increasingly demanding evidence of economic value for new medications. The 2015 Professional Society for Health Economics and Outcomes Research (ISPOR) Task Force on Cost-Effectiveness Analysis Alongside Clinical Trials noted that clinical trials are increasingly including health care utilization endpoints to address this rising interest in economic information.

OBJECTIVES

To (a) describe the prevalence of economic endpoints in clinical trials submitted to ClinicalTrials.gov and (b) examine associations between trial characteristics and the inclusion of economic endpoints.

METHODS

This retrospective review of ClinicalTrials.gov data extracted the characteristics of clinical trials that were submitted to ClinicalTrials.gov from January 2004 to December 2018; studied a drug and/or biological; and had a recruitment status of not yet recruiting, recruiting, active but not recruiting, or completed. Studies were classified as containing an economic endpoint based on 2 independent evaluations of the inclusion of endpoints relevant to costs, resource utilization, cost-effectiveness, productivity, absenteeism, presenteeism, or unemployment. Descriptive statistics were used to summarize trial characteristics, and chi-square analyses were used to evaluate differences in characteristics between trials with and without economic endpoints.

RESULTS

Of the 104,885 trials included in the study, 1,437 (1.37%) included an economic endpoint; among later phase (phase 2/3, 3, 4) trials, 939 (2.54%) included economic endpoints. Compared with studies that did not include economic endpoints, those that did were less often industry funded (48.0% vs. 52.0%, P < 0.001) and were for a high-spend specialty condition (24.1% vs. 27.4%, P < 0.001). The proportion of trials that included economic endpoints increased by a small but significant amount over the time period studied, from 1.2% (2004-2008) to 1.6% (2014-2018; P < 0.001).

CONCLUSIONS

A small but growing number of clinical trials are including economic endpoints. This finding may reflect continued industry concerns surrounding the cost and logistical challenges of piggybacking economic data collection alongside clinical trials and/or manufacturers' preferences for modeling for value demonstration. Future research is needed to better understand barriers to the inclusion of economic endpoints as well as the degree to which incorporating health care resource utilization collected during clinical trials into early economic modeling may reduce payer concerns about model transparency and bias.

DISCLOSURES

No outside funding supported this study. Patterson reports past employment by Indivior, unrelated to this study. Mitchell has nothing to disclose. The research included in this study was presented as a nonreviewed student pharmacist poster at AMCP Nexus 2019; October 30-November 1, 2019; National Harbor, MD.

An analysis of deficiencies in the data of interventional drug trials registered with Clinical Trials Registry - India

Background

Clinical Trials Registry - India (CTRI) was established in July 2007 and today hosts thousands of trials, a significant fraction of them registered in the last couple of years. We wished to undertake an up-to-date analysis of specific fields of the registered trials. In doing so we discovered problems with the quality of the data, which we describe in this paper.

Methods

We downloaded CTRI records and reformatted the data into an SQLite database, which we then queried. We also accessed ClinicalTrials.gov records as needed.

Results

We discovered various categories of problems with the data in the CTRI database, including (1) a lack of clarity in the classification of Types of Study, (2) internal inconsistencies, (3) incomplete or non-standard information, (4) missing data, (5) variations in names or classification, and (6) incomplete or incorrect details of ethics committees. For most of these problems, error rates have been calculated, over time. Most were found to be in single digits, although others were significantly higher. We suggest how data quality in future editions of CTRI could be improved, including (1) a more elaborate and structured way of classifying the Type of Study, (2) the use of logic rules to prevent internal inconsistencies, (3) less use of free text fields and greater use of drop-down menus, (4) more fields to be made compulsory, (5) the pre-registration of individuals’ and organizations’ names and their subsequent selection from drop-down menus while registering a trial, and (6) more information about each ethics committee, including (a) its address and (b) linking the name of the trial site to the relevant ethics committee. As we discuss problems with the data of specific fields, we also examine — where possible — the quality of the data in the corresponding fields in ClinicalTrials.gov, the largest clinical trial registry in the world.

Conclusions

It is a scientific and ethical obligation to correctly record all information pertaining to each trial run in India. CTRI is a valuable database that has proved its worth in terms of improving the record of trials in the country. The suggestions made herein would improve it further.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3592-0) contains supplementary material, which is available to authorized users.

Some data quality issues at ClinicalTrials.gov

Background

Clinical trial registries have been established as a form of public accountability. Sponsors ought to register their trials promptly and accurately, but this is not always done. Some of the problems include non-registration of trials, registration of trials with incomplete information, and non-reporting of trial results on time. In this study we enumerate or quantify some quality issues with respect to Principal Investigator (PI) and Responsible Party data.

Methods

We analyzed interventional trials registered with ClinicalTrials.gov. Using certain selection criteria, we started with 112,013 records, and then applied further filters. The trial had to (a) start between 1 January 2005 and 31 December 2014, (b) include a “drug” or “biological” in the “intervention” field, (c) be registered with an American authority, and (d) list a real person’s name as investigator and also his or her role in the study.

Results

We identified four categories of errors in the ClinicalTrials.gov records. First, some data were missing. The name of the investigator, or his or her role, was missing in 12% of 35,121 trials. In examining 71,359 pairs of names and roles, 17% of the “names” were found to be not those of real persons, but instead junk information. Second, there were variations in a large number of names. We identified 19 categories of variants. We determined that 13% of the names had variants that could not be resolved using a program. Third, some trials listed many PIs each, although only one such person holds overall responsibility for the trial and therefore not more than one person should be listed as PI. Fourth, in examining whether the PI’s name was available as part of the Responsible Party tag, we found that in 1221 (3.5%) of 35,121 trials, the Responsible Party tag is absent.

Conclusions

We have outlined four categories of problems with data hosted by ClinicalTrials.gov and have quantified three of them. We also suggest how these errors could be prevented in future. It is important to carry out various kinds of audits of trial registries, in order to identify lacunae in the records, that they be addressed.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3408-2) contains supplementary material, which is available to authorized users.