Influence of Sponsorship Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions: A Meta-epidemiological Study

International Collaboration and Commercial Involvement in Randomized Controlled Trials From 10 Leading Countries, 1997 Through 2019

Randomized controlled trials (RCTs) affect clinical decisions and their number is increasing. However, trends in international collaboration on RCTs and involvement of healthcare-related industries, the latter of which may contribute to bias, are not known. The objectives were to identify concerns surrounding RCTs, and to quantify changes in (1) the numbers of RCT articles in journals of high clinical importance, (2) international collaboration, and (3) commercial involvement in RCTs by authors in countries that contribute the most to the scientific literature. This was not a systematic review of the medical literature. It is a descriptive study of trends during the past two decades. We extracted RCT articles from MEDLINE data (1997-2019). When grouped by authors' country, the analyses were limited to the 10 leading countries in the natural sciences, as defined by the Nature Index 2019 Annual Tables. The Core Clinical Journals (CCJ) filter in PubMed was used to identify journals that were likely to be highly relevant to clinical practice. RCT articles that included authors from multiple countries were used as examples of international collaboration, and RCTs in which at least one author's affiliation was corporate were considered to have commercial involvement. The annual number of RCT articles more than doubled (from 10,360 to 22,384), but the number published in the CCJ was essentially unchanged (from 2,245 to 2,346). The vast majority of RCT articles had US-based authors. International collaboration increased in nine of the 10 countries studied, and it was particularly common among researchers in Europe, Canada, and Australia. In contrast, international collaboration decreased in China. Regarding commercial involvement, between 1997 and 2019 the proportion of single-country RCTs with commercial involvement decreased (from 12.4% to 3.8% for the United States, and from 2.5% to 0.0% for Europe-Canada-Australia). In contrast, the proportion of international-collaborative RCTs with commercial involvement increased (from 9.2% to 17.6% for the United States, and from 17.9% to 21.3% for Europe-Canada-Australia). The largest change in commercial involvement was the 12-fold increase in Japan: from 3% to 36% (1997-2019). Japan was also noteworthy for its 28-percentage-point decrease in first-authorship of RCT articles from 2012 to 2019. In conclusion, recent increases in the number of RCT articles have occurred almost exclusively outside the CCJ. Thus, many newer RCT articles might have relatively low clinical relevance or impact. International collaboration has generally increased, along with commercial involvement. The latter has become particularly common in Japan, increasing the potential for sponsorship bias. The effects of ongoing attempts to reverse that trend should be evaluated.

Studies with statistically significant effect estimates are more frequently published compared to non-significant estimates in oral health journals

BACKGROUND

Studies reporting statistically significant effect estimates tend to be more frequently published compared to studies reporting non-significant or equivalent estimates. Consequently, this may lead to distortion of the literature. The aim of this study is to assess the prevalence of reporting statistically significant effect estimates in leading oral health journals and to explore associations between the effect estimates and record characteristics.

METHODS

An electronic database search was undertaken of a selection of leading oral health journals including general oral health journals to identify primary oral health records published in 2019. Descriptive statistics and population average GEE logistic regression model was used to assess associations between articles reporting a statistically significant effect estimate and the record characteristics.

RESULTS

In 1335 records, 82.4% records reported a statistically significant effect estimate. All speciality journals compared to general oral health journals were less likely to publish a record with significant effect estimates. Authors based in Asia or other (OR 1.49; 95% CI :1.02,2.19; p = 0.037) were more likely to report significant effect estimates compared to those based in Europe. Interventional (OR 0.35; 0.22,0.58; p < 0.001) and observational (OR 0.56; 0.36, 0.89; p = 0.013) records were less likely to report significant effect estimates compared to in-vitro studies. Registered records were less likely to report significant effect estimates when compared to non-registered studies (OR 0.22; 95% CI :0.14,0.32; p < 0.001).

CONCLUSION

The publishing of records with significant effect estimates is prevalent within the oral health literature. To reduce dissemination bias and overestimation of effect sizes in systematic reviews, the publishing of studies with non-significant or equivalent effect estimates should be encouraged.

Sponsorship Bias in Clinical Trials in the Dental Application of Probiotics: A Meta-Epidemiological Study

Many experimental and clinical trials have investigated the dental application of probiotics, although the evidence concerning the effects of probiotic supplements is conflicting. We aimed to examine whether sponsorship in trials about dental applications of probiotics is associated with biased estimates of treatment effects. Overall, 13 meta-analyses involving 48 randomized controlled trials (23 with high risk of sponsorship bias, 25 with low risk) with continuous outcomes were included. Effect sizes were calculated from differences in means of first reported continuous outcomes, divided by the pooled standard deviation. For each meta-analysis, the difference in standardized mean differences between high-risk and low-risk trials was estimated by random effects meta-regression. Differences in standardized mean differences (DSMDs) were then calculated via meta-analyses in a random effects meta-analysis model. A combined DSMD of greater than zero indicated that high-risk trials showed more significant treatment effects than low-risk trials. The results show that trials with a high risk of sponsorship bias showed more significant intervention effects than did low-risk trials (combined DSMD, 0.06; 95% confidence interval, 0.3 to 0.9; p < 0.001), with low heterogeneity among meta-analyses (I2 = 0%; between-meta-analyses variance τ2 = 0.00). Based on our study, high-risk clinical trials with continuous outcomes reported more favorable intervention effects than did low-risk trials in general.

Combining meta-epidemiological study datasets on commercial funding of randomised clinical trials: Database, methods, and descriptive results of the COMFIT study

Randomised trials are often funded by commercial companies and methodological studies support a widely held suspicion that commercial funding may influence trial results and conclusions. However, these studies often have a risk of confounding and reporting bias. The risk of confounding is markedly reduced in meta-epidemiological studies that compare fairly similar trials within meta-analyses, and risk of reporting bias is reduced with access to unpublished data. Therefore, we initiated the COMmercial Funding In Trials (COMFIT) study aimed at investigating the impact of commercial funding on estimated intervention effects in randomised clinical trials based on a consortium of researchers who agreed to share meta-epidemiological study datasets with information on meta-analyses and trials included in meta-epidemiological studies. Here, we describe the COMFIT study, its database, and descriptive results. We included meta-epidemiological studies with published or unpublished data on trial funding source and results or conclusions. We searched five bibliographic databases and other sources. We invited authors of eligible meta-epidemiological studies to join the COMFIT consortium and to share data. The final construction of the COMFIT database involves checking data quality, identifying trial references, harmonising variable categories, and removing non-informative meta-analyses as well as correlated meta-analyses and trial results. We included data from 17 meta-epidemiological studies, covering 728 meta-analyses and 6841 trials. Seven studies (405 meta-analyses, 3272 trials) had not published analyses on the impact of commercial funding, but shared unpublished data on funding source. On this basis, we initiated the construction of a combined database. Once completed, the database will enable comprehensive analyses of the impact of commercial funding on trial results and conclusions with increased statistical power and a markedly reduced risk of confounding and reporting bias.