Prevention of selective outcome reporting: let us start from the beginning

Dissemination and outcome reporting bias in clinical malaria intervention trials: a cross-sectional analysis

BACKGROUND

Dissemination and outcome reporting biases are a significant problem in clinical research, with far-reaching implications for both scientific understanding and clinical decision-making. This study investigates the prevalence of dissemination- and outcome reporting biases in registered interventional malaria research.

METHODS

All malaria interventional trials registered on ClinicalTrials.gov from 2010 to 2020 were identified. Subsequently, publications that matched the registration were searched. The primary outcome measures were the percentage of registered studies that resulted in subsequent publication of study results, the concordance between registered outcomes, and reported outcomes. Secondary outcomes were compliance with WHO standards for timely publication (issued in 2017) of summary study results in the respective trial registry (within 12 months of study completion) or peer-reviewed publication (within 24 months of study completion) was evaluated.

RESULTS

A total of 579 trials were identified on ClinicalTrials.gov, of which 544 met the inclusion criteria. Notably, almost 36.6% of these trials (199/544) were registered retrospectively, with 129 (23.7%) registered after the first patient enrolment and 70 (12.9%) following study completion. Publications were identified for 351 out of 544 registered trials (64.5%), involving 1,526,081 study participants. Conversely, publications were not found for 193 of the 544 registrations (35.5%), which aimed to enrol 417,922 study participants. Among these 544 registrations, 444 (81.6%) did not meet the WHO standard to post summary results within 12 months of primary study completion (the last visit of the last subject for collection of data on the primary outcome), while 386 out of 544 registrations (71.0%) failed to publish their results in a peer-reviewed journal within 24 months of primary study completion. Discrepancies were noted in the reported primary outcomes compared to the registered primary outcomes in 47.6% (222/466) of the published trials, and an even higher discordance rate of 73.2% (341/466 publications) for secondary outcomes.

CONCLUSIONS

Non-dissemination remains a significant issue in interventional malaria research, with most trials failing to meet WHO standards for timely dissemination of summary results and peer-reviewed journal publications. Additionally, outcome reporting bias is highly prevalent across malaria publications. To address these challenges, it is crucial to implement strategies that enhance the timely reporting of research findings and reduce both non-dissemination and outcome reporting bias.

How can meta-research be used to evaluate and improve the quality of research in the field of traditional, complementary, and integrative medicine?

The field of traditional, complementary, and integrative medicine (TCIM) has garnered increasing attention due to its holistic approach to health and well-being. While the quantity of published research about TCIM has increased exponentially, critics have argued that the field faces challenges related to methodological rigour, reproducibility, and overall quality. This article proposes meta-research as one approach to evaluating and improving the quality of TCIM research. Meta-research, also known as research about research, can be defined as "the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives". By systematically evaluating methodological rigour, identifying biases, and promoting transparency, meta-research can enhance the reliability and credibility of TCIM research. Specific topics of interest that are discussed in this article include the following: 1) study design and research methodology, 2) reporting of research, 3) research ethics, integrity, and misconduct, 4) replicability and reproducibility, 5) peer review and journal editorial practices, 6) research funding: grants and awards, and 7) hiring, promotion, and tenure. For each topic, we provide case examples to illustrate meta-research applications in TCIM. We argue that meta-research initiatives can contribute to maintaining public trust, safeguarding research integrity, and advancing evidence based TCIM practice, while challenges include navigating methodological complexities, biases, and disparities in funding and academic recognition. Future directions involve tailored research methodologies, interdisciplinary collaboration, policy implications, and capacity building in meta-research.

Towards full clinical trial registration and results publication: longitudinal meta-research study in Northwestern and Central Switzerland

OBJECTIVE

The registration of clinical trials is required by law in Switzerland. We investigated (1) the proportion of registered and prospectively registered clinical trials, (2) the availability of results for ethically approved trial protocols, (3) factors associated with increased registration, and (4) reasons for non-registration.

DESIGN AND SETTING

We included all clinical trials with mandatory prospective registration, which were approved by the ethics committee of Northwestern and Central Switzerland between January 1, 2016, and December 31, 2020.

METHODS

We extracted relevant trial characteristics from the Swiss Business Administration System for Ethics Committees and systematically searched the International Clinical Trials Registry Platform and primary trial registries for corresponding registry entries. We used multivariable logistic regression to examine the association between trial characteristics and registration. We qualitatively assessed reasons for non-registration of trials through an email questionnaire for trial investigators.

RESULTS

Of 473 included clinical trials, 432 (91%) were registered at all and 326 (69%) were prospectively registered. While the percentages of registration and prospective registration of investigator-sponsored trials increased from 85 to 93% and from 59 to 70% over 5 years, respectively, industry-sponsored trials consistently remained at a high level of prospective registration (92 to 100%). Trials with multiple centres, higher risk category, or methodological support from the local clinical trials unit were independently associated with increased registration rates. Of 103 clinical trials completed before August 2020, results were available for 70% of industry-sponsored trials and 45% of investigator-sponsored trials as peer-reviewed journal publications or in trial registries. Most common reasons for non-registration provided by investigators were lack of time or resources (53%), lack of knowledge (22%), and lack of reminders by the ethics committee (36%).

CONCLUSIONS

In Northwestern and Central Switzerland about 10% of clinical trials remained unregistered despite the obligation by law. More support for investigators and stricter enforcement by regulators are needed to improve the transparency of investigator-sponsored trials in particular.

Transparency of clinical trials in pancreatic cancer: An analysis of availability of trial results from the ClinicalTrials.gov database

Background

Pancreatic cancer (PC) is a highly malignant tumor of the digestive system. As clinical trials involving PC are increasingly being conducted, the transparency of the generated data has become an important issue of concern. In other areas of medicine, clinical trial transparency presents a worrying state of affairs. However, at present, there has been no study examining the transparency of data derived from PC clinical trials.

Methods

A comprehensive search was conducted in the ClinicalTrial.gov database for clinical trials investigating pancreatic cancer as of June 2022. We examined the availability of clinical trial results and recorded the characteristics of the trials.

Results

A total of 856 trials were included in this study, of which 668 were completed and 188 were terminated or suspended. The results of 626 trials (73.13%) were available, of these 230 trials (26.87%) did not disclose any information on the trial data in any form. The publication rate for trials with available results was 86.10%, but the report rate on ClinicalTrial.gov was only 39.78%.

Conclusion

Although approximately 90% of clinical trial investigating interventions on patients with PC have published study results, 30% of trials did not report any findings, and the disclosure of trial results from ClinicalTrial.gov was unsatisfactory. In general, there is still room for improvement in the transparency of PC clinical trials.

Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis

BACKGROUND

We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.

METHODS AND FINDINGS

We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.

CONCLUSIONS

We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Efficacy of telehealth in integrated chronic disease management for older, multimorbid adults with heart failure: A systematic review

BACKGROUND

We conducted a systematic review to assess the comparative efficacy of integrated telehealth versus other strategies of chronic disease management in older, multimorbid adults with heart failure (HF) in primary care and community settings. Specific efficacy outcomes included CVD-related hospitalizations, rehospitalizations, and mortality.

MATERIALS AND METHODS

We searched for randomized controlled trials (RCTs) in MEDLINE, Scopus, EMBASE, CINAHL, and CENTRAL from the date of each database's inception to January 2020. The literature search retrieved 9,181 articles, which were screened by two independent raters. Twenty-two of these articles were included in the systematic review. Data extraction, risk of bias assessment, and narrative synthesis followed article screening.

RESULTS

This systematic review found that integrated telehealth is efficacious in reducing CVD-related hospitalizations, rehospitalizations, and mortality in older, multimorbid adults within primary care and community settings. However, numerous discrepancies existed between the studies, due largely to differences in telehealth modalities and risk of bias. Overall, the combinations of modalities were so diverse that the reviewed literature did not suggest an optimal integrated telehealth strategy. At most, no more than three studies featured the same combination of telehealth modalities and outcomes. Furthermore, only 3 of the 22 included RCTs scored low on the Cochrane risk of bias tool.

CONCLUSIONS

Researchers should focus on the quality of future RCTs to better assess the efficacy of different telehealth modalities in multimorbid older adults with HF. Also, since all the included RCTs focused on HF, a knowledge gap exists with regard to the efficacy of using integrated telehealth to manage other cardiovascular diseases (CVD).

Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to SPIrit REcommendations (ASPIRE) study and associated projects

BACKGROUND

Clearly structured and comprehensive protocols are an essential component to ensure safety of participants, data validity, successful conduct, and credibility of results of randomized clinical trials (RCTs). Funding agencies, research ethics committees (RECs), regulatory agencies, medical journals, systematic reviewers, and other stakeholders rely on protocols to appraise the conduct and reporting of RCTs. In response to evidence of poor protocol quality, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline was published in 2013 to improve the accuracy and completeness of clinical trial protocols. The impact of these recommendations on protocol completeness and associations between protocol completeness and successful RCT conduct and publication remain uncertain.

OBJECTIVES AND METHODS

Aims of the Adherence to SPIrit REcommendations (ASPIRE) study are to investigate adherence to SPIRIT checklist items of RCT protocols approved by RECs in the UK, Switzerland, Germany, and Canada before (2012) and after (2016) the publication of the SPIRIT guidelines; determine protocol features associated with non-adherence to SPIRIT checklist items; and assess potential differences in adherence across countries. We assembled an international cohort of RCTs based on 450 protocols approved in 2012 and 402 protocols approved in 2016 by RECs in Switzerland, the UK, Germany, and Canada. We will extract data on RCT characteristics and adherence to SPIRIT for all included protocols. We will use multivariable regression models to investigate temporal changes in SPIRIT adherence, differences across countries, and associations between SPIRIT adherence of protocols with RCT registration, completion, and publication of results. We plan substudies to examine the registration, premature discontinuation, and non-publication of RCTs; the use of patient-reported outcomes in RCT protocols; SPIRIT adherence of RCT protocols with non-regulated interventions; the planning of RCT subgroup analyses; and the use of routinely collected data for RCTs.

DISCUSSION

The ASPIRE study and associated substudies will provide important information on the impact of measures to improve the reporting of RCT protocols and on multiple aspects of RCT design, trial registration, premature discontinuation, and non-publication of RCTs observing potential changes over time.

Legal regulations, ethical guidelines and recent policies to increase transparency of clinical trials

Timely and accurate dissemination of outcomes is essential to accomplish main benefits of scientific research including clinical trials. Clinical trial results can be disseminated in two main ways: by publication in a peer-reviewed journal and by posting on a publicly available clinical trial register. The credibility of the literature on clinical trials is significantly diminished because a high percentage of trials is not published. While current legal regulations both in the European Union (EU) and the USA impose a duty to submit summary results of clinical trials to a respective register (EU Clinical Trial Register and ClinicalTrials.gov, respectively), the compliance with this requirement has been generally inadequate. Trial outcomes can be also made accessible by data sharing. However, in spite of the wide promotion of this idea, the access of investigators to participant-level datasets remains limited. The main objective of this review is to discuss current legal regulations, international standards, ethical guidelines and recent policies pertaining to dissemination of clinical trial results.

Assessment of frequency and reporting of design changes among clinical drug trials published in influential medical journals

BACKGROUND

Modifications of primary outcome measures after trial initiation can undermine their scientific validity. We aimed to quantify these changes and to characterize potential predictors in clinical drug trials published in high impact factor general medicine journals.

METHODS

The study cohort included all prospective, adult drug clinical trials published in the New England Journal of Medicine, JAMA and Lancet between June 2017 and May 2018. The matching ClinicalTrials.gov entries effective at trial initiation were compared with those effective on July 2018, thereby identifying amendments to primary outcome definitions and assessment timeframes and the planned sample size (defined as >10% change). The primary publications were reviewed for reporting of these amendments. Associations between identified changes and trial characteristics were explored using logistic regression.

RESULTS

Of the 147 included trials, modifications to primary outcome measures were identified in 80 (54%). Primary outcome definitions, outcome assessment timeframes and the planned sample size were modified in 28 (19%), 12 (8%) and 65 (45%) of registry entries, respectively; of which 21 (75%), 11 (92%), and 33 (51%), respectively, were not reported in the related publications. There were no significant associations between modifications in registry entries and study characteristics.

CONCLUSIONS

Approximately half of trials published in influential medical journals present changes to the design while patient accrual is ongoing, and two thirds of them are unreported. Justification for such modifications should be reported more clearly. Reviewers, editors and readers should consult ClinicalTrials.gov for a more comprehensive report of the evolution of key study methods.

Selective outcome reporting in randomized clinical trials of dental implants

AIM

Selective outcome reporting (SOR) is a type of bias that occurs when the primary outcome of a trial protocol is changed or omitted in the paper. The purpose of this study was to evaluate the prevalence of SOR in publications of randomized clinical trials (RCT) concerning dental implants.

MATERIALS AND METHODS

Two reviewers independently screened protocols registered at ClinicalTrials.gov until February/2019. If the protocol met the eligibility criteria, the reviewers tried to identify the corresponding publication. Data extraction was carried out by the same reviewers. Primary and secondary outcomes were recorded for each trial and compared to outcomes previously described in protocols.

RESULTS

A total of 49 protocols were included. SOR was identified in 27 (55.1%) trials. The major discrepancies were as follows: protocol-defined primary outcome omitted in the publication (n = 6, 12.2%), new primary outcome introduced (n = 8, 16.3%), discrepancy in the primary outcome time frame (n = 17, 34.7%) and new secondary outcome introduced (n = 31, 63.3%). SOR was significantly associated with industry funding (p = 0.04) and timing of registration (p = 0.04).

CONCLUSIONS

Our results indicate a high rate of SOR in dental implants clinical trials. Use of registry data during the peer-review process may help decreasing SOR.

Registration of published randomized trials: a systematic review and meta-analysis

BACKGROUND

Prospective trial registration is a powerful tool to prevent reporting bias. We aimed to determine the extent to which published randomized controlled trials (RCTs) were registered and registered prospectively.

METHODS

We searched MEDLINE and EMBASE from January 2005 to October 2017; we also screened all articles cited by or citing included and excluded studies, and the reference lists of related reviews. We included studies that examined published RCTs and evaluated their registration status, regardless of medical specialty or language. We excluded studies that assessed RCT registration status only through mention of registration in the published RCT, without searching registries or contacting the trial investigators. Two independent reviewers blinded to the other's work performed the selection. Following PRISMA guidelines, two investigators independently extracted data, with discrepancies resolved by consensus. We calculated pooled proportions and 95% confidence intervals using random-effects models.

RESULTS

We analyzed 40 studies examining 8773 RCTs across a wide range of clinical specialties. The pooled proportion of registered RCTs was 53% (95% confidence interval 44% to 58%), with considerable between-study heterogeneity. A subset of 24 studies reported data on prospective registration across 5529 RCTs. The pooled proportion of prospectively registered RCTs was 20% (95% confidence interval 15% to 25%). Subgroup analyses showed that registration was higher for industry-supported and larger RCTs. A meta-regression analysis across 19 studies (5144 RCTs) showed that the proportion of registered trials significantly increased over time, with a mean proportion increase of 27%, from 25 to 52%, between 2005 and 2015.

CONCLUSIONS

The prevalence of trial registration has increased over time, but only one in five published RCTs is prospectively registered, undermining the validity and integrity of biomedical research.

Prospective registration trends, reasons for retrospective registration and mechanisms to increase prospective registration compliance: descriptive analysis and survey

OBJECTIVES

To analyse prospective versus retrospective trial registration trends on the Australian New Zealand Clinical Trials Registry (ANZCTR) and to evaluate the reasons for non-compliance with prospective registration.

DESIGN

Part 1: Descriptive analysis of trial registration trends from 2006 to 2015. Part 2: Online registrant survey.

PARTICIPANTS

Part 1: All interventional trials registered on ANZCTR from 2006 to 2015. Part 2: Random sample of those who had retrospectively registered a trial on ANZCTR between 2010 and 2015.

MAIN OUTCOME MEASURES

Part 1: Proportion of prospective versus retrospective clinical trial registrations (ie, registration before versus after enrolment of the first participant) on the ANZCTR overall and by various key metrics, such as sponsor, funder, recruitment country and sample size. Part 2: Reasons for non-compliance with prospective registration and perceived usefulness of various proposed mechanisms to improve prospective registration compliance.

RESULTS

Part 1: Analysis of the complete dataset of 9450 trials revealed that compliance with prospective registration increased from 48% (216 out of 446 trials) in 2006 to 63% (723/1148) in 2012 and has since plateaued at around 64%. Patterns of compliance were relatively consistent across sponsor and funder types (industry vs non-industry), type of intervention (drug vs non-drug) and size of trial (n<100, 100-500, >500). However, primary sponsors from Australia/New Zealand were almost twice as likely to register prospectively (62%; 4613/7452) compared with sponsors from other countries with a WHO Network Registry (35%; 377/1084) or sponsors from countries without a WHO Registry (29%; 230/781). Part 2: The majority (56%; 84/149) of survey respondents cited lack of awareness as a reason for not registering their study prospectively. Seventy-four per cent (111/149) stated that linking registration to ethics approval would facilitate prospective registration.

CONCLUSIONS

Despite some progress, compliance with prospective registration remains suboptimal. Linking registration to ethics approval was the favoured strategy among those sampled for improving compliance.

Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years

OBJECTIVES

We sought to evaluate the characteristics and publication fate of improperly registered clinical trials submitted to a medical journal (The BMJ) over a 4-year period to identify common types of registration issues and their relation to publication outcomes.

DESIGN

Research articles submitted to The BMJ and identified as unregistered or retrospectively registered by editors were included if they reported outcomes of a clinical trial. Relevant data regarding the trials were then extracted from each paper. Trials were categorised as prospectively registered, registered in an unapproved registry, unregistered or other, and explanations for registration deficiencies were grouped into six categories. We searched PubMed and Google to determine whether, where and when improperly registered studies were subsequently published and whether registration issues were disclosed.

RESULTS

123 research papers reporting apparently unregistered or retrospectively registered clinical trials were identified. 110 studies (89.4%) were retrospectively registered, nine (7.3%) were unregistered, three (2.4%) had been registered in an unapproved registry and one study originally lacking registration details was later discovered to have been prospectively registered. 82 studies (66.6%) were funded entirely or in part by government sources, and only seven studies (5.7%) received funding from industry. Of those papers submitted to The BMJ through the end of 2015, 67 of the 70 papers rejected for registration problems (95.7%) were subsequently published in another journal. The registration problem was disclosed in only 2 (2.9%).

CONCLUSIONS

Improper registration remains a problem, particularly for clinical trials that are government or foundation-funded. Nonetheless, improperly registered trials are almost always published, suggesting that medical journal editors may not actively enforce registration requirements.

Clinical Trials Transparency: Where Are We Today?

To prevent reporting bias, policies and regulations mandating trial prospective registration and disclosure of results have been implemented since 2005. To achieve full trial transparency, open access to participants' deidentified data policies have been issued. Funders and journals have made critical decisions on these requirements in 2017 that will impact the current scenario.

Published intimate partner violence studies often differ from their trial registration records

INTRODUCTION

Registering study protocols in a trial registry is important for methodologic transparency and reducing selective reporting bias. The objective of this investigation was to determine whether published studies of intimate partner violence (IPV) that had been registered matched the registration record on key study design elements.

METHODS

We systematically searched three trial registries to identify registered IPV studies and the published literature for the associated publication. Two authors independently determined for each study whether key study elements in the registry matched those in the published paper.

RESULTS

We included 66 studies published between 2006 and 2017. Nearly half (29/66, 44%) were registered after study completion. Many (26/66, 39%) had discrepancies regarding the primary outcome, and nearly two-thirds (42/66, 64%) had discrepancies in secondary outcomes. Discrepancies in study design were less frequent (13/66, 20%). However, large changes in sample size (26/66, 39%) and discrepancies in funding source (28/66, 42%) were frequently observed.

CONCLUSIONS

Trial registries are important tools for research transparency and identifying and preventing outcome switching and selective outcome reporting bias. Published IPV studies often differ from their records in trial registries. Researchers should pay close attention to the accuracy of trial registry records.