Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting

Use of Evidence-Based Research Approach in RCTs of Acupuncture-Related Therapies for Primary Dysmenorrhea: A Meta-Research

OBJECTIVE

To assess the use of evidence-based research (EBR) approach in randomized controlled trials (RCTs) of acupuncture-related therapies for primary dysmenorrhea (PD).

METHODS

PubMed, EMBASE, Cochrane Library, China National Knowledge Infrastructure, Wanfang Database, Chinese Biomedical Database, and China Science and Technology Journal Database were searched from January 2013 to December 2022 for RCTs of acupuncture on PD. The full text and references of each RCT were read to assess whether systematic reviews (SRs) or other types of studies with similar research questions and end-users' perspectives were cited to justify and design the trial. In addition, the discussion section were analyzed to evaluate whether trials placed the new result in the existing SRs to draw a conclusion. Multivariable logistic regression was used to find variables that associated with 3 aspects of EBR approach: (1) citing clinical studies for justification, (2) citing relevant studies that obtain the perspectives of end users, and (3) citing clinical studies for results discussion.

RESULTS

Of 473 RCTs included, 45.67% (216) of the trials cited relevant similar studies, 21.56% (102) referenced to the studies that collected end-users' perspectives, and 10.99% (52) placed result in the context of the previous research. Few RCTs appropriately applied EBR approach. Among all the included studies, 3.17% (15) of the trials used SRs to inform study questions but none of them used updated SRs with acceptable quality; 1.05% (5) of the trials cited SRs of end-user's perspectives in the justification and design of the study, and only 1 trial added results in existing SR to draw a conclusion. Year of publication, language, funding, registration, ethical approval and number of sites were significantly associated with 1 of the 3 aspects of EBR approach.

CONCLUSIONS

Few RCTs in acupuncture-related therapies for PD used the EBR approach to minimize research redundancy. Researchers, research institutes, funding agencies, ethics committees, journals and peer reviewers in acupuncture should make efforts to use and promote the EBR approach to ensure the value of new trials.

Prevalence of and factors associated with potentially redundant randomized controlled trials: a cross-sectional study

OBJECTIVES

To investigate the prevalence of meta-analyses containing potentially redundant randomized controlled trials (RCTs) and the factors associated with the presence of redundancy.

STUDY DESIGN AND SETTING

This is a cross-sectional study, based on a random sample of references (n = 4500) that were published during 2020 and 2021, indexed in PubMed, Embase, or the Cochrane Database of Systematic Reviews, and retrieved through comprehensive searches using terms about systematic reviews and meta-analysis. From each systematic review, one meta-analysis fulfilling all the following criteria, if available, was included in this study: (1) assessing the effect of the intervention on a primary outcome of the systematic review; (2) combining RCTs only. The primary outcome was prevalence of meta-analyses containing potentially redundant RCTs. Potentially redundant RCTs referred to the trials that started 1 year after the overall effect estimate from cumulative meta-analysis had been statistically robust, as determined by trial sequential analysis when appropriate. The number of potentially redundant trials (if any) in each eligible meta-analysis and the number of participants involved in those trials were documented and contrasted across groups. Logistic regression analysis was conducted to explore the factors associated with presence of potential redundancy.

RESULTS

Of the 448 eligible meta-analyses, 57 (12.7%, 95% confidence interval (CI) 9.8-16.2%) contained potentially redundant RCTs. When limited to the 333 low-heterogeneity meta-analyses, the prevalence was 17.1% (95% CI 13.5-21.5%). The total number of potentially redundant RCTs was 295 (involving 85,385 participants), accounting for 38.5% of the RCTs (and 30.3% of the participants) included in the 57 meta-analyses. In these meta-analyses, the median number of potentially redundant RCTs and the participants involved were 2 (range: 1-50) and 352 (range: 17-26997), respectively. Potentially redundant RCTs were more likely to be present in the meta-analyses evaluating pharmaceutical intervention (odds ratio [OR] 2.31, 95% CI 1.16-4.49), assessing efficacy outcomes (OR 7.25, 95% CI 0.85-61.87), containing more than 5 RCTs (OR 6.47, 95% CI 3.22-12.99), or with the earliest RCT reporting statistically significant effect estimate (OR 5.30, 95% CI 2.64-10.64).

CONCLUSION

This study found that 12.7% to 17.1% of recently published meta-analyses contained potentially redundant RCTs, highlighting the importance of conducting or examining systematic reviews of existing evidence to justify new RCTs. More importantly, the study identified some scenarios in which redundancy was more likely to occur and thus has implications for trialists, funding agencies, ethics committees, and journal editors.

Assessment of redundant randomized clinical trials among patients with ST segment elevation myocardial infarction

BACKGROUND

Redundant clinical trials waste resources and unnecessarily put patients at risk for harm. The objectives of the study were to assess redundant randomized clinical trials (RCTs) conducted in mainland China or the USA among patients with ST segment elevation myocardial infarction (STEMI) and estimate the harm to patients enrolled in redundant RCTs.

METHODS

We searched bibliographic databases for eligible RCTs comparing a routine therapy with a placebo or no treatment among patients with STEMI in mainland China or the United States. The routine therapy for STEMI included reperfusion (percutaneous coronary intervention or fibrinolytic therapy), P2Y₁₂ receptor inhibitors, statins, and anticoagulants. Redundant RCTs were defined as those initiated or continued recruiting new patients 1 year after the experimental intervention was established as routine therapy in clinical practice guidelines. Cumulative meta-analyses were conducted to confirm the efficacy of these routine therapies. The primary outcome was the number of extra major adverse cardiac events (MACEs) attributable to the deprivation of routine therapies among patients in the control groups of redundant RCTs-that is, the number of extra MACEs that could have been prevented had these patients received routine therapy.

RESULTS

Nine hundred eighty-three eligible RCTs conducted in mainland China were identified, of which 775 (78.8%) were redundant. None of the five eligible RCTs conducted in the United States were redundant. All redundant RCTs have reiterated the benefits of routine therapies for patients with STEMI, while none were cited by the 2019 clinical practice guideline for the management of STEMI. The 18,819 patients in the control groups of redundant RCTs experienced 3305 (95% CI: 3169-3441) extra MACEs, including 1091 (1014-1165) deaths, 576 (519-633) recurrent myocardial infarctions, 31 (19-42) revascularizations, 39 (23-54) strokes, 744 (679-810) heart failures, and 823 (754-893) patients with recurrent or exacerbated angina pectoris. Cumulative meta-analyses confirmed the efficacy of the routine therapies among patients in mainland China and supported using practice guidelines to define redundant RCTs.

CONCLUSIONS

Redundant RCTs conducted in mainland China have resulted in unnecessary MACEs among patients with STEMI. While the reasons behind redundant RCTs need to be further investigated, these results suggest potential research waste and violation of research ethics.

Justification of research using systematic reviews continues to be inconsistent in clinical health science-A systematic review and meta-analysis of meta-research studies

BACKGROUND

Redundancy is an unethical, unscientific, and costly challenge in clinical health research. There is a high risk of redundancy when existing evidence is not used to justify the research question when a new study is initiated. Therefore, the aim of this study was to synthesize meta-research studies evaluating if and how authors of clinical health research studies use systematic reviews when initiating a new study.

METHODS

Seven electronic bibliographic databases were searched (final search June 2021). Meta-research studies assessing the use of systematic reviews when justifying new clinical health studies were included. Screening and data extraction were performed by two reviewers independently. The primary outcome was defined as the percentage of original studies within the included meta-research studies using systematic reviews of previous studies to justify a new study. Results were synthesized narratively and quantitatively using a random-effects meta-analysis. The protocol has been registered in Open Science Framework (https://osf.io/nw7ch/).

RESULTS

Twenty-one meta-research studies were included, representing 3,621 original studies or protocols. Nineteen of the 21 studies were included in the meta-analysis. The included studies represented different disciplines and exhibited wide variability both in how the use of previous systematic reviews was assessed, and in how this was reported. The use of systematic reviews to justify new studies varied from 16% to 87%. The mean percentage of original studies using systematic reviews to justify their study was 42% (95% CI: 36% to 48%).

CONCLUSION

Justification of new studies in clinical health research using systematic reviews is highly variable, and fewer than half of new clinical studies in health science were justified using a systematic review. Research redundancy is a challenge for clinical health researchers, as well as for funders, ethics committees, and journals.

Systematic reviews are rarely used to contextualise new results-a systematic review and meta-analysis of meta-research studies

BACKGROUND

Results of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.

METHODS

We searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.

RESULTS

We included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.

CONCLUSION

Our systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.

SYSTEMATIC REVIEW REGISTRATION

Open Science registration number https://osf.io/8gkzu/.

Completeness of clinical evidence citation in trial protocols: A cross-sectional analysis

BACKGROUND

Human protection policies require assessment of how proposed clinical trials relate to prior and ongoing studies testing similar hypotheses. We assessed the extent to which clinical trial protocols cited relevant published and ongoing clinical trials that would have been easily accessible with reference searches.

METHODS

We created a random sample of trial protocols using ClinicalTrials.gov, stratifying by industry and non-industry-sponsored studies. We then conducted reference searches to determine the extent to which protocols cited clinical trials with identical intervention-indication pairings that were accessible in PubMed and ClinicalTrials.gov at the time of trial initiation.

FINDINGS

Of the 101 trial protocols evaluated, 73 had at least one identified citable trial. None contained statements suggesting a systematic search for relevant clinical evidence. Of industry-sponsored trial protocols with at least one identified citable trial, 7 of 23 (30.4%) did not cite any published clinical trials and 10 of 33 (30.3%) did not cite any ongoing relevant trials. Of the non-industry-sponsored trial protocols with at least one identified citable trial, 5 of 28 (17.9%) did not cite any published clinical trials and 14 of 19 (73.7%) did not cite any ongoing trials.

CONCLUSIONS

Clinical trial protocols undercite accessible, relevant trials and do not document systematic searches for relevant clinical trials. Consequently, ethics review committees often receive an incomplete picture of the research landscape if they review protocols similar to those deposited on ClinicalTrials.gov.

FUNDING

This study was funded by the Canadian Institutes of Health Research and the Greenwall Foundation.

Are orthodontic randomised controlled trials justified with a citation of an appropriate systematic review?

Background

A systematic review of the evidence should be undertaken to support the justification for undertaking a clinical trial. The aim of this study was to examine whether reports of orthodontic Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were also explored.

Material and methods

Orthodontic RCTs published between 1st January 2010 to 31st December 2020 in seven orthodontic journals were identified. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were assessed for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report.

Results

301 RCTs fulfilling the eligibility criteria were assessed. 220 SRs were available of which 74.5% (N = 164) were cited, and 24.5% (N = 56) were not included but were available in the literature within 12 months of trial commencement. When a SR was not included in the introduction or no SR was available within 12 months of trial commencement, interventional studies were commonly cited. The continent of the corresponding author predicated the possibility of inclusion of a SR in the introduction (OR 0.36; 95% CI 0.18–0.71; p = 0.003).

Conclusions

A quarter of orthodontic RCTs (24.5%) included in this study did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. To reduce research waste and optimal usage of resources, researchers should identify or conduct a systematic review of the evidence to support the rationale and justification of the trial.

Many Trials of Hydroxychloroquine for SARS-CoV-2 Were Redundant and Potentially Unethical: An Analysis of the NIH Clinical Trials Registry

OBJECTIVE

We sought to map the landscape of trials investigating hydroxychloroquine (HCQ) for SARS-CoV-2 in order to draw conclusions about how clinical trials have been conducted in the pandemic environment and offer potential regulatory recommendations.

STUDY DESIGN AND SETTING

We identified and captured data related to registered studies using HCQ to treat SARS-CoV-2 registered with the publicly available National Institutes of Health (NIH) Clinical Trials Registry between February and November 2020.

RESULTS

Between February and November 2020, 206 studies investigating HCQ in SARS-CoV-2 were registered with the NIH Clinical Trials Registry. As of November 2020, 135 studies were listed as ongoing, 22 have been completed, and 46 are either suspended or have been terminated. Reasons for suspension or termination included difficulties with patient recruitment (n=9), emerging evidence showing a lack of benefit of HCQ (n=7), and recommendations by regulatory boards to discontinue (n=10).

CONCLUSION

 Many clinical trials of HCQ were launched in the first months of the pandemic, and a significant proportion of them remained active as of November 2020. The medical community appears to have responded very quickly to political interest in HCQ, while responding much more slowly to the evolving medical evidence of its lack of efficacy.

Barriers and Facilitating Factors for Conducting Systematic Evidence Assessments in Academic Clinical Trials

IMPORTANCE

A systematic assessment of existing research should justify the conduct and inform the design of new clinical research but is often lacking. There is little research on the barriers to and factors facilitating systematic evidence assessments.

OBJECTIVE

To examine the practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials.

DESIGN, SETTING, AND PARTICIPANTS

In this qualitative study, individual semistructured qualitative interviews were conducted between February and August 2020 with 48 Swiss stakeholder groups (27 primary investigators, 9 funders and sponsors, 6 clinical trial support organizations, and 6 ethics committee members) and between January and March 2021 with 9 international funders of clinical trials from North America and Europe with a reputation for requiring systematic evidence synthesis in applications for academic clinical trials.

MAIN OUTCOMES AND MEASURES

The main outcomes were practices and attitudes of Swiss stakeholders and international funders regarding conducting systematic evidence assessments in academic clinical trials. Interviews were analyzed using conventional content analysis.

RESULTS

Of the 57 participants, 40 (70.2%) were male. Participants universally acknowledged that a comprehensive understanding of the previous evidence is important but reported wide variation regarding how this should be achieved. Participants reported that the conduct of formal systematic reviews was currently not expected before most clinical trials, but most international funders reported expecting a systematic search for the existing evidence. Whereas time and resources were reported by all participants as barriers to conducting systematic reviews, the Swiss research ecosystem was reported not to be as supportive of a systematic approach compared with international settings.

CONCLUSIONS AND RELEVANCE

In this qualitative study, Swiss stakeholders and international funders generally agreed that new clinical trials should be justified by a systematic evidence assessment but that barriers on individual, organizational, and political levels kept them from implementing it. More explicit requirements from funders appear to be needed to clarify the required level of comprehensiveness in summarizing existing evidence for different types of clinical trials.

Resource use during systematic review production varies widely: a scoping review

OBJECTIVE

We aimed to map the resource use during systematic review (SR) production and reasons why steps of the SR production are resource intensive to discover where the largest gain in improving efficiency might be possible.

STUDY DESIGN AND SETTING

We conducted a scoping review. An information specialist searched multiple databases (e.g., Ovid MEDLINE, Scopus) and implemented citation-based and grey literature searching. We employed dual and independent screenings of records at the title/abstract and full-text levels and data extraction.

RESULTS

We included 34 studies. Thirty-two reported on the resource use-mostly time; four described reasons why steps of the review process are resource intensive. Study selection, data extraction, and critical appraisal seem to be very resource intensive, while protocol development, literature search, or study retrieval take less time. Project management and administration required a large proportion of SR production time. Lack of experience, domain knowledge, use of collaborative and SR-tailored software, and good communication and management can be reasons why SR steps are resource intensive.

CONCLUSION

Resource use during SR production varies widely. Areas with the largest resource use are administration and project management, study selection, data extraction, and critical appraisal of studies.

The Bridge Building Model: connecting evidence-based practice, evidence-based research, public involvement and needs led research

This paper describes a model developed by an interdisciplinary team of research and public engagement specialists, with backgrounds in health and social care research, higher education, evidence-based practice, leadership, commissioning research and public involvement and engagement. The model we propose combines evidence-based practice, evidence-based research, public involvement and needs led research. Our aim is to capitalise on the joining of the rationale and methods for these approaches, which have all been highlighted as important, but for which there has been a lack of models for integration. Our ambition is to argue for and show an effective and evidence-based way of working that bridges health and social care needs identification, evidence-based practice and research.

Assessment of early exaggerated treatment effects in orthodontic interventions using cumulative meta-analysis

Background

The reported initial strong treatment effects reported in early trials that are refuted in subsequent future studies assessing the same interventions have been attributed to novelty bias. The aim of this study was to determine whether there is any evidence of novelty bias in the reported treatment effects of orthodontics interventions.

Materials and methods

Relevant orthodontic systematic review (SRs) topics containing at least one meta-analysis on either binary or continuous outcomes with a minimum of three trials considered important areas in the field of orthodontic practice were identified. SR, meta-analysis, and primary study-level characteristics were extracted. Descriptive statistics were calculated at the SRs, meta-analysis, and at the individual study level. All SR and trial-level data were imported into the statistical software and all meta-analyses were replicated using the cumulative random-effects meta-analysis approach. Changes in the size and direction of the estimates between the first trial and the cumulative effect over time were recorded.

Results

Forty-seven meta-analyses were included. The total number of primary studies included within these meta-analyses was 408 (N = 408). Overall, the final effect size estimate decreased in 29 (61.7%, N = 29/47) cumulative meta-analyses whilst it increased in the remaining 18 (38.3%, N = 18/47). No association between the level of risk of bias and the cumulative absolute effect size was evident (OR 1.00; 95% CI: 0.98, 1.03; P = 0.717) after adjusting for year of the primary study (P = 0.22).

Conclusions

Clinicians should be wary of the results of trials reporting the effectiveness of new interventions as there is a possibility that the reported effect size will be often exaggerated.

Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study

INTRODUCTION

Assessing the process used to synthesize the evidence in clinical practice guidelines enables users to determine the trustworthiness of the recommendations. Clinicians are increasingly dependent on guidelines to keep up with vast quantities of medical literature, and guidelines are followed to avoid malpractice suits. We aimed to assess whether systematic methods were used when synthesizing the evidence for guidelines; and to determine the type of review cited in support of recommendations.

METHODS

Guidelines published in 2017 and 2018 were retrieved from the TRIP and Epistemonikos databases. We randomly sorted and sequentially screened clinical guidelines on all topics to select the first 50 that met our inclusion criteria. Our primary outcomes were the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence; and the numbers of recommendations within guidelines based on different types of evidence synthesis (systematic or non-systematic reviews). If a review was cited, we looked for evidence that it was critically appraised, and recorded which quality assessment tool was used. Finally, we examined the relation between the use of the GRADE approach, systematic review process, and type of funder.

RESULTS

Of the 50 guidelines, 17 (34%) systematically synthesised the evidence to inform recommendations. These 17 guidelines clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the quality of the studies. Of the 29/50 guidelines that included reviews, 6 (21%) assessed the risk of bias of the review. The quality of primary studies was reported in 30/50 (60%) guidelines.

CONCLUSIONS

High quality, systematic review products provide the best available evidence to inform guideline recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results.

Citation of prior systematic reviews in reports of randomized controlled trials published in dental speciality journals

OBJECTIVES

To assess the extent to which reports of dental Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were explored.

METHODS

An electronic database search was undertaken to identify dental RCTs published between 1st January 2014 and 31st December 2019. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were calculated for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report.

RESULTS

682 RCTs were analysed. 312 SRs were available of which 62.5 % were cited and 37.5 % were not included but were available in the literature within 12 months of trial commencement. An association between inclusion of SR and trial registration (P = 0.046) was detected. For the inclusion of a SR, authors based in Asia or other had lower odds than those based in Europe (OR: 0.53; 95 % CI:0.34,0.82; p = 0.005). Every unit increase in journal impact factor increased the odds of SR inclusion (OR: 1.23; 95 %: 1.06, 1.43; p = 0.006).

CONCLUSIONS

A relatively high proportion of dental RCTs (37.5 %) did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. Trials conducted by a corresponding author based in Europe and published in journals with an increasing impact factor were also more likely to cite a SR.

CLINICAL SIGNIFICANCE

Further progress is required to minimise research waste and ensure resources are channelled towards clinically useful trials which have an appropriate rationale and justification.

Registered trials address questions already answered with high-certainty evidence: A sample of current redundant research

OBJECTIVE

To identify clinical trials registered later than 2015, that study the effect of an intervention on a primary outcome whose "Certainty of Evidence" (CoE) has already been rated "high" in a Cochrane SR.

STUDY DESIGN AND SETTING

We searched the Cochrane Library for all SRs from 2015. We analyzed SRs of interventions and excluded withdrawn reviews or those with no Summary of Findings (SoF) table. We retrieved the GRADE CoE ratings of each SR's primary outcomes in the SoF tables and identified those rated "high." We searched the World Health Organization International Clinical Trials Registry Platform and ClinicalTrials to identify records of clinical studies that tackled those outcomes and were registered after the date of publication of the respective 2015 SR.

RESULTS

We selected 602 SRs. Eighty-one contained a "high" CoE rating on at least one primary outcome, totaling 152 primary outcomes rated "high." We found 39 clinical trials registered for primary outcomes with evidence already rated as "high" in a 2015 Cochrane SR.

CONCLUSION

This study shows the existence of clinical trials registered to study primary outcomes whose CoE has already been rated "high" in a Cochrane SR.

Redundant trials can be prevented, if the EU clinical trial regulation is applied duly

The problem of wasteful clinical trials has been debated relentlessly in the medical community. To a significant extent, it is attributed to redundant trials - studies that are carried out to address questions, which can be answered satisfactorily on the basis of existing knowledge and accessible evidence from prior research. This article presents the first evaluation of the potential of the EU Clinical Trials Regulation 536/2014, which entered into force in 2014 but is expected to become applicable at the end of 2021, to prevent such trials. Having reviewed provisions related to the trial authorisation, we propose how certain regulatory requirements for the assessment of trial applications can and should be interpreted and applied by national research ethics committees and other relevant authorities in order to avoid redundant trials and, most importantly, preclude the unnecessary recruitment of trial participants and their unjustified exposure to health risks.

Reporting of prior clinical studies in Investigator's Brochures did not adhere to the basic principles of evidence synthesis: a cross-sectional study

OBJECTIVES

International regulations require Investigator's Brochures (IBs) to compile all available evidence that inform the risk-benefit assessment for the newly planned clinical trial. This study examined the adherence of IBs to the basic principles of evidence synthesis when compiling prior clinical studies.

STUDY DESIGN AND SETTING

For 97 IBs for phase I/II trials reviewed at one German research ethics committee we assessed the reporting on search, appraisal, and synthesis procedures for prior clinical studies. For a random subsample of 30 IBs, we evaluated the quality of reporting of the compiled 247 prior clinical studies.

RESULTS

Only 2% of all 97 IBs reported a comprehensive search strategy, provided a critical appraisal of the compiled prior clinical studies or presented respective study results in a structured manner. For the 247 prior clinical studies compiled in 30 IBs, the information required to appraise their risk of bias (eg, sample size calculation or baseline characteristics) was rarely reported.

CONCLUSION

When compiling all available evidence supporting the rationale for the proposed clinical study IBs do not acknowledge the broadly established principles for reviewing and reporting evidence. This may impact negatively on the trustworthiness and efficiency of risk-benefit assessment.

Evidence-Based Research Series-Paper 2 : Using an Evidence-Based Research approach before a new study is conducted to ensure value

BACKGROUND AND OBJECTIVES

There is considerable actual and potential waste in research. The aim of this article is to describe how using an evidence-based research approach before conducting a study helps to ensure that the new study truly adds value.

STUDY DESIGN AND SETTING

Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this second article of the evidence-based research series, we describe how to apply an evidence-based research approach before starting a new study.

RESULTS

Before a new study is performed, researchers need to provide a solid justification for it using the available scientific knowledge as well as the perspectives of end users. The key method for both is to conduct a systematic review of earlier relevant studies.

CONCLUSION

Describing the ideal process illuminates the challenges and opportunities offered through the suggested evidence-based research approach. A systematic and transparent approach is needed to provide justification for and to optimally design a relevant and necessary new study.

Evidence-Based Research Series-Paper 3: Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion

BACKGROUND AND OBJECTIVE

There is considerable actual and potential waste in research. Using evidence-based research (EBR) can ensure the value of a new study. The aim of this article, the third in a series, is to describe an EBR approach to putting research results into context.

STUDY DESIGN AND SETTING

EBR is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this third and final article of a series, we describe how to use the context of existing evidence to reach and present a trustworthy and useful conclusion when reporting results from a new clinical study.

RESULTS

We describe a method, the EBR approach, that by using a systematic and transparent consideration of earlier similar studies when interpreting and presenting results from a new original study will ensure usefulness of the conclusion.

CONCLUSION

Using an EBR approach will improve the usefulness of a clinical study by providing the context to draw more valid conclusions and explicit information about new research needs.

Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?

OBJECTIVES

There is considerable actual and potential waste in research. Evidence-based research ensures worthwhile and valuable research. The aim of this series, which this article introduces, is to describe the evidence-based research approach.

STUDY DESIGN AND SETTING

In this first article of a three-article series, we introduce the evidence-based research approach. Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner.

RESULTS

We describe evidence-based research and provide an overview of the approach of systematically and transparently using previous research before starting a new study to justify and design the new study (article #2 in series) and-on study completion-place its results in the context with what is already known (article #3 in series).

CONCLUSION

This series introduces evidence-based research as an approach to minimize unnecessary and irrelevant clinical health research that is unscientific, wasteful, and unethical.

Are randomized controlled trials being conducted with the right justification?

OBJECTIVE

It has been estimated that much of health research may be wasted, resulting in billions of dollars in wasteful research spending worldwide each year. Given the increased use of randomized trials and their influence on medicine, one method to combat research waste is to conduct randomized clinical trials (RCTs) only when a systematic review (SR) suggests more data are needed or when no previous SRs are identified. Here, we analyzed RCTs to determine whether SRs were cited as justification for conducting a trial.

METHODS

We analyzed phase III RCTs published between 2016 and 2018 in New England Journal of Medicine, Lancet, and JAMA. We performed duplicate and independent data extraction to ensure the accuracy and validity of our data. For each trial, we extracted whether SRs were cited as justification for conducting the clinical trial.

RESULTS

We examined 637 RCTs that cited 728 SRs. Overall, 38.1% (243/637) of RCTs cited an SR as either verbatim (6.9%, 44/637) or inferred (31.2%, 199/637) for trial justification. The 79 remaining RCTs cited SRs in other ways. Approximately, 49.5% (315/637) of RCTs did not cite a SR.

CONCLUSIONS

Less than half of the analyzed clinical trials cited a SRs as the basis for undertaking the trial. We believe trialists should be required to present relevant SRs to an ethics or peer review committee demonstrating an unmet need prior to initiating a trial. Eliminating research waste is both a scientific and ethical responsibility.

The use of systematic reviews to justify phase III ophthalmology trials: an analysis

BACKGROUND/OBJECTIVE

Given the drastic increase in publication output in recent years, minimizing research waste should be a top priority. There are established areas of concern regarding research waste within ophthalmology along with a lack of systematic review usage to inform trial design in other areas of medicine. Given these concerns, the aim of this study is to evaluate the use of systematic reviews as justification for conducting randomized controlled trials (RCT) in top ophthalmology and optometry journals.

METHODS

We searched PubMed on December 5, 2018 for RCTs published in one of the top five Google Scholar h-5 index journals within Ophthalmology and Optometry. We used a pilot-tested Google Form and searched each RCT for systematic reviews. Each systematic review was then given the designation of "verbatim", "inferred", or "not used as justification for conducting the RCT" based on the context the systematic review was used.

RESULTS

Our analysis yielded 152 included phase III RCTs. We found 22.4% (34 of 152) of phase III ophthalmology clinical trials cited a systematic review as justification for conducting the trial. A total of 102 systematic reviews were cited in the 152 RCTs. Fifty-seven of the one hundred fifty-two (37.5%) RCTs cited a systematic review somewhere in the manuscript.

CONCLUSION

Less than one-quarter of phase III RCTs cited systematic reviews as justification for conducting the RCT. We believe placing a higher priority on justifying RCTs with systematic reviews would go a long way to minimizing research waste within ophthalmology.

Do protocols for new randomised trials take previous similar trials into account? Cohort study of contemporary trial protocols

OBJECTIVE

To investigate to what extent evidence from previous similar trials or systematic reviews was considered before conducting new trials.

DESIGN

Cohort study of contemporary protocols for trials with ethical approval.

METHODS

All protocols for randomised trials approved by the five ethical committees in Denmark between January 2012 and March 2013 were screened for eligibility. Included protocols were read in full to determine whether a systematic search had been conducted and references were checked to evaluate whether trial rationale and design could be challenged for not adequately considering previous evidence. To investigate whether protocols cited relevant trials, we used simple search strategies that could easily be conducted by researchers without experience with literature searches.

RESULTS

Sixty-seven protocols were included. Only two (3%) of the protocols explicitly stated to have conducted a literature search and only one (1%) provided information that allowed the search to be replicated. Eleven (16%) of the protocols described trials where we found the information insufficient to judge if the trial was ethically justified, either due to a comparator that was not supported by the presented evidence (six protocols), because they did not present a rationale for conducting the trial (two protocols), or for both reasons (three protocols). For eight (12%) of the protocols, our search identified trials that could have been relevant to cite as justification.

CONCLUSIONS

While most protocols seem to adequately consider existing evidence, a substantial minority of trials might lack a sufficient evidence base. Very few trials seemed to have been based on a literature search which makes it impossible to know whether all relevant previous trials had been considered. Rules for ethical approval should include requirements for systematic literature searches to ensure that research participants are not exposed to sub-optimal treatments or unnecessary harms as well as to reduce research waste.

Practical help for specifying the target difference in sample size calculations for RCTs: the DELTA2 five-stage study, including a workshop

BACKGROUND

The randomised controlled trial is widely considered to be the gold standard study for comparing the effectiveness of health interventions. Central to its design is a calculation of the number of participants needed (the sample size) for the trial. The sample size is typically calculated by specifying the magnitude of the difference in the primary outcome between the intervention effects for the population of interest. This difference is called the 'target difference' and should be appropriate for the principal estimand of interest and determined by the primary aim of the study. The target difference between treatments should be considered realistic and/or important by one or more key stakeholder groups.

OBJECTIVE

The objective of the report is to provide practical help on the choice of target difference used in the sample size calculation for a randomised controlled trial for researchers and funder representatives.

METHODS

The Difference ELicitation in TriAls2 (DELTA2) recommendations and advice were developed through a five-stage process, which included two literature reviews of existing funder guidance and recent methodological literature; a Delphi process to engage with a wider group of stakeholders; a 2-day workshop; and finalising the core document.

RESULTS

Advice is provided for definitive trials (Phase III/IV studies). Methods for choosing the target difference are reviewed. To aid those new to the topic, and to encourage better practice, 10 recommendations are made regarding choosing the target difference and undertaking a sample size calculation. Recommended reporting items for trial proposal, protocols and results papers under the conventional approach are also provided. Case studies reflecting different trial designs and covering different conditions are provided. Alternative trial designs and methods for choosing the sample size are also briefly considered.

CONCLUSIONS

Choosing an appropriate sample size is crucial if a study is to inform clinical practice. The number of patients recruited into the trial needs to be sufficient to answer the objectives; however, the number should not be higher than necessary to avoid unnecessary burden on patients and wasting precious resources. The choice of the target difference is a key part of this process under the conventional approach to sample size calculations. This document provides advice and recommendations to improve practice and reporting regarding this aspect of trial design. Future work could extend the work to address other less common approaches to the sample size calculations, particularly in terms of appropriate reporting items.

FUNDING

Funded by the Medical Research Council (MRC) UK and the National Institute for Health Research as part of the MRC-National Institute for Health Research Methodology Research programme.

Synthesizing existing evidence to design future trials: survey of methodologists from European institutions

Background

‘Conditional trial design’ is a framework for efficiently planning new clinical trials based on a network of relevant existing trials. The framework considers whether new trials are required and how the existing evidence can be used to answer the research question and plan future research. The potential of this approach has not been fully realized.

Methods

We conducted an online survey among trial statisticians, methodologists, and users of evidence synthesis research using referral sampling to capture opinions about the conditional trial design framework and current practices among clinical researchers. The questions included in the survey were related to the decision of whether a meta-analysis answers the research question, the optimal way to synthesize available evidence, which relates to the acceptability of network meta-analysis, and the use of evidence synthesis in the planning of new studies.

Results

In total, 76 researchers completed the survey. Two out of three survey participants (65%) were willing to possibly or definitely consider using evidence synthesis to design a future clinical trial and around half of the participants would give priority to such a trial design. The median rating of the frequency of using such a trial design was 0.41 on a scale from 0 (never) to 1 (always). Major barriers to adopting conditional trial design include the current regulatory paradigm and the policies of funding agencies and sponsors.

Conclusions

Participants reported moderate interest in using evidence synthesis methods in the design of future trials. They indicated that a major paradigm shift is required before the use of network meta-analysis is regularly employed in the design of trials.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3449-6) contains supplementary material, which is available to authorized users.

Evidence-based orthodontics: Too many systematic reviews, too few trials

Why are there so many systematic reviews in orthodontics, and what are the negative consequences of this for our speciality as an academic discipline?

Independent discussion sections for improving inferential reproducibility in published research

There is a reproducibility crisis in science. There are many potential contributors to replication failure in research across the translational continuum. In this perspective piece, we focus on the narrow topic of inferential reproducibility. Although replication of methods and results is necessary to demonstrate reproducibility, it is not sufficient. Also fundamental is consistent interpretation in the Discussion section. Current deficiencies in the Discussion sections of manuscripts might limit the inferential reproducibility of scientific research. Lack of contextualisation using systematic reviews, overinterpretation and misinterpretation of results, and insufficient acknowledgement of limitations are common problems in Discussion sections; these deficiencies can harm the translational process. Proposed solutions include eliminating or not reading Discussions, writing accompanying editorials, and post-publication review and comments; however, none of these solutions works very well. A second Discussion written by an independent author with appropriate expertise in research methodology is a new testable solution that could help probe inferential reproducibility, and address some deficiencies in primary Discussion sections.

Use of available clinical evidence to extrapolate drug effects from adults to children

The extrapolation of the benefit risk ratio from adults to children is performed during drug development and often implicitly used by many paediatricians when prescribing off-label drugs in children. This is due to the specific constraints of paediatric clinical research leading to a lack of safety and efficacy data in children. Extrapolation frameworks for drug development have been proposed by several regulatory agencies. Using a meta-epidemiological approach, we explored the similarities and differences of the benefit, the benefit risk ratio and the perceived placebo effect between adults and children from meta-analyses including randomized double-blinded placebo-controlled trials evaluating a drug intervention in an indication in adults and children with separate data for both populations. We also explored the use of the effect model using adult data to predict the treatment effect in children and to calibrate future paediatric clinical trials. Our research highlights the importance of using all available evidence and quantitative methods before extrapolating the benefit risk ratio from adults to children and carrying out new studies in the context of the existing evidence. More generally, this should be applied to any research to avoid a waste of time and resources invested.

Many randomized clinical trials may not be justified: a cross-sectional analysis of the ethics and science of randomized clinical trials

OBJECTIVE

We have proposed that three scientific criteria are important for the ethical justification of randomized clinical trials (RCTs): (1) they should be designed around a clear hypothesis; (2) uncertainty should exist around that hypothesis; (3) that uncertainty should be as established through a systematic review. We hypothesized that the majority of a sample of recently published RCTs would not explicitly incorporate these criteria, therefore rendering them potentially unjustified on scientific grounds.

STUDY DESIGN AND SETTING

Cross-sectional analysis of all RCTs published in the New England Journal of Medicine and the Journal of the American Medical Association in 2015. Each article and protocol was reviewed for: (1) a clearly stated central hypothesis; (2) references to "equipoise," or "consensus;" (3) some indication of evidentiary uncertainty; (4) a meta-analysis or systematic review surrounding the hypothesis or study question.

RESULTS

We included 208 RCT articles and 199 protocols. Among combined articles and protocols, 76% had a clearly stated hypothesis, 99% referenced some form of uncertainty, and 54% cited a relevant systematic review or meta-analysis. Only 44% of combined texts contained all three scientific criteria. There were few references to "equipoise" (10%) or "consensus" (11%), and those references to equipoise were most often inconsistent with accepted definitions.

CONCLUSION

The majority of RCTs (56%) did not meet the three scientific criteria described previously and therefore may be scientifically and therefore ethically unjustified. We recommend that "equipoise," "clinical equipoise," and "lack of consensus" be abandoned as scientific criteria for RCTs and be replaced by an expectation that RCTs have a clearly stated, meaningful hypothesis around which uncertainty has been established through a systematic review of the literature.

Delayed vs early umbilical cord clamping for preterm infants: a systematic review and meta-analysis

BACKGROUND

The effects of delayed cord clamping of the umbilical cord in preterm infants are unclear.

OBJECTIVE

We sought to compare the effects of delayed vs early cord clamping on hospital mortality (primary outcome) and morbidity in preterm infants using Cochrane Collaboration neonatal review group methodology.

STUDY DESIGN

We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Chinese articles, cross-referencing citations, expert informants, and trial registries to July 31, 2017, for randomized controlled trials of delayed (≥30 seconds) vs early (<30 seconds) clamping in infants born <37 weeks' gestation. Before searching the literature, we specified that trials estimated to have cord milking in >20% of infants in any arm would be ineligible. Two reviewers independently selected studies, assessed bias, and extracted data. Relative risk (ie, risk ratio), risk difference, and mean difference with 95% confidence intervals were assessed by fixed effects models, heterogeneity by I² statistics, and the quality of evidence by Grading of Recommendations, Assessment, Development, and Evaluations.

RESULTS

Eighteen randomized controlled trials compared delayed vs early clamping in 2834 infants. Most infants allocated to have delayed clamping were assigned a delay of ≥60 seconds. Delayed clamping reduced hospital mortality (risk ratio, 0.68; 95% confidence interval, 0.52-0.90; risk difference, -0.03; 95% confidence interval, -0.05 to -0.01; P = .005; number needed to benefit, 33; 95% confidence interval, 20-100; Grading of Recommendations, Assessment, Development, and Evaluations = high, with I² = 0 indicating no heterogeneity). In 3 trials in 996 infants ≤28 weeks' gestation, delayed clamping reduced hospital mortality (risk ratio, 0.70; 95% confidence interval, 0.51-0.95; risk difference, -0.05; 95% confidence interval, -0.09 to -0.01; P = .02, number needed to benefit, 20; 95% confidence interval, 11-100; I² = 0). In subgroup analyses, delayed clamping reduced the incidence of low Apgar score at 1 minute, but not at 5 minutes, and did not reduce the incidence of intubation for resuscitation, admission temperature, mechanical ventilation, intraventricular hemorrhage, brain injury, chronic lung disease, patent ductus arteriosus, necrotizing enterocolitis, late onset sepsis or retinopathy of prematurity. Delayed clamping increased peak hematocrit by 2.73 percentage points (95% confidence interval, 1.94-3.52; P < .00001) and reduced the proportion of infants having blood transfusion by 10% (95% confidence interval, 6-13%; P < .00001). Potential harms of delayed clamping included polycythemia and hyperbilirubinemia.

CONCLUSION

This systematic review provides high-quality evidence that delayed clamping reduced hospital mortality, which supports current guidelines recommending delayed clamping in preterm infants. This review does not evaluate cord milking, which may also be of benefit. Analyses of individual patient data in these and other randomized controlled trials will be critically important in reliably evaluating important secondary outcomes.

Using Bayesian statistics to estimate the likelihood a new trial will demonstrate the efficacy of a new treatment

Background

The common frequentist approach is limited in providing investigators with appropriate measures for conducting a new trial. To answer such important questions and one has to look at Bayesian statistics.

Methods

As a worked example, we conducted a Bayesian cumulative meta-analysis to summarize the benefit of patient-specific instrumentation on the alignment of total knee replacement from previously published evidence. Data were sourced from Medline, Embase, and Cochrane databases. All randomised controlled comparisons of the effect of patient-specific instrumentation on the coronal alignment of total knee replacement were included. The main outcome was the risk difference measured by the proportion of failures in the control group minus the proportion of failures in the experimental group. Through Bayesian statistics, we estimated cumulatively over publication time of the trial results: the posterior probabilities that the risk difference was more than 5 and 10%; the posterior probabilities that given the results of all previous published trials an additional fictive trial would achieve a risk difference of at least 5%; and the predictive probabilities that observed failure rate differ from 5% across arms.

Results

Thirteen trials were identified including 1092 patients, 554 in the experimental group and 538 in the control group. The cumulative mean risk difference was 0.5% (95% CrI: −5.7%; +4.5%). The posterior probabilities that the risk difference be superior to 5 and 10% was less than 5% after trial #4 and trial #2 respectively. The predictive probability that the difference in failure rates was at least 5% dropped from 45% after the first trial down to 11% after the 13th. Last, only unrealistic trial design parameters could change the overall evidence accumulated to date.

Conclusions

Bayesian probabilities are readily understandable when discussing the relevance of performing a new trial. It provides investigators the current probability that an experimental treatment be superior to a reference treatment. In case a trial is designed, it also provides the predictive probability that this new trial will reach the targeted risk difference in failure rates.

Trial registration

CRD42015024176.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0401-x) contains supplementary material, which is available to authorized users.

An increasing problem in publication ethics: Publication bias and editors' role in avoiding it

Publication bias is defined as "the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or the strength of the study findings."Publication bias distorts the accumulated data in the literature, causes the over estimation of potential benefits of intervention and mantles the risks and adverse effects, and creates a barrier to assessing the clinical utility of drugs as well as evaluating the long-term safety of medical interventions. The World Medical Association, the International Committee of Medical Journals, and the Committee on Publication Ethics have conferred responsibilities and ethical obligations to editors concerning the avoidance of publication bias. Despite the explicit statements in these international documents, the editors' role in and ability to avoid publication bias is still being discussed. Unquestionably, all parties involved in clinical research have the ultimate responsibility to sustain the research integrity and validity of accumulated general knowledge. Cooperation and commitment is required at every step of a clinical trial. However, this holistic approach does not exclude effective measures to be taken at the editors' level. The editors of major medical journals concluded that one precaution that editors can take is to mandate registration of all clinical trials in a public repository as a precondition to submitting manuscripts to journals. Raising awareness regarding the value of publishing negative data for the scientific community and human health, and increasing the number of journals that are dedicated to publishing negative results or that set aside a section in their pages to do so, are positive steps editors can take to avoid publication bias.

Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research

BACKGROUND

A meta-analysis as part of a systematic review aims to provide a thorough, comprehensive and unbiased statistical summary of data from the literature. However, relevant study results could be missing from a meta-analysis because of selective publication and inadequate dissemination. If missing outcome data differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention effect. As part of the EU-funded OPEN project (www.open-project.eu) we conducted a systematic review that assessed whether the inclusion of data that were not published at all and/or published only in the grey literature influences pooled effect estimates in meta-analyses and leads to different interpretation.

METHODS AND FINDINGS

Systematic review of published literature (methodological research projects). Four bibliographic databases were searched up to February 2016 without restriction of publication year or language. Methodological research projects were considered eligible for inclusion if they reviewed a cohort of meta-analyses which (i) compared pooled effect estimates of meta-analyses of health care interventions according to publication status of data or (ii) examined whether the inclusion of unpublished or grey literature data impacts the result of a meta-analysis. Seven methodological research projects including 187 meta-analyses comparing pooled treatment effect estimates according to different publication status were identified. Two research projects showed that published data showed larger pooled treatment effects in favour of the intervention than unpublished or grey literature data (Ratio of ORs 1.15, 95% CI 1.04-1.28 and 1.34, 95% CI 1.09-1.66). In the remaining research projects pooled effect estimates and/or overall findings were not significantly changed by the inclusion of unpublished and/or grey literature data. The precision of the pooled estimate was increased with narrower 95% confidence interval.

CONCLUSIONS

Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data.

Trial Sequential Analysis in systematic reviews with meta-analysis

BACKGROUND

Most meta-analyses in systematic reviews, including Cochrane ones, do not have sufficient statistical power to detect or refute even large intervention effects. This is why a meta-analysis ought to be regarded as an interim analysis on its way towards a required information size. The results of the meta-analyses should relate the total number of randomised participants to the estimated required meta-analytic information size accounting for statistical diversity. When the number of participants and the corresponding number of trials in a meta-analysis are insufficient, the use of the traditional 95% confidence interval or the 5% statistical significance threshold will lead to too many false positive conclusions (type I errors) and too many false negative conclusions (type II errors).

METHODS

We developed a methodology for interpreting meta-analysis results, using generally accepted, valid evidence on how to adjust thresholds for significance in randomised clinical trials when the required sample size has not been reached.

RESULTS

The Lan-DeMets trial sequential monitoring boundaries in Trial Sequential Analysis offer adjusted confidence intervals and restricted thresholds for statistical significance when the diversity-adjusted required information size and the corresponding number of required trials for the meta-analysis have not been reached. Trial Sequential Analysis provides a frequentistic approach to control both type I and type II errors. We define the required information size and the corresponding number of required trials in a meta-analysis and the diversity (D²) measure of heterogeneity. We explain the reasons for using Trial Sequential Analysis of meta-analysis when the actual information size fails to reach the required information size. We present examples drawn from traditional meta-analyses using unadjusted naïve 95% confidence intervals and 5% thresholds for statistical significance. Spurious conclusions in systematic reviews with traditional meta-analyses can be reduced using Trial Sequential Analysis. Several empirical studies have demonstrated that the Trial Sequential Analysis provides better control of type I errors and of type II errors than the traditional naïve meta-analysis.

CONCLUSIONS

Trial Sequential Analysis represents analysis of meta-analytic data, with transparent assumptions, and better control of type I and type II errors than the traditional meta-analysis using naïve unadjusted confidence intervals.

The pathway to RCTs: how many roads are there? Examining the homogeneity of RCT justification

Background

Randomized controlled trials (RCTs) form the foundational background of modern medical practice. They are considered the highest quality of evidence, and their results help inform decisions concerning drug development and use, preventive therapies, and screening programs. However, the inputs that justify an RCT to be conducted have not been studied.

Methods

We reviewed the MEDLINE and EMBASE databases across six specialties (Ophthalmology, Otorhinolaryngology (ENT), General Surgery, Psychiatry, Obstetrics-Gynecology (OB-GYN), and Internal Medicine) and randomly chose 25 RCTs from each specialty except for Otorhinolaryngology (20 studies) and Internal Medicine (28 studies). For each RCT, we recorded information relating to the justification for conducting RCTs such as average study size cited, number of studies cited, and types of studies cited. The justification varied widely both within and between specialties.

Results

For Ophthalmology and OB-GYN, the average study sizes cited were around 1100 patients, whereas they were around 500 patients for Psychiatry and General Surgery. Between specialties, the average number of studies cited ranged from around 4.5 for ENT to around 10 for Ophthalmology, but the standard deviations were large, indicating that there was even more discrepancy within each specialty. When standardizing by the sample size of the RCT, some of the discrepancies between and within specialties can be explained, but not all. On average, Ophthalmology papers cited review articles the most (2.96 studies per RCT) compared to less than 1.5 studies per RCT for all other specialties.

Conclusions

The justifications for RCTs vary widely both within and between specialties, and the justification for conducting RCTs is not standardized.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-017-1804-z) contains supplementary material, which is available to authorized users.