Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents?

Use of Evidence-Based Research Approach in RCTs of Acupuncture-Related Therapies for Primary Dysmenorrhea: A Meta-Research

OBJECTIVE

To assess the use of evidence-based research (EBR) approach in randomized controlled trials (RCTs) of acupuncture-related therapies for primary dysmenorrhea (PD).

METHODS

PubMed, EMBASE, Cochrane Library, China National Knowledge Infrastructure, Wanfang Database, Chinese Biomedical Database, and China Science and Technology Journal Database were searched from January 2013 to December 2022 for RCTs of acupuncture on PD. The full text and references of each RCT were read to assess whether systematic reviews (SRs) or other types of studies with similar research questions and end-users' perspectives were cited to justify and design the trial. In addition, the discussion section were analyzed to evaluate whether trials placed the new result in the existing SRs to draw a conclusion. Multivariable logistic regression was used to find variables that associated with 3 aspects of EBR approach: (1) citing clinical studies for justification, (2) citing relevant studies that obtain the perspectives of end users, and (3) citing clinical studies for results discussion.

RESULTS

Of 473 RCTs included, 45.67% (216) of the trials cited relevant similar studies, 21.56% (102) referenced to the studies that collected end-users' perspectives, and 10.99% (52) placed result in the context of the previous research. Few RCTs appropriately applied EBR approach. Among all the included studies, 3.17% (15) of the trials used SRs to inform study questions but none of them used updated SRs with acceptable quality; 1.05% (5) of the trials cited SRs of end-user's perspectives in the justification and design of the study, and only 1 trial added results in existing SR to draw a conclusion. Year of publication, language, funding, registration, ethical approval and number of sites were significantly associated with 1 of the 3 aspects of EBR approach.

CONCLUSIONS

Few RCTs in acupuncture-related therapies for PD used the EBR approach to minimize research redundancy. Researchers, research institutes, funding agencies, ethics committees, journals and peer reviewers in acupuncture should make efforts to use and promote the EBR approach to ensure the value of new trials.

SARS-CoV-2 Genomic Surveillance in Brazil: A Systematic Review with Scientometric Analysis

Several studies have monitored the SARS-CoV-2 variants in Brazil throughout the pandemic. Here, we systematically reviewed and conducted a scientometric analysis of the SARS-CoV-2 genomic surveillance studies using Brazilian samples. A Pubmed database search on October 2022 returned 492 articles, of which 106 were included. Ninety-six different strains were reported, with variant of concern (VOC) gamma (n = 35,398), VOC delta (n = 15,780), and the variant of interest zeta (n = 1983) being the most common. The top three states with the most samples in the published articles were São Paulo, Rio de Janeiro, and Minas Gerais. Whereas the first year of the pandemic presented primary circulation of B.1.1.28 and B.1.1.33 variants, consecutive replacements were observed between them and VOI zeta, VOC gamma, VOC delta, and VOC omicron. VOI mu, VOI lambda, VOC alpha, and VOC beta were also detected but failed to reach significant circulation. Co-infection, re-infection, and vaccine breakthrough reports were found. Article co-citation differed from the co-authorship structure. Despite the limitations, we expect to give an overview of Brazil's genomic surveillance studies and contribute to future research execution.

Meta-research evaluating redundancy and use of systematic reviews when planning new studies in health research: a scoping review

BACKGROUND

Several studies have documented the production of wasteful research, defined as research of no scientific importance and/or not meeting societal needs. We argue that this redundancy in research may to a large degree be due to the lack of a systematic evaluation of the best available evidence and/or of studies assessing societal needs.

OBJECTIVES

The aim of this scoping review is to (A) identify meta-research studies evaluating if redundancy is present within biomedical research, and if so, assessing the prevalence of such redundancy, and (B) to identify meta-research studies evaluating if researchers had been trying to minimise or avoid redundancy.

ELIGIBILITY CRITERIA

Meta-research studies (empirical studies) were eligible if they evaluated whether redundancy was present and to what degree; whether health researchers referred to all earlier similar studies when justifying and designing a new study and/or when placing new results in the context of earlier similar trials; and whether health researchers systematically and transparently considered end users' perspectives when justifying and designing a new study.

SOURCES OF EVIDENCE

The initial overall search was conducted in MEDLINE, Embase via Ovid, CINAHL, Web of Science, Social Sciences Citation Index, Arts & Humanities Citation Index, and the Cochrane Methodology Register from inception to June 2015. A 2nd search included MEDLINE and Embase via Ovid and covered January 2015 to 26 May 2021. No publication date or language restrictions were applied.

CHARTING METHODS

Charting methods included description of the included studies, bibliometric mapping, and presentation of possible research gaps in the identified meta-research.

RESULTS

We identified 69 meta-research studies. Thirty-four (49%) of these evaluated the prevalence of redundancy and 42 (61%) studies evaluated the prevalence of a systematic and transparent use of earlier similar studies when justifying and designing new studies, and/or when placing new results in context, with seven (10%) studies addressing both aspects. Only one (1%) study assessed if the perspectives of end users had been used to inform the justification and design of a new study. Among the included meta-research studies evaluating whether redundancy was present, only two of nine health domains (medical areas) and only two of 10 research topics (different methodological types) were represented. Similarly, among the included meta-research studies evaluating whether researchers had been trying to minimise or avoid redundancy, only one of nine health domains and only one of 10 research topics were represented.

CONCLUSIONS THAT RELATE TO THE REVIEW QUESTIONS AND OBJECTIVES

Even with 69 included meta-research studies, there was a lack of information for most health domains and research topics. However, as most included studies were evaluating across different domains, there is a clear indication of a high prevalence of redundancy and a low prevalence of trying to minimise or avoid redundancy. In addition, only one meta-research study evaluated whether the perspectives of end users were used to inform the justification and design of a new study.

SYSTEMATIC REVIEW REGISTRATION

Protocol registered at Open Science Framework: https://osf.io/3rdua/ (15 June 2021).

Systematic reviews are rarely used to contextualise new results-a systematic review and meta-analysis of meta-research studies

BACKGROUND

Results of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.

METHODS

We searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.

RESULTS

We included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.

CONCLUSION

Our systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.

SYSTEMATIC REVIEW REGISTRATION

Open Science registration number https://osf.io/8gkzu/.

Deploying social robots in museum settings: A quasi-systematic review exploring purpose and acceptability

Social robots are increasingly used within public spaces, including museum settings. This quasi-systematic review identifies and synthesizes the evidence on social robots that have recently been deployed in museum settings. It specifically focuses on their intended purpose, their acceptability and factors important for successful human–robot interaction in this setting. Four databases (PsycINFO, SCOPUS, ACM Digital Library and IEEE Xplore) were systematically searched to retrieve literature published within the last 10 years on human–robot interaction studies with social robots deployed in museum settings. Due to the heterogeneous nature of the studies, qualitative and quantitative findings were summarized. A total of 604 items were identified, of which 12 were included in the review. Robots in 11 studies were physical and 1 was an embodied conversational agent presented as a virtual robot. In 75% of the studies ( n = 9), the purpose of the robots was to act as museum guides, while in 17% ( n = 2) they entertained visitors and in 8% ( n = 1) the robot taught visitors in a museum outreach programme. Overall, many of the robots were found to be acceptable for use within museum settings. Three main themes for successful social human–robot interaction were evident across the findings: (1) facial expressions, (2) movement and (3) communication and speech. There is a great opportunity for social robots to be deployed within museum settings, as guides, educators, entertainers or a combination thereof. State-of-the-art methods have led to the development of museum robots that are more capable of social interaction; however, more work is required to develop speech capabilities that work in the ‘wild’. Future work should combine the factors that have been identified within this review to improve human–robot interaction.

Are orthodontic randomised controlled trials justified with a citation of an appropriate systematic review?

Background

A systematic review of the evidence should be undertaken to support the justification for undertaking a clinical trial. The aim of this study was to examine whether reports of orthodontic Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were also explored.

Material and methods

Orthodontic RCTs published between 1st January 2010 to 31st December 2020 in seven orthodontic journals were identified. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were assessed for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report.

Results

301 RCTs fulfilling the eligibility criteria were assessed. 220 SRs were available of which 74.5% (N = 164) were cited, and 24.5% (N = 56) were not included but were available in the literature within 12 months of trial commencement. When a SR was not included in the introduction or no SR was available within 12 months of trial commencement, interventional studies were commonly cited. The continent of the corresponding author predicated the possibility of inclusion of a SR in the introduction (OR 0.36; 95% CI 0.18–0.71; p = 0.003).

Conclusions

A quarter of orthodontic RCTs (24.5%) included in this study did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. To reduce research waste and optimal usage of resources, researchers should identify or conduct a systematic review of the evidence to support the rationale and justification of the trial.

Citation of prior systematic reviews in reports of randomized controlled trials published in dental speciality journals

OBJECTIVES

To assess the extent to which reports of dental Randomised Clinical Trials (RCTs) cite prior systematic reviews (SR) to explain the rationale or justification of the trial. Study characteristics that predicated the citation of SR in the RCT report were explored.

METHODS

An electronic database search was undertaken to identify dental RCTs published between 1st January 2014 and 31st December 2019. All titles and abstracts were screened independently by two authors. Descriptive statistics and associations were calculated for the study characteristics. Logistic regression was used to identify predicators of SR inclusion in the trial report.

RESULTS

682 RCTs were analysed. 312 SRs were available of which 62.5 % were cited and 37.5 % were not included but were available in the literature within 12 months of trial commencement. An association between inclusion of SR and trial registration (P = 0.046) was detected. For the inclusion of a SR, authors based in Asia or other had lower odds than those based in Europe (OR: 0.53; 95 % CI:0.34,0.82; p = 0.005). Every unit increase in journal impact factor increased the odds of SR inclusion (OR: 1.23; 95 %: 1.06, 1.43; p = 0.006).

CONCLUSIONS

A relatively high proportion of dental RCTs (37.5 %) did not cite a SR in the introduction section to justify the rationale of the trial when a relevant SR was available. Trials conducted by a corresponding author based in Europe and published in journals with an increasing impact factor were also more likely to cite a SR.

CLINICAL SIGNIFICANCE

Further progress is required to minimise research waste and ensure resources are channelled towards clinically useful trials which have an appropriate rationale and justification.

Evidence-Based Research Series-Paper 3: Using an Evidence-Based Research approach to place your results into context after the study is performed to ensure usefulness of the conclusion

BACKGROUND AND OBJECTIVE

There is considerable actual and potential waste in research. Using evidence-based research (EBR) can ensure the value of a new study. The aim of this article, the third in a series, is to describe an EBR approach to putting research results into context.

STUDY DESIGN AND SETTING

EBR is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. In this third and final article of a series, we describe how to use the context of existing evidence to reach and present a trustworthy and useful conclusion when reporting results from a new clinical study.

RESULTS

We describe a method, the EBR approach, that by using a systematic and transparent consideration of earlier similar studies when interpreting and presenting results from a new original study will ensure usefulness of the conclusion.

CONCLUSION

Using an EBR approach will improve the usefulness of a clinical study by providing the context to draw more valid conclusions and explicit information about new research needs.

Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?

OBJECTIVES

There is considerable actual and potential waste in research. Evidence-based research ensures worthwhile and valuable research. The aim of this series, which this article introduces, is to describe the evidence-based research approach.

STUDY DESIGN AND SETTING

In this first article of a three-article series, we introduce the evidence-based research approach. Evidence-based research is the use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner.

RESULTS

We describe evidence-based research and provide an overview of the approach of systematically and transparently using previous research before starting a new study to justify and design the new study (article #2 in series) and-on study completion-place its results in the context with what is already known (article #3 in series).

CONCLUSION

This series introduces evidence-based research as an approach to minimize unnecessary and irrelevant clinical health research that is unscientific, wasteful, and unethical.

The use of systematic reviews to justify orthopaedic trauma randomized controlled trials: A cross-sectional analysis

OBJECTIVE

Orthopaedic trauma is one of the largest surgical fields in medicine, and as such, requires the latest evidence to ensure the best standard of care. Systematic reviews are an invaluable resource that compiles an exhaustive summary of the most current evidence on a given clinical question. The primary aim of this study is to evaluate the use of systematic reviews as justification in conducting randomized controlled trials published in high impact orthopaedic trauma journals.

METHODS

We analyzed randomized controlled trials published in the top three high impact orthopaedic trauma journals between 2015 and 2018. We performed data extraction blind, independent, and in duplicative manner to ensure the validity of the findings. For each trial, data was extracted by the number of systematic reviews cited in each clinical trial and whether or not the study used a systematic review as justification for conducting the trial. A subgroup of general orthopaedic clinical trials were included for comparison.

RESULTS

Of 144 articles retrieved, 128 were included. Overall, 71.1% (91/128; [95% CI, 63.2-78.9]) of included orthopaedic trauma randomized controlled trials referenced a systematic review and 28.9% (37/128) of studies did not cite a systematic review. Of the 91 trials that referenced a systematic review, 33.0% (30/91; [95% CI, 23.3-42.6]) of RCTs cited a systematic review as trial justification, whether that be "verbatim" or "inferred". "Verbatim" justifications occurred in 20.0% (6/30; [95% CI, 5.7-34.3]) of included trauma RCTs that cited a systematic review as justification for conducting the trial and "inferred" justifications occurred in 80.0% (24/30; [95% CI, 65.7-94.3]).

CONCLUSION

Systematic reviews are frequently cited in orthopaedic trauma RCTs but are not commonly cited as justification for conducting a clinical trial. Ideally, evidentiary uncertainty regarding a research question should be established by existing literature through a systematic review to reduce research waste.

Evidence maps: a tool to guide research agenda setting

A surprisingly huge proportion of medical research still shows poor quality in design, conduct and analysis, leading to far from optimal robustness of findings and validity of conclusions. Research waste remains a problem caused by a number of reasons. Asking the wrong research questions and ignoring the existing evidence are possible preventable ones. Evidence maps are tools that may aid in guiding clinical investigators and help in agenda setting of future research. In this article, we explain how they serve such a goal and outline the steps required to build effective evidence maps.

Do protocols for new randomised trials take previous similar trials into account? Cohort study of contemporary trial protocols

OBJECTIVE

To investigate to what extent evidence from previous similar trials or systematic reviews was considered before conducting new trials.

DESIGN

Cohort study of contemporary protocols for trials with ethical approval.

METHODS

All protocols for randomised trials approved by the five ethical committees in Denmark between January 2012 and March 2013 were screened for eligibility. Included protocols were read in full to determine whether a systematic search had been conducted and references were checked to evaluate whether trial rationale and design could be challenged for not adequately considering previous evidence. To investigate whether protocols cited relevant trials, we used simple search strategies that could easily be conducted by researchers without experience with literature searches.

RESULTS

Sixty-seven protocols were included. Only two (3%) of the protocols explicitly stated to have conducted a literature search and only one (1%) provided information that allowed the search to be replicated. Eleven (16%) of the protocols described trials where we found the information insufficient to judge if the trial was ethically justified, either due to a comparator that was not supported by the presented evidence (six protocols), because they did not present a rationale for conducting the trial (two protocols), or for both reasons (three protocols). For eight (12%) of the protocols, our search identified trials that could have been relevant to cite as justification.

CONCLUSIONS

While most protocols seem to adequately consider existing evidence, a substantial minority of trials might lack a sufficient evidence base. Very few trials seemed to have been based on a literature search which makes it impossible to know whether all relevant previous trials had been considered. Rules for ethical approval should include requirements for systematic literature searches to ensure that research participants are not exposed to sub-optimal treatments or unnecessary harms as well as to reduce research waste.

Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks

BACKGROUND

A standard or consensus definition of a systematic review does not exist. Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition.

METHODS

We included overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks. We extracted the definitions of SRs, as well as the inclusion and exclusion criteria that could indicate which definition of a SR the authors used. We extracted individual elements of SR definitions, categorised and quantified them.

RESULTS

Among the 535 analysed sources of information, 188 (35%) provided a definition of a SR. The most commonly used reference points for the definitions of SRs were Cochrane and the PRISMA statement. We found 188 different elements of SR definitions and divided them into 14 categories. The highest number of SR definition elements was found in categories related to searching (N = 51), analysis/synthesis (N = 23), overall methods (N = 22), quality/bias/appraisal/validity (N = 22) and aim/question (N = 13). The same five categories were also the most commonly used combination of categories in the SR definitions.

CONCLUSION

Currently used definitions of SRs are vague and ambiguous, often using terms such as clear, explicit and systematic, without further elaboration. In this manuscript we propose a more specific definition of a systematic review, with the ultimate aim of motivating the research community to establish a clear and unambiguous definition of this type of research.

Synthesizing existing evidence to design future trials: survey of methodologists from European institutions

Background

‘Conditional trial design’ is a framework for efficiently planning new clinical trials based on a network of relevant existing trials. The framework considers whether new trials are required and how the existing evidence can be used to answer the research question and plan future research. The potential of this approach has not been fully realized.

Methods

We conducted an online survey among trial statisticians, methodologists, and users of evidence synthesis research using referral sampling to capture opinions about the conditional trial design framework and current practices among clinical researchers. The questions included in the survey were related to the decision of whether a meta-analysis answers the research question, the optimal way to synthesize available evidence, which relates to the acceptability of network meta-analysis, and the use of evidence synthesis in the planning of new studies.

Results

In total, 76 researchers completed the survey. Two out of three survey participants (65%) were willing to possibly or definitely consider using evidence synthesis to design a future clinical trial and around half of the participants would give priority to such a trial design. The median rating of the frequency of using such a trial design was 0.41 on a scale from 0 (never) to 1 (always). Major barriers to adopting conditional trial design include the current regulatory paradigm and the policies of funding agencies and sponsors.

Conclusions

Participants reported moderate interest in using evidence synthesis methods in the design of future trials. They indicated that a major paradigm shift is required before the use of network meta-analysis is regularly employed in the design of trials.

Electronic supplementary material

The online version of this article (10.1186/s13063-019-3449-6) contains supplementary material, which is available to authorized users.

Independent discussion sections for improving inferential reproducibility in published research

There is a reproducibility crisis in science. There are many potential contributors to replication failure in research across the translational continuum. In this perspective piece, we focus on the narrow topic of inferential reproducibility. Although replication of methods and results is necessary to demonstrate reproducibility, it is not sufficient. Also fundamental is consistent interpretation in the Discussion section. Current deficiencies in the Discussion sections of manuscripts might limit the inferential reproducibility of scientific research. Lack of contextualisation using systematic reviews, overinterpretation and misinterpretation of results, and insufficient acknowledgement of limitations are common problems in Discussion sections; these deficiencies can harm the translational process. Proposed solutions include eliminating or not reading Discussions, writing accompanying editorials, and post-publication review and comments; however, none of these solutions works very well. A second Discussion written by an independent author with appropriate expertise in research methodology is a new testable solution that could help probe inferential reproducibility, and address some deficiencies in primary Discussion sections.

Citation of prior research has increased in introduction and discussion sections with time: A survey of clinical trials in physiotherapy

BACKGROUND/AIMS

Many clinical trials are reported without reference to the existing relevant high-quality research. This study aimed to investigate the extent to which authors of reports of clinical trials of physiotherapy interventions try to use high-quality clinical research to (1) help justify the need for the trial in the introduction and (2) help interpret the trial's results in the discussion.

METHODS

Data were extracted from 221 clinical trials that were randomly selected from the Physiotherapy Evidence Database: 70 published in 2001 (10% sample) and 151 published in 2015 (10% sample). The Physiotherapy Evidence Database score (which rates methodological quality and completeness of reporting) for each trial was also downloaded.

RESULTS

Overall 41% of trial reports cited a systematic review or the results of a search for other evidence in the introduction section: 20% for 2001 and 50% for 2015 (relative risk = 2.3, 95% confidence interval = 1.5-3.8). For the discussion section, only 1 of 221 trials integrated the results of the trial into an existing meta-analysis, but citation of a relevant systematic review did increase from 17% in 2001 to 34% in 2015. There was no relationship between citation of existing research and the total Physiotherapy Evidence Database score.

CONCLUSION

Published reports of clinical trials of physiotherapy interventions increasingly cite a systematic review or the results of a search for other evidence in the introduction, but integration with existing research in the discussion section is very rare. To encourage the use of existing research, stronger recommendations to refer to existing systematic reviews (where available) could be incorporated into reporting checklists and journal editorial guidelines.

The citation of relevant systematic reviews and randomised trials in published reports of trial protocols

BACKGROUND

It is important that planned randomised trials are justified and placed in the context of the available evidence. The SPIRIT guidelines for reporting clinical trial protocols recommend that a recent and relevant systematic review should be included. The aim of this study was to assess the use of the existing evidence in order to justify trial conduct.

METHODS

Protocols of randomised trials published over a 1-month period (December 2015) indexed in PubMed were obtained. Data on trial characteristics relating to location, design, funding, conflict of interest and type of evidence included for trial justification was extracted in duplicate and independently by two investigators. The frequency of citation of previous research including relevant systematic reviews and randomised trials was assessed.

RESULTS

Overall, 101 protocols for RCTs were identified. Most proposed trials were parallel-group (n = 74; 73.3%). Reference to an earlier systematic review with additional randomised trials was found in 9.9% (n = 10) of protocols and without additional trials in 30.7% (n = 31), while reference was made to randomised trials in isolation in 21.8% (n = 22). Explicit justification for the proposed randomised trial on the basis of being the first to address the research question was made in 17.8% (n = 18) of protocols. A randomised controlled trial was not cited in 10.9% (95% CI: 5.6, 18.7) (n = 11), while in 8.9% (95% CI: 4.2, 16.2) (n = 9) of the protocols a systematic review was cited but did not inform trial design.

CONCLUSIONS

A relatively high percentage of protocols of randomised trials involves prior citation of randomised trials, systematic reviews or both. However, improvements are required to ensure that it is explicit that clinical trials are justified and shaped by contemporary best evidence.

Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding?

Background Aprotinin is a serine protease inhibitor used to limit perioperative bleeding and reduce the need for donated blood transfusions during cardiac surgery. Randomized controlled trials of aprotinin evaluating its effect on the outcome of perioperative transfusion have been published since 1987, and systematic reviews were conducted in 1992 and 1997.

Methods A systematic search was conducted for all RCTs of aprotinin that used placebo controls or were open-label with no active control treatment. Data collected included the primary outcome, objective of each study, whether a systematic review was cited or conducted as part of the background and/or rationale for the study and the number of previously published RCTs cited. Cumulative meta-analyses were performed.

Results Sixty-four randomized, controlled trials of aprotinin were found, conducted between 1987 and 2002, reporting an endpoint of perioperative transfusion. Median trial size was 64 subjects, with a range of 20 to 1784. A cumulative meta-analysis indicated that aprotinin greatly decreased the need for perioperative transfusion, stabilizing at an odds ratio of 0.25 (p, 10 2 6) by the 12th study, published in June of 1992. The upper limit of the confidence interval never exceeded 0.65 and results were similar in all subgroups. Citation of previous RCTs was extremely low, with a median of 20% of prior trials cited. Only 7 of 44 (15%) of subsequent reports referenced the largest trial (N 1/4 1784), which was 28 times larger than the median trial size.

Conclusions This study demonstrates that investigators evaluating aprotinin were not adequately citing previous research, resulting in a large number of RCTs being conducted to address efficacy questions that prior trials had already definitively answered. Institutional review boards and journals could reduce the number of redundant trials by requiring investigators to conduct adequate searches for prior evidence and conducting systematic reviews.

Why Most Clinical Research Is Not Useful

John Ioannidis argues that problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency define useful clinical research. He suggests most clinical research is not useful and reform is overdue.

Impact of the Consolidated Standards of Reporting Trials (CONSORT) checklist on reporting of randomized clinical trials in traditional Chinese medicine

OBJECTIVE

To examine the reporting characteristics and key methodological factors of randomized controlled trials (RCTs) in the field of traditional Chinese medicine and assess whether use of the Consolidated Standards of Reporting Trials (CONSORT) statement is associated with improvement in the quality of reports of RCTs.

METHOD

RCTs published in two major traditional Chinese medicine journals were retrieved and were systematically reviewed. Of those journals, one adopted the CONSORT statement (Journal of Chinese Integrative Medicine (JCIM)) and one did not (Chinese Journal of Integrative Medicine (CJIM)). Items within the CONSORT 2001 and the CONSORT 2010 statements were taken to develop a 63-item coding manual. The Jadad scale was also used to assess methodological quality of RCTs. Two observers assessed the reporting of reports and extracted data independently. P < 0.05 was considered statistically significant. SPSS was used for all analyses.

RESULTS

A total of 76 RCTs were included for the analysis (38 from JCIM and 38 from CJIM). Significant improvements both in the overall quality of reporting (CONSORT score) and reporting of methodological items (Jadad scale) over time were observed in adopter journal (JCIM) (P = 0.001). Overall CONSORT score of studies published during 2010 to 2011 (post-CONSORT) increased 15.30 averagely with 95%CI 8.34 to 22.26 (P = 0.001). Of those, items addressed on reporting of methodological quality were different significantly too. JCIM had more 1.30 items addressed than CJIM in average (95% CI 0.52 to 2.08, P = 0.003). In the multivariate analysis regression, the page length and CONSORT-promoting status have statistically significant associated with the (P = 0.001 and P = 0.040, respectively).

CONCLUSION

Although a large room needs to improve the reporting of randomized clinical trials in traditional Chinese medicine, the impact for improvement of reporting of TCM RCTs has been proven in some extent.

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

A generalized view of self-citation: direct, co-author, collaborative, and coercive induced self-citation

The phenomenon of self-citation can present in many different forms, including direct, co-author, collaborative, and coercive induced self-citation. It can also pertain to the citation of single scientists, groups of scientists, journals, and institutions. This article presents some case studies of extreme self-citation practices. It also discusses the implications of different types of self-citation. Self-citation is not necessarily inappropriate by default. In fact, usually it is fully appropriate but often it is even necessary. Conversely, inappropriate self-citation practices may be highly misleading and may distort the scientific literature. Coercive induced self-citation is the most difficult to discover. Coercive Induced self-citation may happen directly from reviewers of articles, but also indirectly from reviewers of grants, scientific advisors who steer a research agenda, and leaders of funding agencies who may espouse spending disproportionately large funds in research domains that perpetuate their own self-legacy. Inappropriate self-citation can be only a surrogate marker of what might be much greater distortions of the scientific corpus towards conformity to specific opinions and biases. Inappropriate self-citations eventually affect also impact metrics. Different impact metrics vary in the extent to which they can be gamed through self-citation practices. Citation indices that are more gaming-proof are available and should be more widely used. We need more empirical studies to dissect the impact of different types of inappropriate self-citation and to examine the effectiveness of interventions to limit them.

Ill communication: what's wrong with the medical literature and how to fix it

Current conventions for the writing of medical scientific papers impede clear communication of scientific research results. This article discusses the reasons for this and how to ameliorate them.

Harbin consensus conference and quality of infertility trials: reflections of a scientist on the Italian experience

During the days August 22–24, 2013 has been held in Harbin (China) an International Consensus Conference aimed to improve the quality and the reporting of the randomized controlled trials (RCTs) in infertility and subfertility field. I, as Italian scientist with experience in clinical infertility trials, was invited to have a speech on the Italian experience in RCTs, with particular regard for the surgical trials. Considerations on this subject were particularly interesting to highlight pitfalls and triumphs of research in Italy.

Poor citation of prior evidence in hip fracture trials

BACKGROUND

Failure to cite prior evidence in the medical literature may result in publication redundancy and inefficient use of research funding. We evaluated trials in which internal fixation was compared with arthroplasty for the treatment of hip fractures in order to determine the extent to which these randomized trials cited all relevant previous trials.

METHODS

We searched MEDLINE and Embase for all relevant articles on four topics: internal fixation compared with arthroplasty, total hip arthroplasty compared with hemiarthroplasty, sliding hip screws compared with other fixation devices, and surgical delay of hip fracture treatment. We determined the proportion of previous studies that were cited in comparison with the total number of previous studies that were citable (i.e., the citation rate) as well as the proportion of times that a study was cited in comparison with the total number of times that it could have been cited (i.e., the hit rate). A cumulative meta-analysis was performed for the "internal fixation compared with arthroplasty" topic to determine whether compelling evidence favoring one intervention existed at an earlier time.

RESULTS

In total, sixty studies were assessed and yielded an overall citation rate of 48%. All "highly cited" studies reported a positive result (favoring arthroplasty), and 60% were published in The Journal of Bone and Joint Surgery (American or British volume). The results of a study and the journal of publication significantly affected the hit rate (p < 0.05).

CONCLUSIONS

Our review of studies of hip fracture treatment suggests poor citation of the previous literature. Studies in higher-impact journals with positive results are more likely to be cited in subsequent studies. Therefore, redundancy in publication and unnecessary surgical trials often occur.

Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration

The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice. The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality. Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction. A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines. These recommendations aim to enhance the transparency, quality and completeness of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.

Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration

• The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice. • The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality. • Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction. • A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines. • These recommendations aim to enhance the transparency, quality and completeness of study reporting and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.

Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration

The rapid and continuing progress in gene discovery for complex diseases is fueling interest in the potential application of genetic risk models for clinical and public health practice. The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality. Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction. A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by previous reporting guidelines. These recommendations aim to enhance the transparency, quality and completeness of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.

Clarity and strength of implications for practice in medical journal articles: an exploratory analysis

OBJECTIVE

To examine how leading clinical journals report research findings, aiming to assess how they frame their implications for medical practice and to compare that literature's patterns with those of the management literature.

DATA SOURCE

Clinically relevant research articles from three leading clinical journals (N Engl J Med, JAMA, and Ann Intern Med).

METHODS

Review of wording of a sequential sample from 2010, with categorisation, comparison among journals, and comparison with management literature.

RESULTS

Clinical journals usually state that one approach did or did not differ from another approach (35 of 51 articles, 68.6%), but they recommended a specific course of action ('therefore, x should be done') in just 25.5%. One article gave instruction on how to implement the changes. Two-thirds of the reports called for further research. Half used tentative language. Management research articles nearly always specified who should use the information and drew from over 60 types of potential users, whereas the clinical literature named the audience in only 23.5% of clinicians.

CONCLUSIONS

Authors and editors of the clinical literature could test being more clear and direct in presenting implications of research findings for practice, including stating when the findings do not justify changes in practice.

Strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS): explanation and elaboration

The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice. The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality. Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction. A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines. These recommendations aim to enhance the transparency, quality and completeness of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.

Significance testing as perverse probabilistic reasoning

Truth claims in the medical literature rely heavily on statistical significance testing. Unfortunately, most physicians misunderstand the underlying probabilistic logic of significance tests and consequently often misinterpret their results. This near-universal misunderstanding is highlighted by means of a simple quiz which we administered to 246 physicians at two major academic hospitals, on which the proportion of incorrect responses exceeded 90%. A solid understanding of the fundamental concepts of probability theory is becoming essential to the rational interpretation of medical information. This essay provides a technically sound review of these concepts that is accessible to a medical audience. We also briefly review the debate in the cognitive sciences regarding physicians' aptitude for probabilistic inference.

Cochrane systematic reviews in the field of addiction: what's there and what should be

The Cochrane Drugs and Alcohol Group aims to produce, update, and disseminate systematic reviews on the prevention, treatment, and rehabilitation of problematic drug and alcohol use. The objective of the present paper was to summarize the main characteristics of the published systematic reviews in the field of drug and alcohol dependence, in terms of the topics covered, methods used to produce the reviews, and available evidence. By January 2010, the Group had published 52 reviews with 694 primary studies included out of 2059 studies considered for inclusion. Of these publications, 44% were published in 12 journals, including Drug and Alcohol Dependence (11%) with the highest number of publications, and 68% were conducted in North America. The majority of included studies (90%) were randomized controlled trials. Evaluating their methodological quality, we found that allocation concealment methods were not properly described in the majority of studies (18% adequate, 73% unclear, 9% inadequate). The percentage of interventions shown to be beneficial varied according to the substance considered: 42% for opioids, 37% for alcohol, 14% for psychostimulants, 7% for polydrugs, and 33% for prevention. Furthermore, 75% of the reviews provided specific information on further research needs. Cochrane reviews provide information on the most effective treatments, particularly in the area of opioid and alcohol dependence, and help clarify areas for further research.

Getting a Better Grip on Research: The Maze of the Most Busy Life

This is the fifth paper in a series of five describing the use of evidence to support decisions made in clinical practice. In the 21st century, health care clinicians, managers and patients expect to see the findings of research incorporated into clinical practice, taking into account the needs and wishes of individual patients. In the previous four papers in this series of five, we have examined why that happens—and often does not happen—and what clinicians and managers can do to improve the use of evidence in consultations. This, the final paper, is written as if from some point in the near future and uses a narrative approach to describe a clinician's progress on a journey to meet the real-world challenges of using evidence in practice.

Getting a Better Grip on Research: The Comfort of Opinion

In the 21st century, health care clinicians, managers and patients expect to see the findings of research incorporated into clinical practice, taking into account the needs and wishes of individual patients.

The origins, evolution, and future of The Cochrane Database of Systematic Reviews

The Cochrane Database of Systematic Reviews (CDSR) evolved in response to Archie Cochrane's challenge to the medical profession to assemble "a critical summary, adapted periodically, of all ... relevant randomized controlled trials". CDSR has been an electronic publication from its inception and this has meant that Cochrane reviews (i) need not be constrained by lack of space; (ii) can be updated as new information becomes available and when mistakes or other ways of improving them are identified; and (iii) can be cross-linked to other, related sources of relevant information. Although CDSR has become widely cited, it must continue to evolve in the light of technological and methodological developments, and in response to the needs of people making decisions about health care.

Interpreting trial results in light of conflicting evidence: a Bayesian analysis of adjuvant chemotherapy for non-small-cell lung cancer

PURPOSE

When successive randomized trials contradict prior evidence, clinicians may be unsure how to evaluate them: Does accumulating evidence warrant changing practice? An increasingly popular solution, Bayesian statistics quantitatively evaluate new results in context. This study provides a clinically relevant example of Bayesian methods.

METHODS

Three recent non-small-cell lung cancer adjuvant chemotherapy trials were evaluated in light of prior conflicting data. Results were used from International Adjuvant Lung Trial (IALT), JBR.10, and Adjuvant Navelbine International Trialist Association (ANITA). Prior evidence was sequentially updated to calculate the probability of each survival benefit level (overall and by stage) and variance. Sensitivity analysis was performed using expert opinion and uninformed estimates of survival benefit prior probability.

RESULTS

The probability of a 4% survival benefit increased from 33% before IALT to 64% after IALT. After sequential updating with JBR.10 and ANITA, this probability was 82% (hazard ratio = 0.84; 95% CI, 0.77 to 0.91). IALT produced the largest decrease in variance (61%) and decreased the chance of survival decrement to 0%. Sensitivity analysis did not support a survival benefit after IALT. However, sequential updating substantiated a 4% survival benefit and, for stage II and III, more than 90% probability of a 6% benefit and 50% probability of a 12% benefit.

CONCLUSION

When evaluated in context with prior data, IALT did not support a 4% survival benefit. However, sequential updating with JBR.10 and ANITA did. A model for future assessments, this study demonstrates the unique ability of Bayesian analysis to evaluate results that contradict prior evidence.