How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing

Assessing the transparency in reporting of clinical trials investigating manual therapy interventions for low back pain: A methodological review

RATIONALE

Low back pain (LBP) is a common condition with a significant societal burden. Manual therapy is an effective treatment for LBP and recommended in clinical practice guidelines. While the quantity of literature supporting the use of manual therapy is large, the methodological quality and transparency of this collective work are unclear.

AIMS AND OBJECTIVES

Explore the transparency in reporting of clinical trials assessing manual therapy interventions in patients with LBP by comparing planned components in the trial registration with what was reported in the published manuscript.

METHODS

Three databases were searched to identify trials assessing the treatment effect of manual therapy for LBP from January 2005 to May 2023. Studies were included if the manual therapy consisted of thrust manipulations, mobilizations or muscle energy techniques.

RESULTS

From 4462 studies initially identified, 167 studies remained in the final review after title, abstract and full-text review. Only 87 (52.1%) of the 167 studies were registered (n = 57 prospectively and n = 30 retrospectively). Primary outcomes in the publications were identical to the registration in 54 (62.1%) of the registered trials. Secondary outcomes in the publication were identical to the registration in 27 (31.0%) of the registered trials. The CONSORT reporting guideline was referenced in only 19 (21.8%) trials. Multiple discrepancies between registration and publication were noted for primary and secondary outcomes. All trials had eligibility criteria in the registration that matched their corresponding manuscript, while only four (4.6%) trial registrations addressed any type of statistical analysis plan.

CONCLUSION

Approximately half of the trials were not registered. Of those registered, only half were registered prospectively. Substantial discrepancies existed between registered and published outcomes that were never addressed by the authors, raising questions about potential bias. Transparency can be improved through more stringent requirements during manuscript submission to journals, and better reporting of the rationale for discrepancies between registration and publication.

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

A web-based tool for automatically linking clinical trials to their publications

OBJECTIVE

Evidence synthesis teams, physicians, policy makers, and patients and their families all have an interest in following the outcomes of clinical trials and would benefit from being able to evaluate both the results posted in trial registries and in the publications that arise from them. Manual searching for publications arising from a given trial is a laborious and uncertain process. We sought to create a statistical model to automatically identify PubMed articles likely to report clinical outcome results from each registered trial in ClinicalTrials.gov.

MATERIALS AND METHODS

A machine learning-based model was trained on pairs (publications known to be linked to specific registered trials). Multiple features were constructed based on the degree of matching between the PubMed article metadata and specific fields of the trial registry, as well as matching with the set of publications already known to be linked to that trial.

RESULTS

Evaluation of the model using known linked articles as gold standard showed that they tend to be top ranked (median best rank = 1.0), and 91% of them are ranked in the top 10.

DISCUSSION

Based on this model, we have created a free, public web-based tool that, given any registered trial in ClinicalTrials.gov, presents a ranked list of the PubMed articles in order of estimated probability that they report clinical outcome data from that trial. The tool should greatly facilitate studies of trial outcome results and their relation to the original trial designs.

High incidence of outcome switching observed in follow-up publications of randomized controlled trials: Meta-research study

OBJECTIVES

To determine the incidence of outcome switching in follow-up publications of randomized controlled trials. Outcome switching leads to bias where treatment benefits are more likely to be overestimated or based on chance.

STUDY DESIGN AND SETTING

Meta-research study including all follow-up publications 2014-2018 in the New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, and the British Medical Journal. Two independent reviewers compared the primary outcomes of follow-up publications with the original RCT publication and the trial protocol.

RESULTS

Seventy-eight follow-up publications were identified. Thirty-one (40%) used different primary outcomes in the follow-up publication compared with the original RCT. In seventeen (55%) of these the outcome switch was neither pre-specified nor explained in the journal publication. The incidence of outcome switching in follow-up studies rose to 70% when preceded by outcome switching in the corresponding initial RCT (P< 0.001).

CONCLUSION

In this study, outcome switching occurred in 40% of follow-up publications of previously published RCTs. The majority is neither pre-specified nor explained.

Outcome Measures Reported in Published Clinical Research Studies in Craniosynostosis: A Systematic Review

CONTEXT

The fair comparison of treatment interventions for craniosynostosis across different studies is expected to be impaired by incomplete reporting and the use of inconsistent outcomes.

OBJECTIVE

This review assessed the outcomes currently reported in studies of craniosynostosis, and whether these outcomes are formally defined and prespecified in the study methods.

DATA SOURCES, SEARCH TERMS, AND STUDY SELECTION

Studies were sourced via an electronic, multi-database literature search for "craniosynostosis." All primary, interventional research studies published from 2011 to 2015 were reviewed.

DATA EXTRACTION

Two independent researchers assessed each study for inclusion and performed the data extraction. For each study, data were extracted on the individual outcomes reported, and whether these outcomes were defined and prespecified in the methods.

DATA SYNTHESIS AND RESULTS

Of 1027 studies screened, 240 were included and proceeded to data extraction. These studies included 18,365 patients.2192 separate outcomes were reported. Of these, 851 outcomes (38.8%) were clearly defined, 1394 (63.6%) were prespecified in the study methods."Clinical and functional" was the most commonly reported outcome theme (900 outcomes, 41.1%), and "patient-reported" outcomes the least (7 outcomes, 0.3%)."Duration of surgery" was the most commonly reported single outcome (reported 80 times). "Cranial index" was the most variably defined outcome (18 different definitions used).

CONCLUSION

The outcomes reported following treatment interventions for craniosynostosis are incompletely and variably defined. Improving definitions for these outcomes may aid comparison of different management strategies and improve craniosynostosis care. Suboptimal prespecification of these outcomes in the study methods implied that outcome reporting bias cannot be excluded.

Analysis of articles directly related to randomized trials finds poor protocol availability and inconsistent linking of articles

OBJECTIVES

Interpreting a randomized trial requires access to more than the main results paper. We aimed to determine the (1) proportion of trials referring to the protocol in the trial report and their accessibility, (2) proportion of protocols accessible from trial registry entry and by trial registration number search, and (3) types of additional publications associated with trial reports.

STUDY DESIGN AND SETTING

A previously gathered sample of randomized trials of nonpharmacological interventions published in 2009 was used. Trial reports and registry entries were searched for protocol mentions and obtained when possible. Related publications were identified using citation searching.

RESULTS

Of 133 trials, 96 (72%) mentioned the protocol within the report, 61 (64%) contained details about protocol acquisition, with 48 (36%) protocols obtainable. Of the 129 registered trials, 32 (25%) had protocols obtainable from registry entry. Citation tracking identified 1,030 related publications, most common were secondary analyses and qualitative studies.

CONCLUSION

Trial protocols facilitate good trial conduct and interpretation. However, they are often not linked to the main report or registry and can be difficult to obtain. Many trials have related publications that are inconsistently linked. Trial registries and registration numbers could facilitate the threading of articles related to a trial, but currently do not.

The clinical trial transparency in oncology significantly increased over the recent years

OBJECTIVE

Persistent efforts have been made to promote clinical trial transparency, which included encouraging trial registration and prospective registration, as well as protocol disclosure. This study aims to analyze the extent of registration, prospective registration and protocol disclosure in oncology clinical trials and their changing trends.

STUDY DESIGN AND SETTING

All phase II and phase III oncology clinical trials published in 5 major journals, the Annals of Oncology, the Journal of Clinical Oncology, JAMA Oncology, The Lancet Oncology, and The New England Journal of Medicine, between January 2013 and December 2017, were included. Data on trial characteristics as well as registration status and availability of protocol and its location were collected.

RESULTS

In total, 625 articles were included, 92% were registered, of which 76% were prospectively registered. Overall, 27% provided protocols. Increasing trends were observed in registration, prospective registration, and protocol disclosure (all P < 0.001). Studies with enrollment number larger than median number were more likely to be registered (adjusted odds ratio (aOR), 3.14 [95% confidence interval (CI), 1.21-8.15]) and to provide protocols (aOR, 3.84 [95% CI, 2.24-6.57]) than those with smaller enrollment number. Studies with nonindustry funding was less likely to be prospectively registered (aOR, 0.37 [95% CI, 0.25-0.55]) but more likely to provide protocols (aOR, 1.69 [95% CI, 1.13-2.52]) than those with industry funding only.

CONCLUSION

Although the rates of registration, prospective registration, and protocol disclosure of oncology trials have significantly increased over the years, there is still room for improvement.

Tracking the timely dissemination of clinical studies. Characteristics and impact of 10 tracking variables

Background: Several meta-research studies and benchmarking activities have assessed how comprehensively and timely, academic institutions and private companies publish their clinical studies. These current "clinical trial tracking" activities differ substantially in how they sample relevant studies, and how they follow up on their publication. Methods: To allow informed policy and decision making on future publication assessment and benchmarking of institutions and companies, this paper outlines and discusses 10 variables that influence the tracking of timely publications. Tracking variables were initially selected by experts and by the authors through discussion. To validate the completeness of our set of variables, we conducted i) an explorative review of tracking studies and ii) an explorative tracking of registered clinical trials of three leading German university medical centres. Results: We identified the following 10 relevant variables impacting the tracking of clinical studies: 1) responsibility for clinical studies, 2) type and characteristics of clinical studies, 3) status of clinical studies, 4) source for sampling, 5) timing of registration, 6) determination of completion date, 7) timeliness of dissemination, 8) format of dissemination, 9) source for tracking, and 10) inter-rater reliability. Based on the description of these tracking variables and their influence, we discuss which variables could serve in what ways as a standard assessment of "timely publication". Conclusions: To facilitate the tracking and consequent benchmarking of how often and how timely academic institutions and private companies publish clinical study results, we have two core recommendations. First, the improvement in the link between registration and publication, for example via institutional policies for academic institutions and private companies. Second, the comprehensive and transparent reporting of tracking studies according to the 10 variables presented in this paper.

A novel approach to sharing all available information from funded health research: the NIHR Journals Library

Background

Relevant information on health research must be made publicly available in an accurate, timely and accessible manner if evidence is to inform practice and benefit patient care. Failure to publish research information represents a significant waste of research funds. However, recent studies have demonstrated that non-publication and selective or biased reporting remains a significant problem. The role of online publications in rectifying these issues by providing open access to study information is increasingly recognised.

Objective

This paper details a novel approach to publishing research information developed by the National Institute for Health Research (NIHR), a major funder of health research in the United Kingdom. The NIHR has enhanced its Journals Library (www.journalslibrary.nihr.ac.uk), providing an online repository of information from research funded through five programmes. We describe how the NIHR Journals Library provides a ‘thread’ of relevant information for each study, including protocols, participant information sheets, data linkages, final reports, publications and diverse knowledge products. We also discuss the Library as a ‘living’ resource, one that is updated as each study progresses from inception to completion. Finally, we consider the implications of the Library for the NIHR, other journals and research teams submitting information.

Conclusion

Openly publishing information from funded research in the NIHR Journals Library serves as a model of knowledge sharing, maximising return on investment and enhancing the usability and replicability of research findings for different evidence-user communities. The Library also supports wider ‘research on research’ ambitions, enabling users to interrogate the repository of NIHR-funded studies, enhancing the understanding of research commissioning, design, dissemination and impact.

Video abstract: www.youtube.com/watch?v=8H03uxN_iTE.

Electronic supplementary material

The online version of this article (10.1186/s12961-018-0339-4) contains supplementary material, which is available to authorized users.

Promoting public access to clinical trial protocols: challenges and recommendations

BACKGROUND

Recognizing the value of promoting public access to clinical trial protocols, Trials pioneered the way for their publication over a decade ago. However, despite major advances in the public accessibility of information about trial methods and results, protocol sharing remains relatively rare.

MAIN BODY

Protocol sharing facilitates the critical appraisal of clinical trials and helps to identify and deter the selective reporting of outcomes and analyses. Challenges to the routine availability of high quality trial protocols include the gaps in incentives and adherence mechanisms, limited venues for sharing the original and final protocol versions, and the need for mechanisms to ensure transparent and complete protocol content.

CONCLUSIONS

We propose recommendations for addressing key challenges to protocol sharing in order to promote routine public access to protocols for the benefit of patients and other users of evidence from clinical trials.

Knowledge translation tools for parents on child health topics: a scoping review

BACKGROUND

An emerging field of knowledge translation (KT) research has begun to focus on health consumers, particularly in child health. KT tools provide health consumers with research knowledge to inform health decision-making and may foster 'effective consumers'. Thus, the purpose of this scoping review was to describe the state of the field of previously published effectiveness research on child health-related KT tools for parents/caregivers to understand the evidence-base, identify gaps, and guide future research efforts.

METHODS

A health research librarian developed and implemented search strategies in 8 databases. One reviewer conducted screening using pre-determined criteria. A second reviewer verified 10% of screening decisions. Data extraction was performed by one reviewer. A descriptive analysis was conducted and included patient-important outcome classification, WIDER Recommendation checklist, and methodological quality assessment.

RESULTS

Seven thousand nine hundred fifty two independent titles and abstracts were reviewed, 2267 full-text studies were retrieved and reviewed, and 18 articles were included in the final data set. A variety of KT tools, including single- (n = 10) and multi-component tools (n = 10), were evaluated spanning acute (n = 4), chronic (n = 5) and public/population health (n = 9) child health topics. Study designs included: cross-sectional (n = 4), before-after (n = 1), controlled before-after (n = 2), cohort (n = 1), and RCTs (n = 10). The KT tools were evaluated via single primary outcome category (n = 11) and multiple primary outcome categories (n = 7). Two studies demonstrated significant positive effects on primary outcome categories; the remaining studies demonstrated mixed effects (n = 9) and no effect (n = 3). Overall, methodological quality was poor; studies lacked a priori protocols (n = 18) and sample size calculations (n = 13). Overall, intervention reporting was also poor; KT tools lacked description of theoretical underpinnings (n = 14), end-user engagement (n = 13), and preliminary research (n = 9) to inform the current effectiveness evaluation.

CONCLUSIONS

A number of child health-related knowledge translation tools have been developed for parents/caregivers. However, numerous outcomes were used to assess impact and there is limited evidence demonstrating their effectiveness. Moreover, the methodological rigor and reporting of effectiveness studies is limited. Careful tool development involving end-users and preliminary research, including usability testing and mixed methods, prior to large-scale studies may be important to advance the science of KT for health consumers.

Systematic review of control groups in nutrition education intervention research

Background

Well-designed research trials are critical for determining the efficacy and effectiveness of nutrition education interventions. To determine whether behavioral and/or cognition changes can be attributed to an intervention, the experimental design must include a control or comparison condition against which outcomes from the experimental group can be compared. Despite the impact different types of control groups can have on study outcomes, the treatment provided to participants in the control condition has received limited attention in the literature.

Methods

A systematic review of control groups in nutrition education interventions was conducted to better understand how control conditions are described in peer-reviewed journal articles compared with experimental conditions. To be included in the systematic review, articles had to be indexed in CINAHL, PubMed, PsycINFO, WoS, and/or ERIC and report primary research findings of controlled nutrition education intervention trials conducted in the United States with free-living consumer populations and published in English between January 2005 and December 2015. Key elements extracted during data collection included treatment provided to the experimental and control groups (e.g., overall intervention content, tailoring methods, delivery mode, format, duration, setting, and session descriptions, and procedures for standardizing, fidelity of implementation, and blinding); rationale for control group type selected; sample size and attrition; and theoretical foundation.

Results

The search yielded 43 publications; about one-third of these had an inactive control condition, which is considered a weak study design. Nearly two-thirds of reviewed studies had an active control condition considered a stronger research design; however, many failed to report one or more key elements of the intervention, especially for the control condition. None of the experimental and control group treatments were sufficiently detailed to permit replication of the nutrition education interventions studied.

Conclusions

Findings advocate for improved intervention study design and more complete reporting of nutrition education interventions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12966-017-0546-3) contains supplementary material, which is available to authorized users.

The use of study registration and protocols in plastic surgery research: A systematic review

BACKGROUND

In 2013, the Declaration of Helsinki changed to mandate that all research studies involving human subjects, rather than just clinical trials alone, must have a protocol registered in a publicly accessible database prior to the enrolment of the first patient. The objective of this work was to assess the number of research studies involving human participants published in leading journals of plastic surgery that had either published a protocol or registered a protocol with a publicly accessible database.

MATERIALS AND METHODS

This systematic review examined all research articles involving human participants published in Plastic and Reconstructive Surgery, The Journal of Plastic Reconstructive and Aesthetic Surgery and The Annals of Plastic Surgery from 1st April 2014-31st March 2015. The primary outcome measure was whether each study had either published or registered a protocol with any mainstream registry database. ClinicalTrials.gov, the International Standard Randomized Control Trial Number (ISRCTN) registry, the WHO (World Health Organisation) International Clinical Trials Registry Platform, The Cochrane Collaboration, the Research Registry, PROSPERO and PubMed were all reviewed.

RESULTS

Of 595 included articles, the most common study designs were case series (n = 185, 31.1%). There were 24 randomized controlled trials (RCTs, 4.0%). A total of 24 studies had a protocol registered (4.0%). The most common database to register a protocol was with ClinicalTrials.gov (n = 17). The study design that most commonly had a registered protocol was the RCT (n = 8 of 24, 33.3% of RCTs). Three studies published a protocol in a journal (0.6%).

CONCLUSION

Publication or registration of protocols for recent studies involving human participants in major plastic surgery journals is low. There is considerable scope to improve this and guidance is provided.

Major discrepancies between what clinical trial registries record and paediatric randomised controlled trials publish

BACKGROUND

Whether information from clinical trial registries (CTRs) and published randomised controlled trial (RCTs) differs remains unknown. Knowing more about discrepancies should alert those who rely on RCTs for medical decision-making to possible dissemination or reporting bias. To provide help in critically appraising research relevant for clinical practice we sought possible discrepancies between what CTRs record and paediatric RCTs actually publish. For this purpose, after identifying six reporting domains including funding, design, and outcomes, we collected data from 20 consecutive RCTs published in a widely read peer-reviewed paediatric journal and cross-checked reported features with those in the corresponding CTRs.

METHODS

We collected data for 20 unselected, consecutive paediatric RCTs published in a widely read peer-reviewed journal from July to November 2013. To assess discrepancies, two reviewers identified and scored six reporting domains: funding and conflict of interests; sample size, inclusion and exclusion criteria or crossover; primary and secondary outcomes, early study completion, and main outcome reporting. After applying the Critical Appraisal Skills Programme (CASP) checklist, five reviewer pairs cross-checked CTRs and matching RCTs, then mapped and coded the reporting domains and scored combined discrepancy as low, medium and high.

RESULTS

The 20 RCTs were registered in five different CTRs. Even though the 20 RCTs fulfilled the CASP general criteria for assessing internal validity, 19 clinical trials had medium or high combined discrepancy scores for what the 20 RCTs reported and the matched five CTRs stated. All 20 RCTs selectively reported or failed to report main outcomes, 9 had discrepancies in declaring sponsorship, 8 discrepancies in the sample size, 9 failed to respect inclusion or exclusion criteria, 11 downgraded or modified primary outcome or upgraded secondary outcomes, and 13 completed early without justification. The CTRs for seven trials failed to index automatically the URL address or the RCT reference, and for 12 recorded RCT details, but the authors failed to report the results.

CONCLUSIONS

Major discrepancies between what CTRs record and paediatric RCTs publish raise concern about what clinical trials conclude. Our findings should make clinicians, who rely on RCT results for medical decision-making, aware of dissemination or reporting bias. Trialists need to bring CTR data and reported protocols into line with published data.

OpenTrials: towards a collaborative open database of all available information on all clinical trials

OpenTrials is a collaborative and open database for all available structured data and documents on all clinical trials, threaded together by individual trial. With a versatile and expandable data schema, it is initially designed to host and match the following documents and data for each trial: registry entries; links, abstracts, or texts of academic journal papers; portions of regulatory documents describing individual trials; structured data on methods and results extracted by systematic reviewers or other researchers; clinical study reports; and additional documents such as blank consent forms, blank case report forms, and protocols. The intention is to create an open, freely re-usable index of all such information and to increase discoverability, facilitate research, identify inconsistent data, enable audits on the availability and completeness of this information, support advocacy for better data and drive up standards around open data in evidence-based medicine. The project has phase I funding. This will allow us to create a practical data schema and populate the database initially through web-scraping, basic record linkage techniques, crowd-sourced curation around selected drug areas, and import of existing sources of structured and documents. It will also allow us to create user-friendly web interfaces onto the data and conduct user engagement workshops to optimise the database and interface designs. Where other projects have set out to manually and perfectly curate a narrow range of information on a smaller number of trials, we aim to use a broader range of techniques and attempt to match a very large quantity of information on all trials. We are currently seeking feedback and additional sources of structured data.

Re: "Did you write a protocol before starting your project?"

We read the article "did you write a protocol before starting your project?" by Bando et al. In this letter, we express our opinion regarding the importance of writing protocols prior to starting projects.

Should there be greater use of preprint servers for publishing reports of biomedical science?

Vitek Tracz and Rebecca Lawrence declare the current journal publishing system to be broken beyond repair. They propose that it should be replaced by immediate publication followed by transparent peer review as the starting place for more open and efficient reporting of science. While supporting this general objective, we suggest that research is needed both to understand why biomedical scientists have been slow to take up preprint options, as well as to assess the relative merits of this and other alternatives to journal publishing.

How individual participant data meta-analyses have influenced trial design, conduct, and analysis

Objectives

To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer.

Study Design and Setting

Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses.

Results

We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials.

Conclusions

IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.

A living document: reincarnating the research article

The limitations of the traditional research paper are well known and widely discussed; however, rather than seeking solutions to the problems created by this model of publication, it is time to do away with a print era anachronism and design a new model of publication, with modern technology embedded at its heart. Instead of the current system with multiple publications, across multiple journals, publication could move towards a single, evolving document that begins with trial registration and then extends to include the full protocol and results as they become available, underpinned by the raw clinical data and all code used to obtain the result. This model would lead to research being evaluated prospectively, based on its hypothesis and methodology as stated in the study protocol, and move away from considering serendipitous results to be synonymous with quality, while also presenting readers with the opportunity to reliably evaluate bias or selective reporting in the published literature.

Increasing value and reducing waste: addressing inaccessible research

The methods and results of health research are documented in study protocols, full study reports (detailing all analyses), journal reports, and participant-level datasets. However, protocols, full study reports, and participant-level datasets are rarely available, and journal reports are available for only half of all studies and are plagued by selective reporting of methods and results. Furthermore, information provided in study protocols and reports varies in quality and is often incomplete. When full information about studies is inaccessible, billions of dollars in investment are wasted, bias is introduced, and research and care of patients are detrimentally affected. To help to improve this situation at a systemic level, three main actions are warranted. First, academic institutions and funders should reward investigators who fully disseminate their research protocols, reports, and participant-level datasets. Second, standards for the content of protocols and full study reports and for data sharing practices should be rigorously developed and adopted for all types of health research. Finally, journals, funders, sponsors, research ethics committees, regulators, and legislators should endorse and enforce policies supporting study registration and wide availability of journal reports, full study reports, and participant-level datasets.

Reporting of interventions in randomised trials: an audit of journal instructions to authors

Background

A complete description of the intervention in a published trial report is necessary for readers to be able to use the intervention, yet the completeness of intervention descriptions in trials is very poor. Low awareness of the issue by authors, reviewers, and editors is part of the cause and providing specific instructions about intervention reporting to authors and encouraging full sharing of intervention materials is important. We assessed the extent to which: 1) journals’ Instructions to Authors provide instructions about how interventions that have been evaluated in a randomised controlled trial (RCT) should be reported in the paper; and 2) journals offer the option of authors providing online supplementary materials.

Methods

We examined the web-based Instructions to Authors of 106 journals (the six leading general medical journals, 50 randomly selected journals from the National Library of Medicine’s Core Clinical Journals, and 50 randomly selected journals from the remainder of the journal collection indexed by PubMed). To be eligible, each journal must have published at least one randomised trial involving human participants each year from 2008 to 2012. We extracted all information related to the reporting of interventions, reporting of randomised trials in general, and online supplementary materials.

Results

Of the 106 journals’ Instructions to Authors, only 15 (14%) specifically mentioned the reporting of interventions and most of these provided non-specific advice such as ‘describe essential features’. Just over half (62, 58%) of the journals mentioned the Consolidated Standards of Reporting Trials (CONSORT) statement in their author instructions. Seventy-eight (74%) of the journals’ instructions mentioned the option of providing supplementary content online as part of the paper; however, only four of these journals explicitly encouraged or mandated use of this option for providing intervention information or materials.

Conclusions

Most journals’ Instructions to Authors do not provide any specific instructions regarding reporting of interventions or encourage authors to provide online supplementary materials to enhance intervention reporting. Journals can help to improve the problem of incomplete intervention reporting by providing specific instructions to authors and peer reviewers about intervention reporting and requiring full intervention descriptions to be provided.

A randomised trial of adaptive pacing therapy, cognitive behaviour therapy, graded exercise, and specialist medical care for chronic fatigue syndrome (PACE): statistical analysis plan

Background

The publication of protocols by medical journals is increasingly becoming an accepted means for promoting good quality research and maximising transparency. Recently, Finfer and Bellomo have suggested the publication of statistical analysis plans (SAPs).The aim of this paper is to make public and to report in detail the planned analyses that were approved by the Trial Steering Committee in May 2010 for the principal papers of the PACE (Pacing, graded Activity, and Cognitive behaviour therapy: a randomised Evaluation) trial, a treatment trial for chronic fatigue syndrome. It illustrates planned analyses of a complex intervention trial that allows for the impact of clustering by care providers, where multiple care-providers are present for each patient in some but not all arms of the trial.

Results

The trial design, objectives and data collection are reported. Considerations relating to blinding, samples, adherence to the protocol, stratification, centre and other clustering effects, missing data, multiplicity and compliance are described. Descriptive, interim and final analyses of the primary and secondary outcomes are then outlined.

Conclusions

This SAP maximises transparency, providing a record of all planned analyses, and it may be a resource for those who are developing SAPs, acting as an illustrative example for teaching and methodological research. It is not the sum of the statistical analysis sections of the principal papers, being completed well before individual papers were drafted.

Trial registration

ISRCTN54285094 assigned 22 May 2003; First participant was randomised on 18 March 2005.

Improving the reporting of randomised trials: the CONSORT Statement and beyond

An extensive and growing number of reviews of the published literature demonstrate that health research publications have frequent deficiencies. Of particular concern are poor reports of randomised trials, which make it difficult or impossible for readers to assess how the research was conducted, to evaluate the reliability of the findings, or to place them in the context of existing research evidence. As a result, published reports of trials often cannot be used by clinicians to inform patient care or to inform public health policy, and the data cannot be included in systematic reviews. Reporting guidelines are designed to identify the key information that researchers should include in a report of their research. We describe the history of reporting guidelines for randomised trials culminating in the CONSORT Statement in 1996. We detail the subsequent development and extension of CONSORT and consider related initiatives aimed at improving the reliability of the medical research literature.

Direct improvement of quality of life using a tailored quality of life diagnosis and therapy pathway: randomised trial in 200 women with breast cancer

BACKGROUND

Despite thousands of papers, the value of quality of life (QoL) in curing disease remains uncertain. Until now, we lacked tools for the diagnosis and specific treatment of diseased QoL. We approached this problem stepwise by theory building, modelling, an exploratory trial and now a definitive randomised controlled trial (RCT) in breast cancer, whose results we report here.

METHODS

In all, 200 representative Bavarian primary breast cancer patients were recruited by five hospitals and treated by 146 care professionals. Patients were randomised to either (1) a novel care pathway including diagnosis of 'diseased' QoL (any QoL measure below 50 points) using a QoL profile and expert report sent to the patient's coordinating practitioner, who arranged QoL therapy consisting of up to five standardised treatments for specific QoL defects or (2) standard postoperative care adhering to the German national guideline for breast cancer. The primary end point was the proportion of patients in each group with diseased QoL 6 months after surgery. Patients were blinded to their allocated group.

RESULTS

At 0 and 3 months after surgery, diseased QoL was diagnosed in 70% of patients. The QoL pathway reduced rates of diseased QoL to 56% at 6 months, especially in emotion and coping, compared with 71% in controls (P=0.048). Relative risk reduction was 21% (95% confidence interval (CI): 0-37), absolute risk reduction 15% (95% CI: 0.3-29), number needed to treat (NNT)=7 (95% CI: 3-37). When QoL therapy finished after successful treatment, diseased QoL often returned again, indicating good responsiveness of the QoL pathway.

CONCLUSION

A three-component outcome system including clinician-derived objective, patient-reported subjective end points and qualitative analysis of clinical relevance was developed in the last 10 years for cancer as a complex intervention. A separate QoL pathway was implemented for the diagnosis and treatment of diseased QoL and its effectiveness tested in a community-based, pragmatic, definitive RCT. While the pathway was active, it was effective with an NNT of 7.

Five years of Trials

This editorial marks the launch of a special collection of articles highlighting 'Five years of Trials' (http://www.trialsjournal.com/series/5years). The journal's achievements on its objectives since 2006 are described and some of the challenges still ahead are outlined - in particular further innovating in the reporting of trials and the publication of negative results. The other articles in this series are examples of where Trials has demonstrated progress on its objectives. These include the publication of raw data, extended versions of previously published trial-related articles, descriptions of 'lessons learned', negative results, and educational articles regarding ethics and reporting bias.

Common statistical and research design problems in manuscripts submitted to high-impact medical journals

Background

To assist educators and researchers in improving the quality of medical research, we surveyed the editors and statistical reviewers of high-impact medical journals to ascertain the most frequent and critical statistical errors in submitted manuscripts.

Findings

The Editors-in-Chief and statistical reviewers of the 38 medical journals with the highest impact factor in the 2007 Science Journal Citation Report and the 2007 Social Science Journal Citation Report were invited to complete an online survey about the statistical and design problems they most frequently found in manuscripts. Content analysis of the responses identified major issues. Editors and statistical reviewers (n = 25) from 20 journals responded. Respondents described problems that we classified into two, broad themes: A. statistical and sampling issues and B. inadequate reporting clarity or completeness. Problems included in the first theme were (1) inappropriate or incomplete analysis, including violations of model assumptions and analysis errors, (2) uninformed use of propensity scores, (3) failing to account for clustering in data analysis, (4) improperly addressing missing data, and (5) power/sample size concerns. Issues subsumed under the second theme were (1) Inadequate description of the methods and analysis and (2) Misstatement of results, including undue emphasis on p-values and incorrect inferences and interpretations.

Conclusions

The scientific quality of submitted manuscripts would increase if researchers addressed these common design, analytical, and reporting issues. Improving the application and presentation of quantitative methods in scholarly manuscripts is essential to advancing medical research.

The ISRCTN Register: achievements and challenges 8 years on

The ISRCTN register has been operational for the past 8 years and is approaching 10,000 trial records. It complies with international guidelines and pools its data in the International Trial Search Portal initiated by the World Health Organisation. Through its ongoing collaboration with the Department of Health in England, the register has been able to participate in a national initiative aiming to bring clinical trials to the attention of a wider public with the objective of maximising participation. As part of BioMed Central, the register provides the first step in the concept of threaded publications, enabling the tracking of clinical research studies from inception and the linking of all resulting publications including the raw data where this is available.

A systematic review of the reporting of Data Monitoring Committees' roles, interim analysis and early termination in pediatric clinical trials

Background

Decisions about interim analysis and early stopping of clinical trials, as based on recommendations of Data Monitoring Committees (DMCs), have far reaching consequences for the scientific validity and clinical impact of a trial. Our aim was to evaluate the frequency and quality of the reporting on DMC composition and roles, interim analysis and early termination in pediatric trials.

Methods

We conducted a systematic review of randomized controlled clinical trials published from 2005 to 2007 in a sample of four general and four pediatric journals. We used full-text databases to identify trials which reported on DMCs, interim analysis or early termination, and included children or adolescents. Information was extracted on general trial characteristics, risk of bias, and a set of parameters regarding DMC composition and roles, interim analysis and early termination.

Results

110 of the 648 pediatric trials in this sample (17%) reported on DMC or interim analysis or early stopping, and were included; 68 from general and 42 from pediatric journals. The presence of DMCs was reported in 89 of the 110 included trials (81%); 62 papers, including 46 of the 89 that reported on DMCs (52%), also presented information about interim analysis. No paper adequately reported all DMC parameters, and nine (15%) reported all interim analysis details. Of 32 trials which terminated early, 22 (69%) did not report predefined stopping guidelines and 15 (47%) did not provide information on statistical monitoring methods.

Conclusions

Reporting on DMC composition and roles, on interim analysis results and on early termination of pediatric trials is incomplete and heterogeneous. We propose a minimal set of reporting parameters that will allow the reader to assess the validity of trial results.

What is missing from descriptions of treatment in trials and reviews?

Replicating non-pharmacological treatments in practice depends on how well they have been described in research studies

Protecting privacy using k-anonymity

OBJECTIVE

There is increasing pressure to share health information and even make it publicly available. However, such disclosures of personal health information raise serious privacy concerns. To alleviate such concerns, it is possible to anonymize the data before disclosure. One popular anonymization approach is k-anonymity. There have been no evaluations of the actual re-identification probability of k-anonymized data sets.

DESIGN

Through a simulation, we evaluated the re-identification risk of k-anonymization and three different improvements on three large data sets.

MEASUREMENT

Re-identification probability is measured under two different re-identification scenarios. Information loss is measured by the commonly used discernability metric.

RESULTS

For one of the re-identification scenarios, k-Anonymity consistently over-anonymizes data sets, with this over-anonymization being most pronounced with small sampling fractions. Over-anonymization results in excessive distortions to the data (i.e., high information loss), making the data less useful for subsequent analysis. We found that a hypothesis testing approach provided the best control over re-identification risk and reduces the extent of information loss compared to baseline k-anonymity.

CONCLUSION

Guidelines are provided on when to use the hypothesis testing approach instead of baseline k-anonymity.

Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report

OBJECTIVE

Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported.

DESIGN

Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals.

SETTING

Reports of randomized trials in five general medical journals.

PARTICIPANTS

Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine.

INTERVENTIONS

None.

MAIN OUTCOME MEASURES

The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed.

RESULTS

We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration - quantitative or qualitative - of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials.

CONCLUSIONS

There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.

Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report

OBJECTIVE

Scientific and ethical justification for new clinical trials requires them to have been designed in the light of scientifically defensible assessments of relevant previous research. Reliable interpretation of the results of new clinical trials entails setting them in the context of updates of the reviews upon which they were deemed scientifically and ethically justifiable. We have shown previously that most reports of randomized trials published in five general medical journals in May 1997 and in May 2001 failed to set their results in the context of the findings from similar research. In the current study, we assess whether there had been progress in this respect in 2005 and also investigate the extent to which reports begin by referring to systematic reviews providing the justification for the new research reported.

DESIGN

Assessment of the Introduction and Discussion sections in all reports of randomized trials published during May 2005 in five general medical journals.

SETTING

Reports of randomized trials in five general medical journals.

PARTICIPANTS

Annals of Internal Medicine, BMJ, JAMA, Lancet and New England Journal of Medicine.

INTERVENTIONS

None.

MAIN OUTCOME MEASURES

The inclusion or mention of one or more systematic reviews in the Introduction or Discussion section of each report assessed.

RESULTS

We found 18 reports of randomized trials. The Introduction sections referred to systematic reviews in five (27%) of these reports. None of the discussion sections of the 15 reports of trials that were not the first published trials to address the question studied placed the results of the new trial in the context of an updated systematic review of other research. Although reference was made to relevant systematic reviews in five of these 15 reports, there was no integration - quantitative or qualitative - of the results of the new trials in an update of these reviews. In the remaining ten reports there was no evidence that any systematic attempt had been made to set the new results in the context of previous trials.

CONCLUSIONS

There is no evidence of progress between 1997 and 2005 in the proportion of reports of trials published in general medical journals which discussed new results within the context of up-to-date systematic reviews of relevant evidence from other controlled trials. Although the proportion of trials referring to systematic reviews in Discussion sections has increased, the majority of reports continued to fail even to do this. Similarly, most researchers appear not to have considered a systematic review when designing their trial. Researchers and journal editors do a disservice to the interests of the public and others involved in healthcare decision-making by acquiescing in this situation.

The use of the World Wide Web by medical journals in 2003 and 2005: an observational study

OBJECTIVES

The 2- to 6-page print journal article has been the standard for 200 years, yet this format severely limits the amount of detailed information that can be conveyed. The World Wide Web provides a low-cost option for posting extended text and supplementary information. It also can enhance the experience of journal editors, reviewers, readers, and authors through added functionality (eg, online submission and peer review, postpublication critique, and e-mail notification of table of contents.) Our aim was to characterize ways that journals were using the World Wide Web in 2005 and note changes since 2003.

METHODS

We analyzed the Web sites of 138 high-impact print journals in 3 ways. First, we compared the print and Web versions of March 2003 and 2005 issues of 28 journals (20 of which were randomly selected from the 138) to determine how often articles were published Web only and how often print articles were augmented by Web-only supplements. Second, we examined what functions were offered by each journal Web site. Third, for journals that offered Web pages for reader commentary about each article, we analyzed the number of comments and characterized these comments.

RESULTS

Fifty-six articles (7%) in 5 journals were Web only. Thirteen of the 28 journals had no supplementary online content. By 2005, several journals were including Web-only supplements in >20% of their papers. Supplementary methods, tables, and figures predominated. The use of supplementary material increased by 5% from 2% to 7% in the 20-journal random sample from 2003 to 2005. Web sites had similar functionality with an emphasis on linking each article to related material and e-mailing readers about activity related to each article. There was little evidence of journals using the Web to provide readers an interactive experience with the data or with each other. Seventeen of the 138 journals offered rapid-response pages. Only 18% of eligible articles had any comments after 5 months.

CONCLUSIONS

Journal Web sites offer similar functionality. The use of online-only articles and online-only supplements is increasing.