Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update

Reporting completeness of nutrition and diet-related randomised controlled trials protocols

BACKGROUND AND AIMS

There is a need to consolidate reporting guidance for nutrition randomised controlled trial (RCT) protocols. The reporting completeness in nutrition RCT protocols and study characteristics associated with adherence to SPIRIT and TIDieR reporting guidelines are unknown. We, therefore, assessed reporting completeness and its potential predictors in a random sample of published nutrition and diet-related RCT protocols.

METHODS

We conducted a meta-research study of 200 nutrition and diet-related RCT protocols published in 2019 and 2021 (aiming to consider periods before and after the start of the COVID pandemic). Data extraction included bibliometric information, general study characteristics, compliance with 122 questions corresponding to items and subitems in the SPIRIT and TIDieR checklists combined, and mention to these reporting guidelines in the publications. We calculated the proportion of protocols reporting each item and the frequency of items reported for each protocol. We investigated associations between selected publication aspects and reporting completeness using linear regression analysis.

RESULTS

The majority of protocols included adults and elderly as their study population (n = 73; 36.5%), supplementation as intervention (n = 96; 48.0%), placebo as comparator (n = 89; 44.5%), and evaluated clinical status as the outcome (n = 80; 40.0%). Most protocols described a parallel RCT (n = 188; 94.0%) with a superiority framework (n = 141; 70.5%). Overall reporting completeness was 52.0% (SD = 10.8%). Adherence to SPIRIT items ranged from 0% (n = 0) (data collection methods) to 98.5% (n = 197) (eligibility criteria). Adherence to TIDieR items ranged from 5.5% (n = 11) (materials used in the intervention) to 98.5% (n = 197) (description of the intervention). The multivariable regression analysis suggests that a higher number of authors [β = 0.53 (95%CI: 0.28-0.78)], most recent published protocols [β = 3.19 (95%CI: 0.24-6.14)], request of reporting guideline checklist during the submission process by the journal [β = 6.50 (95%CI: 2.56-10.43)] and mention of SPIRIT by the authors [β = 5.15 (95%CI: 2.44-7.86)] are related to higher reporting completeness scores.

CONCLUSIONS

Reporting completeness in a random sample of 200 diet or nutrition-related RCT protocols was low. Number of authors, year of publication, self-reported adherence to SPIRIT, and journals' endorsement of reporting guidelines seem to be positively associated with reporting completeness in nutrition and diet-related RCT protocols.

Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study

OBJECTIVES

The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting.

DESIGN

Methodological cross-sectional study.

METHODS

Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively.

RESULTS

We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037).

CONCLUSIONS

Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result.

STUDY REGISTRATION NUMBER

The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).

Automatic categorization of self-acknowledged limitations in randomized controlled trial publications

OBJECTIVE

Acknowledging study limitations in a scientific publication is a crucial element in scientific transparency and progress. However, limitation reporting is often inadequate. Natural language processing (NLP) methods could support automated reporting checks, improving research transparency. In this study, our objective was to develop a dataset and NLP methods to detect and categorize self-acknowledged limitations (e.g., sample size, blinding) reported in randomized controlled trial (RCT) publications.

METHODS

We created a data model of limitation types in RCT studies and annotated a corpus of 200 full-text RCT publications using this data model. We fine-tuned BERT-based sentence classification models to recognize the limitation sentences and their types. To address the small size of the annotated corpus, we experimented with data augmentation approaches, including Easy Data Augmentation (EDA) and Prompt-Based Data Augmentation (PromDA). We applied the best-performing model to a set of about 12K RCT publications to characterize self-acknowledged limitations at larger scale.

RESULTS

Our data model consists of 15 categories and 24 sub-categories (e.g., Population and its sub-category DiagnosticCriteria). We annotated 1090 instances of limitation types in 952 sentences (4.8 limitation sentences and 5.5 limitation types per article). A fine-tuned PubMedBERT model for limitation sentence classification improved upon our earlier model by about 1.5 absolute percentage points in F1 score (0.821 vs. 0.8) with statistical significance (p<.001). Our best-performing limitation type classification model, PubMedBERT fine-tuning with PromDA (Output View), achieved an F1 score of 0.7, improving upon the vanilla PubMedBERT model by 2.7 percentage points, with statistical significance (p<.001).

CONCLUSION

The model could support automated screening tools which can be used by journals to draw the authors' attention to reporting issues. Automatic extraction of limitations from RCT publications could benefit peer review and evidence synthesis, and support advanced methods to search and aggregate the evidence from the clinical trial literature.

CONSORT-TM: Text classification models for assessing the completeness of randomized controlled trial publications

Objective

To develop text classification models for determining whether the checklist items in the CONSORT reporting guidelines are reported in randomized controlled trial publications.

Materials and Methods

Using a corpus annotated at the sentence level with 37 fine-grained CONSORT items, we trained several sentence classification models (PubMedBERT fine-tuning, BioGPT fine-tuning, and in-context learning with GPT-4) and compared their performance. To address the problem of small training dataset, we used several data augmentation methods (EDA, UMLS-EDA, text generation and rephrasing with GPT-4) and assessed their impact on the fine-tuned PubMedBERT model. We also fine-tuned PubMedBERT models limited to checklist items associated with specific sections (e.g., Methods) to evaluate whether such models could improve performance compared to the single full model. We performed 5-fold cross-validation and report precision, recall, F1 score, and area under curve (AUC).

Results

Fine-tuned PubMedBERT model that takes as input the sentence and the surrounding sentence representations and uses section headers yielded the best overall performance (0.71 micro-F1, 0.64 macro-F1). Data augmentation had limited positive effect, UMLS-EDA yielding slightly better results than data augmentation using GPT-4. BioGPT fine-tuning and GPT-4 in-context learning exhibited suboptimal results. Methods-specific model yielded higher performance for methodology items, other section-specific models did not have significant impact.

Conclusion

Most CONSORT checklist items can be recognized reasonably well with the fine-tuned PubMedBERT model but there is room for improvement. Improved models can underpin the journal editorial workflows and CONSORT adherence checks and can help authors in improving the reporting quality and completeness of their manuscripts.

Reporting of equity in observational epidemiology: A methodological review

Background

Observational studies can inform how we understand and address persisting health inequities through the collection, reporting and analysis of health equity factors. However, the extent to which the analysis and reporting of equity-relevant aspects in observational research are generally unknown. Thus, we aimed to systematically evaluate how equity-relevant observational studies reported equity considerations in the study design and analyses.

Methods

We searched MEDLINE for health equity-relevant observational studies from January 2020 to March 2022, resulting in 16 828 articles. We randomly selected 320 studies, ensuring a balance in focus on populations experiencing inequities, country income settings, and coronavirus disease 2019 (COVID-19) topic. We extracted information on study design and analysis methods.

Results

The bulk of the studies were conducted in North America (n = 95, 30%), followed by Europe and Central Asia (n = 55, 17%). Half of the studies (n = 171, 53%) addressed general health and well-being, while 49 (15%) focused on mental health conditions. Two-thirds of the studies (n = 220, 69%) were cross-sectional. Eight (3%) engaged with populations experiencing inequities, while 22 (29%) adapted recruitment methods to reach these populations. Further, 67 studies (21%) examined interaction effects primarily related to race or ethnicity (48%). Two-thirds of the studies (72%) adjusted for characteristics associated with inequities, and 18 studies (6%) used flow diagrams to depict how populations experiencing inequities progressed throughout the studies.

Conclusions

Despite over 80% of the equity-focused observational studies providing a rationale for a focus on health equity, reporting of study design features relevant to health equity ranged from 0-95%, with over half of the items reported by less than one-quarter of studies. This methodological study is a baseline assessment to inform the development of an equity-focussed reporting guideline for observational studies as an extension of the well-known Strengthening Reporting of Observational Studies in Epidemiology (STROBE) guideline.

Guidelines for reporting pediatric and child health clinical trial protocols and reports: study protocol for SPIRIT-Children and CONSORT-Children

BACKGROUND

Despite the critical importance of clinical trials to provide evidence about the effects of intervention for children and youth, a paucity of published high-quality pediatric clinical trials persists. Sub-optimal reporting of key trial elements necessary to critically appraise and synthesize findings is prevalent. To harmonize and provide guidance for reporting in pediatric controlled clinical trial protocols and reports, reporting guideline extensions to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) and Consolidated Standards of Reporting Trials (CONSORT) guidelines specific to pediatrics are being developed: SPIRIT-Children (SPIRIT-C) and CONSORT-Children (CONSORT-C).

METHODS

The development of SPIRIT-C/CONSORT-C will be informed by the Enhancing the Quality and Transparency of Health Research Quality (EQUATOR) method for reporting guideline development in the following stages: (1) generation of a preliminary list of candidate items, informed by (a) items developed during initial development efforts and child relevant items from recent published SPIRIT and CONSORT extensions; (b) two systematic reviews and environmental scan of the literature; (c) workshops with young people; (2) an international Delphi study, where a wide range of panelists will vote on the inclusion or exclusion of candidate items on a nine-point Likert scale; (3) a consensus meeting to discuss items that have not reached consensus in the Delphi study and to "lock" the checklist items; (4) pilot testing of items and definitions to ensure that they are understandable, useful, and applicable; and (5) a final project meeting to discuss each item in the context of pilot test results. Key partners, including young people (ages 12-24 years) and family caregivers (e.g., parents) with lived experiences with pediatric clinical trials, and individuals with expertise and involvement in pediatric trials will be involved throughout the project. SPIRIT-C/CONSORT-C will be disseminated through publications, academic conferences, and endorsement by pediatric journals and relevant research networks and organizations.

DISCUSSION

SPIRIT/CONSORT-C may serve as resources to facilitate comprehensive reporting needed to understand pediatric clinical trial protocols and reports, which may improve transparency within pediatric clinical trials and reduce research waste.

TRIAL REGISTRATION

The development of these reporting guidelines is registered with the EQUATOR Network: SPIRIT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials-protocols/#35 ) and CONSORT-Children ( https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-clinical-trials/#CHILD ).

STROBE, CONSORT, PRISMA, MOOSE, STARD, SPIRIT, and other guidelines - Overview and application

The purpose of research is to seek answers and new knowledge. When conducted properly and systematically, research adds to humanity's corpus of knowledge and hence to our general advancement. However, this is only possible if reported research is accurate and transparent. Guidelines for all the major types of studies (STROBE, CONSORT, PRISMA, MOOSE, STARD, and SPIRIT) have been developed and refined over the years, and their inception, development, and application are briefly discussed in this paper. Indeed, there are currently over 250 of these guidelines for various types of medical research, and these are published by the EQUATOR network. This paper will also briefly review progress in acceptance and adoption of these guidelines.

A Systematic Review to Inform the Development of a Reporting Guideline for Concept Mapping Research

Concept mapping is a phased, mixed-method approach that is increasingly used in health research to develop an understanding of complex phenomena. The six phases of concept mapping are preparation, idea generation, structuring (clustering and prioritization), data analysis, interpretation, and utilization of the map. The reporting of concept mapping research requires the development of a specific reporting guideline. We conducted a systematic review to identify candidate reporting items for inclusion in a reporting guideline. Three databases (MEDLINE, CINAHL, and PsycInfo) were searched to identify studies that used concept mapping methodology. We included 75 concept mapping studies published since 2019 from which we extracted information about the quality of reporting. A third of the studies focused on public health. We identified 71 candidate items that relate to the quality of reporting concept mapping research. The rationale for the study, the focus prompt, procedures for brainstorming, and structuring statements were consistently reported across the included studies. The process for developing the focus prompt, the rationale for the size of the stakeholder groups, and the process for determining the final concept map were generally not reported. The findings from the review will be used to inform the development of our reporting guideline for concept mapping research.

Methodology reporting improved over time in 176,469 randomized controlled trials

OBJECTIVES

To describe randomized controlled trial (RCT) methodology reporting over time.

STUDY DESIGN AND SETTING

We used a deep learning-based sentence classification model based on the Consolidated Standards of Reporting Trials (CONSORT) statement, considered minimum requirements for reporting RCTs. We included 176,469 RCT reports published between 1966 and 2018. We analyzed the reporting trends over 5-year time periods, grouping trials from 1966 to 1990 in a single stratum. We also explored the effect of journal impact factor (JIF) and medical discipline.

RESULTS

Population, Intervention, Comparator, Outcome (PICO) items were commonly reported during each period, and reporting increased over time (e.g., interventions: 79.1% during 1966-1990 to 87.5% during 2010-2018). Reporting of some methods information has increased, although there is room for improvement (e.g., sequence generation: 10.8-41.8%). Some items are reported infrequently (e.g., allocation concealment: 5.1-19.3%). The number of items reported and JIF are weakly correlated (Pearson's r (162,702) = 0.16, P < 0.001). The differences in the proportion of items reported between disciplines are small (<10%).

CONCLUSION

Our analysis provides large-scale quantitative support for the hypothesis that RCT methodology reporting has improved over time. Extending these models to all CONSORT items could facilitate compliance checking during manuscript authoring and peer review, and support metaresearch.

Methods and results of studies on reporting guideline adherence are poorly reported: a meta-research study

OBJECTIVES

We investigated recent meta-research studies on adherence to four reporting guidelines to determine the proportion that provided (1) an explanation for how adherence to guideline items was rated and (2) results from all included individual studies. We examined conclusions of each meta-research study to evaluate possible repetitive and similar findings.

STUDY DESIGN AND SETTING

A cross-sectional meta-research study. MEDLINE (Ovid) was searched on July 5, 2022 for studies that used any version of the Consolidated Standards of Reporting Trials, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Standards for the Reporting of Diagnostic Accuracy Studies, or Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines or their extensions to evaluate reporting.

RESULTS

Of 148 included meta-research studies published between August 2020 and June 2022, 14 (10%, 95% confidence interval [CI] 6%-15%) provided a fully replicable explanation of how they coded the adherence ratings and 49 (33%, 95% CI 26%-41%) completely reported individual study results. Of 90 studies that classified reporting as adequate or inadequate in the study abstract, six (7%, 95% CI 3%-14%) concluded that reporting was adequate, but none of those six studies provided information on how items were coded or provided item-level results for included studies.

CONCLUSION

Almost all included meta-research studies found that reporting in health research is suboptimal. However, few of these reported enough information for verification or replication.

Reminding Peer Reviewers of Reporting Guideline Items to Improve Completeness in Published Articles: Primary Results of 2 Randomized Trials

Importance

Numerous studies have shown that adherence to reporting guidelines is suboptimal.

Objective

To evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles.

Design, Setting, and Participants

Two parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group.

Interventions

The first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded.

Main Outcomes and Measures

The difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles.

Results

In the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%).

Conclusions and Relevance

These 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future.

Trial Registration

ClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

Data source profile reporting by studies that use routinely collected health data to explore the effects of drug treatment

BACKGROUND

Routinely collected health data (RCD) are important resource for exploring drug treatment effects. Adequate reporting of data source profiles may increase the credibility of evidence generated from these data. This study conducted a systematic literature review to evaluate the reporting characteristics of databases used by RCD studies to explore the effects of drug treatment.

METHODS

Observational studies published in 2018 that used RCD to explore the effects of drug treatment were identified by searching PubMed. We categorized eligible reports into two groups by journal impact factor (IF), including the top 5 general medical journals (NEJM, Lancet, JAMA, BMJ and JAMA Internal Medicine) and the other journals. The reporting characteristics of the databases used were described and compared between the two groups and between studies citing and not citing database references.

RESULTS

A total of 222 studies were included, of which 53 (23.9%) reported that they applied data linkage, 202 (91.0%) reported the type of database, and 211 (95.0%) reported the coverage of the data source. Only 81 (36.5%) studies reported the timeframe of the database. Studies in high-impact journals were more likely to report that they applied data linkage (65.1% vs. 20.2%) and used electronic medical records (EMR) (73.7% vs. 30.0%) and national data sources (77.8% vs. 51.3%) than those published in other medical journals. There were 137/222 (61.7%) cited database references. Studies with database-specific citations had better reporting of the data sources and were more likely to publish in high-impact journals than those without (mean IF, 6.08 vs. 4.09).

CONCLUSIONS

Some deficits were found in the reporting quality of databases in studies that used RCD to explore the effects of drug treatment. Studies citing database-specific references may provide detailed information regarding data source characteristics. The adoption of reporting guidelines and education on their use is urgently needed to promote transparency by research groups.

Incomplete reporting of complex interventions: a call to action for journal editors to review their submission guidelines

Reporting of intervention research has been inadequate for many years. The development and promotion of freely available checklists aims to address this problem by providing researchers with a list of items that require reporting to enable study interpretation and replication. In this commentary, we present evidence from a recent systematic review of 51 randomised controlled trials published 2015-2020 that inadequate intervention reporting remains a widespread issue and that checklists are not being used to describe all intervention components. In 2022, we assessed the submission guidelines of 33 journals that published articles included in our review and found that just one at the time encouraged the use of reporting checklists for all intervention components. To drive progress, we contacted the editors of the other 32 journals and requested that they update their submission guidelines in response. We conclude by highlighting the waste associated with current practices and encourage journals from all fields to urgently review their submission guidelines. Only through collective action can we build an evidence base that is fit for purpose.

Quality of Reporting Using Good Reporting of A Mixed Methods Study Criteria in Chiropractic Mixed Methods Research: A Methodological Review

OBJECTIVE

The purpose of this review was to examine the reporting in chiropractic mixed methods research using Good Reporting of A Mixed Methods Study (GRAMMS) criteria.

METHODS

In this methodological review, we searched MEDLINE, Embase, CINAHL, and the Index to Chiropractic Literature from the inception of each database to December 31, 2020, for chiropractic studies reporting the use of both qualitative and quantitative methods or mixed qualitative methods. Pairs of reviewers independently screened titles, abstracts, and full-text studies, extracted data, and appraised reporting using the GRAMMS criteria and risk of bias with the Mixed Methods Appraisal Tool (MMAT). Generalized estimating equations were used to explore factors associated with reporting using GRAMMS criteria.

RESULTS

Of 1040 citations, 55 studies were eligible for review. Thirty-seven of these 55 articles employed either a multistage or convergent mixed methods design, and, on average, 3 of 6 GRAMMS items were reported among included studies. We found a strong positive correlation in scores between the GRAMMS and MMAT instruments (r = 0.78; 95% CI, 0.66-0.87). In our adjusted analysis, publications in journals indexed in Web of Science (adjusted odds ratio = 2.71; 95% CI, 1.48-4.95) were associated with higher reporting using GRAMMS criteria. Three of the 55 studies fully adhered to all 6 GRAMMS criteria, 4 studies adhered to 5 criteria, 10 studies adhered to 4 criteria, and the remaining 38 adhered to 3 criteria or fewer.

CONCLUSION

Our findings suggest that reporting in chiropractic mixed methods research using GRAMMS criteria was poor, particularly among studies with a higher risk of bias.

Recommendations for Intraoperative Adverse Events Data Collection in Clinical Studies and Study Protocols. An ICARUS Global Surgical Collaboration Study

Introduction

Intraoperative adverse events (iAEs) occur and have the potential to impact the postoperative course. However, iAEs are underreported and are not routinely collected in the contemporary surgical literature. There is no widely utilized system for the collection of essential aspects of iAEs, and there is no established database for the standardization and dissemination of this data that likely have implications for outcomes and patient safety. The Intraoperative Complication Assessment and Reporting with Universal Standards (ICARUS) Global Surgical Collaboration initiated a global effort to address these shortcomings, and the establishment of an adverse event data collection system is an essential step. In this study, we present the core-set variables for collecting iAEs that were based on the globally validated ICARUS criteria for surgical/interventional and anesthesiologic intraoperative adverse event collection and reporting.

Material and Methods

This article includes three tools to capture the essential aspects of iAEs. The core-set variables were developed from the globally validated ICARUS criteria for reporting iAEs (item 1). Next, the summary table was developed to guide researchers in summarizing the accumulated iAE data in item 1 (item 2). Finally, this article includes examples of the method and results sections to include in a manuscript reporting iAE data (item 3). Then, 5 scenarios demonstrating best practices for completing items 1-3 were presented both in prose and in a video produced by the ICARUS collaboration.

Dissemination

This article provides the surgical community with the tools for collecting essential iAE data. The ICARUS collaboration has already published the 13 criteria for reporting surgical adverse events, but this article is unique and essential as it actually provides the tools for iAE collection. The study team plans to collect feedback for future directions of adverse event collection and reporting.

Highlights

This article represents a novel, fully-encompassing system for the data collection of intraoperative adverse events.The presented core-set variables for reporting intraoperative adverse events are not based solely on our opinion, but rather are synthesized from the globally validated ICARUS criteria for reporting intraoperative adverse events.Together, the included text, figures, and ICARUS collaboration-produced video should equip any surgeon, anesthesiologist, or nurse with the tools to properly collect intraoperative adverse event data.Future directions include translation of this article to allow for the widest possible adoption of this important collection system.

Implementation of e-Mental Health Interventions for Informal Caregivers of Adults With Chronic Diseases: Mixed Methods Systematic Review With a Qualitative Comparative Analysis and Thematic Synthesis

BACKGROUND

Informal caregivers commonly experience mental health difficulties related to their caregiving role. e-Mental health interventions provide mental health support in a format that may be more accessible to informal caregivers. However, e-mental health interventions are seldom implemented in real-world practice.

OBJECTIVE

This mixed methods systematic review aimed to examine factors associated with the effectiveness and implementation of e-mental health interventions for informal caregivers of adults with chronic diseases. To achieve this aim, two approaches were adopted: combinations of implementation and intervention characteristics sufficient for intervention effectiveness were explored using qualitative comparative analysis, and barriers to and facilitators of implementation of e-mental health interventions for informal caregivers were explored using thematic synthesis.

METHODS

We identified relevant studies published from January 1, 2007, to July 6, 2022, by systematically searching 6 electronic databases and various secondary search strategies. Included studies reported on the effectiveness or implementation of e-mental health interventions for informal caregivers of adults with cancer, chronic obstructive pulmonary disease, dementia, diabetes, heart disease, or stroke. Randomized controlled trials reporting on caregivers' mental health outcomes were included in a crisp-set qualitative comparative analysis. We assessed randomized controlled trials for bias using the Risk of Bias 2.0 tool, and we assessed how pragmatic or explanatory their trial design was using the Pragmatic Explanatory Continuum Indicator Summary 2 tool. Studies of any design reporting on implementation were included in a thematic synthesis using the Consolidated Framework for Implementation Research to identify barriers to and facilitators of implementation.

RESULTS

Overall, 53 reports, representing 29 interventions, were included in the review. Most interventions (27/29, 93%) focused on informal cancer or dementia caregivers. In total, 14 reports were included in the qualitative comparative analysis, exploring conditions including the presence of peer or professional support and key persuasive design features. Low consistency and coverage prevented the determination of condition sets sufficient for intervention effectiveness. Overall, 44 reports were included in the thematic synthesis, and 152 barriers and facilitators were identified, with the majority related to the intervention and individual characteristic domains of the Consolidated Framework for Implementation Research. Implementation barriers and facilitators in the inner setting (eg, organizational culture) and outer setting (eg, external policies and resources) domains were largely unexplored.

CONCLUSIONS

e-Mental health interventions for informal caregivers tend to be well-designed, with several barriers to and facilitators of implementation identified related to the intervention and individual user characteristics. Future work should focus on exploring the views of stakeholders involved in implementation to determine barriers to and facilitators of implementing e-mental health interventions for informal caregivers, focusing on inner and outer setting barriers and facilitators.

TRIAL REGISTRATION

PROSPERO (International Prospective Register of Systematic Reviews) CRD42020155727; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020155727.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2019-035406.

Is the future of peer review automated?

The rising rate of preprints and publications, combined with persistent inadequate reporting practices and problems with study design and execution, have strained the traditional peer review system. Automated screening tools could potentially enhance peer review by helping authors, journal editors, and reviewers to identify beneficial practices and common problems in preprints or submitted manuscripts. Tools can screen many papers quickly, and may be particularly helpful in assessing compliance with journal policies and with straightforward items in reporting guidelines. However, existing tools cannot understand or interpret the paper in the context of the scientific literature. Tools cannot yet determine whether the methods used are suitable to answer the research question, or whether the data support the authors' conclusions. Editors and peer reviewers are essential for assessing journal fit and the overall quality of a paper, including the experimental design, the soundness of the study's conclusions, potential impact and innovation. Automated screening tools cannot replace peer review, but may aid authors, reviewers, and editors in improving scientific papers. Strategies for responsible use of automated tools in peer review may include setting performance criteria for tools, transparently reporting tool performance and use, and training users to interpret reports.

Reporting of health equity considerations in equity-relevant observational studies: Protocol for a systematic assessment

Background: The mitigation of unfair and avoidable differences in health is an increasing global priority. Observational studies including cohort, cross-sectional and case-control studies tend to report social determinants of health which could inform evidence syntheses on health equity and social justice. However, the extent of reporting and analysis of equity in equity-relevant observational studies is unknown. Methods: We define studies which report outcomes for populations at risk of experiencing inequities as “equity-relevant”. Using a random sampling technique we will identify 320 equity-relevant observational studies published between 1 January 2020 to 27 April 2022 by searching the MEDLINE database. We will stratify sampling by 1) studies in high-income countries (HIC) and low- and middle-income countries (LMIC) according to the World Bank classification, 2) studies focused on COVID and those which are not, 3) studies focused on populations at risk of experiencing inequities and those on general populations that stratify their analyses. We will use the PROGRESS framework which stands for place of residence, race or ethnicity, occupation, gender or sex, religion, education, socioeconomic status, social capital, to identify dimensions where inequities may exist. Using a previously developed data extraction form we will pilot-test on eligible studies and revise as applicable. Conclusions: The proposed methodological assessment of reporting will allow us to systematically understand the current reporting and analysis practices for health equity in observational studies. The findings of this study will help inform the development of the equity extension for the STROBE (Strengthening the Reporting of Observational studies in Epidemiology) reporting guidelines.

TIDieR-telehealth: precision in reporting of telehealth interventions used in clinical trials - unique considerations for the Template for the Intervention Description and Replication (TIDieR) checklist

Background

Recent international health events have led to an increased proliferation of remotely delivered health interventions. Even with the pandemic seemingly coming under control, the experiences of the past year have fueled a growth in ideas and technology for increasing the scope of remote care delivery. Unfortunately, clinicians and health systems will have difficulty with the adoption and implementation of these interventions if ongoing and future clinical trials fail to report necessary details about execution, platforms, and infrastructure related to these interventions. The purpose was to develop guidance for reporting of telehealth interventions.

Methods

A working group from the US Pain Management Collaboratory developed guidance for complete reporting of telehealth interventions. The process went through 5-step process from conception to final checklist development with input for many stakeholders, to include all 11 primary investigators with trials in the Collaboratory.

Results

An extension focused on unique considerations relevant to telehealth interventions was developed for the Template for the Intervention Description and Replication (TIDieR) checklist.

Conclusion

The Telehealth Intervention guideline encourages use of the Template for the Intervention Description and Replication (TIDieR) checklist as a valuable tool (TIDieR-Telehealth) to improve the quality of research through a reporting guide of relevant interventions that will help maximize reproducibility and implementation.

Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis

BACKGROUND

We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs.

METHODS AND FINDINGS

We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.

CONCLUSIONS

We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Risk of bias in chiropractic mixed methods research: a secondary analysis of a meta-epidemiological review

OBJECTIVE

To examine the risk of bias in chiropractic mixed methods research.

METHODS

We performed a secondary analysis of a meta-epidemiological review of chiropractic mixed methods studies. We assessed risk of bias with the Mixed Methods Appraisal Tool (MMAT) and used generalized estimating equations to explore factors associated with risk of bias.

RESULTS

Among 55 eligible studies, a mean of 62% (6.8 [2.3]/11) of MMAT items were fulfilled. In our adjusted analysis, studies published since 2010 versus pre-2010 (adjusted odds ratio [aOR] = 2.26; 95% confidence interval [CI], 1.39 to 3.68) and those published in journals with an impact factor versus no impact factor (aOR = 2.21; 95% CI, 1.33 to 3.68) were associated with lower risk of bias.

CONCLUSION

Our findings suggest opportunities for improvement in the quality of conduct among published chiropractic mixed methods studies. Author compliance with the MMAT criteria may reduce methodological bias in future mixed methods research.

Writing up your clinical trial report for a scientific journal: the REPORT trial guide for effective and transparent research reporting without spin

The REPORT guide is a ‘How to’ guide to help you report your clinical research in an effective and transparent way. It is intended to supplement established first choice reporting tools, such as Consolidated Standards of Reporting Trials (CONSORT), by adding tacit knowledge (ie, learnt, informal or implicit knowledge) about reporting topics that we have struggled with as authors or see others struggle with as journal reviewers or editors. We focus on the randomised controlled trial, but the guide also applies to other study designs. Topics included in the REPORT guide cover reporting checklists, trial report structure, choice of title, writing style, trial registry and reporting consistency, spin or reporting bias, transparent data presentation (figures), open access considerations, data sharing and more. Preprint (open access): https://doi.org/10.31219/osf.io/qsxdz.

Reporting of abstracts in studies that used routinely collected data for exploring drug treatment effects: a cross-sectional survey

Background

In recent years, studies that used routinely collected data (RCD), such as electronic medical records and administrative claims, for exploring drug treatment effects, including effectiveness and safety, have been increasingly published. Abstracts of such studies represent a highly attended source for busy clinicians or policy-makers, and are important for indexing by literature database. If less clearly presented, they may mislead decisions or indexing. We thus conducted a cross-sectional survey to systematically examine how the abstracts of such studies were reported.

Methods

We searched PubMed to identify all observational studies published in 2018 that used RCD for assessing drug treatment effects. Teams of methods-trained collected data from eligible studies using pilot-tested, standardized forms that were developed and expanded from “The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology” (RECORD-PE) statement. We used descriptive analyses to examine how authors reported data source, study design, data analysis, and interpretation of findings.

Results

A total of 222 studies were included, of which 118 (53.2%) reported type of database used, 17 (7.7%) clearly reported database linkage, and 140 (63.1%) reported coverage of data source. Only 44 (19.8%) studies stated a predefined hypothesis, 127 (57.2%) reported study design, 140 (63.1%) reported statistical models used, 142 (77.6%) reported adjusted estimates, 33 (14.9%) mentioned sensitivity analyses, and 39 (17.6%) made a strong claim about treatment effect. Studies published in top 5 general medicine journals were more likely to report the name of data source (94.7% vs. 67.0%) and study design (100% vs. 53.2%) than those in other journals.

Conclusions

The under-reporting of key methodological features in abstracts of RCD studies was common, which would substantially compromise the indexing of this type of literature and prevent the effective use of study findings. Substantial efforts to improve the reporting of abstracts in these studies are highly warranted.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01482-9.

Predictors of Higher Quality of Systematic Reviews Addressing Nutrition and Cancer Prevention

Systematic reviews/meta-analyses (SR/MAs) are considered a reliable source of information in healthcare. We aimed to explore the association of several characteristics of SR/MAs addressing nutrition in cancer prevention and their quality/risk of bias (using assessments from AMSTAR-2 and ROBIS tools). The analysis included 101 SR/MAs identified in a systematic survey. Associations of each specified characteristic (e.g., information about the protocol, publication year, reported use of GRADE, or other methods for assessing overall certainty of evidence) with the number of AMSTAR-2 not met (‘No’ responses) and the number of ROBIS items met (‘Probably Yes’ or “Yes’ responses) were examined. Poisson regression was used to identify predictors of the number of ‘No’ answers (indicating lower quality) for all AMSTAR-2 items and the number of ‘Yes’ or ‘Probably Yes’ answers (indicating higher quality/lower concern for bias) for all ROBIS items. Logistic regression was used to identify variables associated with at least one domain assessed as ‘low concern for bias’ in the ROBIS tool. In multivariable analysis, SR/MAs not reporting use of any quality/risk of bias assessment instrument for primary studies were associated with a higher number of ‘No’ answers for all AMSTAR-2 items (incidence rate ratio (IRR) 1.26, 95% confidence interval (CI) 1.09–1.45), and a lower number of ‘Yes’ or ‘Probably Yes’ answers for all ROBIS items (IRR 0.76, 95% CI 0.66–0.87). Providing information about the protocol and search for unpublished studies was associated with a lower number of ‘No’ answers (IRR 0.73, 95% CI 0.56–0.97 and IRR 0.75, 95% CI 0.59–0.95, respectively) and a higher number of ‘Yes’ or ‘Probably Yes’ answers (IRR 1.43, 95% CI 1.17–1.74 and IRR 1.28, 95% CI 1.07–1.52, respectively). Not using at least one quality/risk of bias assessment tool for primary studies within an SR/MA was associated with lower odds that a study would be assessed as ‘low concern for bias’ in at least one ROBIS domain (odds ratio 0.061, 95% CI 0.007–0.527). Adherence to methodological standards in the development of SR/MAs was associated with a higher overall quality of SR/MAs addressing nutrition for cancer prevention.

Evaluation of quality of abstracts of randomized controlled trials on procedural sedation in children after publication of CONSORT guidelines for abstracts: A systematic review

The extension of the Consolidated Standards of Reporting Trials (CONSORT) statement provides guidelines for abstracts of randomized controlled trials (RCTs). This study was done to assess the reporting quality of abstracts of RCTs, on procedural sedation in children and identify factors associated with better quality. A PubMed search was conducted from inception of database till July 2017 to identify RCTs on procedural sedation in children. Search terms used were (procedural [All Fields] AND sedation [All Fields]) AND ("child" [MeSH Terms] OR "child" [All Fields] OR "children" [All Fields]) were included in the analysis, while primary RCTs, published in the English language unstructured abstracts, secondary analysis of primary RCTs and studies not exclusively on children we excluded. Our search strategy initially yielded 582 abstracts. Out of these, 535 abstracts were excluded. 47 articles were included in the final analysis. We extracted basic information and data on CONSORT items from abstracts. Each abstract was assessed using a 16-item composite abstract score (CAS) based on the CONSORT guidelines. This abstract quality was further explored by Method Score and by Result Score. Regression analysis was conducted to analyze factors associated with reporting quality. In majority of the abstracts, only objectives and conclusion were adequately reported. Inadequately reported items in >90% of abstracts included randomization, trial status, registration & funding. There was no significant difference in the CAS of abstracts (mean ± SD) published in & before 2008 (12.63 ± 4.0), to those published after 2009 (12.48 ± 4.23). Similarly, there was no significant difference in Result Score and Method Score of the abstracts. After the publication of 'CONSORT for abstracts' guideline, the quality of abstracts of RCTs on procedural sedation has shown suboptimal improvement. We suggest stricter adherence to guidelines by editors and reviewers. A checklist for adherence to CONSORT guidelines could be introduced during submission for the same.

Effects of Setting on Psychedelic Experiences, Therapies, and Outcomes: A Rapid Scoping Review of the Literature

The health and well-being impacts of art and aesthetic experiences have been rigorously studied by a range of disciplines, including cognitive neuroscience, psychiatry, public health, and translational clinical research. These experiences, encompassed in the concepts of set and setting, have long been claimed to be pivotal in determining the acute and enduring effects of psychedelic experiences. Responding to the field's longstanding emphasis on the role and value of setting, a rapid scoping review was undertaken to identify the extent to which effects of setting and aesthetics on psychedelic experiences and therapies have been explicitly studied. It offers an analysis of the strengths and limitations of the extant literature and discusses evidentiary gaps as well as evidentiary opportunities for the field. The 43 included studies indicate apparent consensus regarding the importance of setting in psychedelic therapies, as well as consistent interest in theorizing about these effects. However, this consensus has yet to generate consistent, prospective, rigorous tests of setting and its complexities. As a result, the field continues to lack understanding or agreement regarding the effects of various specific elements of setting, the mechanisms by which they affect outcomes, for whom these effects occur, under what circumstances, given what conditions, and other critical factors. Further studies of setting and aesthetics in the context of psychedelic therapies are likely to not only improve these therapies and their delivery, but also inform considerations of setting and aesthetics for non-psychedelic interventions.

Are systematic reviews addressing nutrition for cancer prevention trustworthy? A systematic survey of quality and risk of bias

Context

The last 30 years have yielded a vast number of systematic reviews and/or meta-analyses addressing the link between nutrition and cancer risk.

Objective

The aim of this survey was to assess overall quality and potential for risk of bias in systematic reviews and meta-analyses (SRMAs) that examined the role of nutrition in cancer prevention.

Data Sources

MEDLINE, Embase, and the Cochrane Library databases were searched (last search performed November 2018).

Study Selection

Studies identified as SRMAs that investigated a nutritional or dietary intervention or exposure for cancer prevention in the general population or in people at risk of cancer and in which primary studies had a comparison group were eligible for inclusion. Screening, data extraction, and quality assessment were conducted independently by 2 reviewers.

Data Extraction

Altogether, 101 studies were randomly selected for analysis. The methodological quality and risk of bias were evaluated using the AMSTAR-2 and ROBIS tools, respectively.

Results

Most SRMAs included observational studies. Less than 10% of SRMAs reported a study protocol, and only 51% of SRMAs assessed the risk of bias in primary studies. Most studies conducted subgroup analyses, but only a few reported tests of interaction or specified subgroups of interest a priori. Overall, according to AMSTAR-2, only 1% of SRMAs were of high quality, while 97% were of critically low quality. Only 3% had a low risk of bias, according to ROBIS.

Conclusions

This systematic survey revealed substantial limitations with respect to quality and risk of bias of SRMAs. SRMAs examining nutrition and cancer prevention cannot be considered trustworthy, and results should be interpreted with caution. Peer reviewers as well as users of SRMAs should be advised to use the AMSTAR-2 and/or ROBIS instruments to help to determine the overall quality and risk of bias of SRMAs.

Systematic Review Registration

PROSPERO registration number CRD42019121116.

EMBRACE-WATERS statement: Recommendations for reporting of studies on antimicrobial resistance in wastewater and related aquatic environments

BACKGROUND

A One Health approach requires integrative research to elucidate antimicrobial resistance (AMR) in the environment and the risks it poses to human health. Research on this topic involves experts from diverse backgrounds and professions. Shortcomings exist in terms of consistent, complete, and transparent reporting in many environmental studies. Standardized reporting will improve the quality of scientific papers, enable meta-analyses and enhance the communication among different experts. In this study, we aimed to generate a consensus of reporting standards for AMR research in wastewater and related aquatic environments.

METHODS

Based on a risk of bias assessment of the literature in a systematic review, we proposed a set of study quality indicators. We then used a multistep modified Delphi consensus to develop the EMBRACE-WATERS statement (rEporting antiMicroBial ResistAnCE in WATERS), a checklist of recommendations for reporting in studies of AMR in wastewater and related aquatic environments.

FINDINGS

Consensus was achieved among a multidisciplinary panel of twenty-one experts in three steps. The developed EMBRACE-WATERS statement incorporates 21 items. Each item contains essential elements of high-quality reporting and is followed by an explanation of their rationale and a reporting-example. The EMBRACE-WATERS statement is primarily intended to be used by investigators to ensure transparent and comprehensive reporting of their studies. It can also guide peer-reviewers and editors in evaluation of manuscripts on AMR in the aquatic environment. This statement is not intended to be used to guide investigators on the methodology of their research.

INTERPRETATION

We are hopeful that this statement will improve the reporting quality of future studies of AMR in wastewater and related aquatic environments. Its uptake would generate a common language to be used among researchers from different disciplines, thus advancing the One Health approach towards understanding AMR spread across aquatic environments. Similar initiatives are needed in other areas of One Health research.

Reliability of Trial Information Across Registries for Trials With Multiple Registrations: A Systematic Review

IMPORTANCE

Clinical trial registries are important for gaining an overview of ongoing research efforts and for deterring and identifying publication bias and selective outcome reporting. The reliability of the information in trial registries is uncertain.

OBJECTIVE

To assess the reliability of information across registries for trials with multiple registrations.

EVIDENCE REVIEW

For this systematic review, 360 protocols of randomized clinical trials (RCTs) approved by research ethics committees in Switzerland, the UK, Canada, and Germany in 2012 were evaluated. Clinical trial registries were searched from March to September 2019 for corresponding registrations of these RCTs. For RCTS that were recorded in more than 1 clinical trial registry, key trial characteristics that should be identical among all trial registries (ie, sponsor, funding source, primary outcome, target sample size, trial status, date of first patient enrollment, results available, and main publication indexed) were extracted in duplicate. Agreement between the different trial registries for these key characteristics was analyzed descriptively. Data analyses were conducted from May 1 to November 30, 2020. Representatives from clinical trial registries were interviewed to discuss the study findings between February 1 and March 31, 2021.

FINDINGS

The analysis included 197 RCTs registered in more than 1 trial registry (151 in 2 registries and 46 in 3 registries), with 188 trials in ClinicalTrials.gov, 185 in the European Union Drug Regulating Authorities Clinical Trials Database (EudraCT), 20 in ISRCTN, and 47 in other registries. The agreement of key information across all registries was as follows: 178 of 197 RCTs (90%; 95% CI, 85%-94%) for sponsor, 18 of 20 (90%; 95% CI, 68%-99%) for funding source (funding was not reported on ClinicalTrials.gov), 154 of 197 (78%; 95% CI, 72%-84%) for primary outcome, 90 of 197 (46%; 95% CI, 39%-53%) for trial status, 122 of 194 (63%; 95% CI, 56%-70%) for target sample size, and 43 of 57 (75%; 95% CI, 62%-86%) for the date of first patient enrollment when the comparison time was increased to 30 days (date of first patient enrollment was not reported on EudraCT). For results availability in trial registries, agreement was 122 of 197 RCTs (62%; 95% CI, 55%-69%) for summary data reported in the registry and 91 of 197 (46%; 95% CI, 39%-53%) for whether a published article with the main results was indexed. Different legal requirements were stated as the main reason for inconsistencies by representatives of clinical trial registries.

CONCLUSIONS AND RELEVANCE

In this systematic review, for a substantial proportion of registered RCTs, information about key trial characteristics was inconsistent across trial registries, raising concerns about the reliability of the information provided in these registries. Further harmonization across clinical trial registries may be necessary to increase their usefulness.

Reporting Policies in Neurosurgical Journals: A Meta-Science Study of the Current State and Case for Standardization

BACKGROUND

Reporting quality within the neurosurgical literature is low, limiting the ability of journals to act as gatekeepers for evidence-based neurosurgical care. Journal policies during article submission aim to improve reporting quality. We conducted a meta-science study characterizing the reporting policies of neurosurgical journals and other related peer-reviewed publications.

METHODS

Journals were retrieved in 7 searches using Journal Citation Reports and Google Scholar. Characteristics, impact metrics, and submission policies were extracted.

RESULTS

Of 486 results, 54 journals were included, including 27 neurosurgical and 27 related topical journals. Thirty-eight (70.4%) adopted authorship guidelines and 20 (37.0%) disclosure standards of the International Council of Medical Journal Editors. Twenty-six (48.1%) required data availability statement and 33 (61.1%) clinical trials registration. Twenty-one (38.9%) required and 11 (20.4%) recommended adherence to reporting guidelines. Twenty (37.0%) endorsed EQUATOR network guidelines. PRISMA was mentioned by 30 (55.6%) journals, CONSORT by 28 (51.9%), and STROBE by 18 (33.3%). Among neurosurgical journals, factors associated with a requirement or recommendation to follow reporting guidelines among neurosurgical journals included impact factor (P = 0.0013), Article Influence Score (P = 0.0236), SCImago h-index (P = 0.0152), SCImago journal rank (P = 0.002), and CiteScore (P = 0.0023), as well as recommendations pertaining to International Council of Medical Journal Editors authorship guidelines (P = 0.0085), ORCID (P = 0.014), clinical trials registration (P = 0.0369), or data availability statement (P = 0.0047). CONSORT, PRISMA, or STROBE delineations were significantly associated with the mention of another guideline (P < 0.01).

CONCLUSIONS

Neurosurgical journal submission policies are inconsistent. Frameworks to improve reporting quality are uncommonly used. Increasing rigor and standardization of reporting policies across journals publishers may improve quality.

GoodReports: developing a website to help health researchers find and use reporting guidelines

BACKGROUND

Th EQUATOR Network improves the quality and transparency in health research, primarily by promoting awareness and use of reporting guidelines. In 2018, the UK EQUATOR Centre launched GoodReports.org , a website that helps authors find and use reporting guidelines. This paper describes the tool's development so far. We describe user experience and behaviour of using GoodReports.org both inside and outside a journal manuscript submission process. We intend to use our findings to inform future development and testing of the tool.

METHODS

We conducted a survey to collect data on user experience of the GoodReports website. We cross-checked a random sample of 100 manuscripts submitted to a partner journal to describe the level of agreement between the tool's checklist recommendation and what we would have recommended. We compared the proportion of authors submitting a completed reporting checklist alongside their manuscripts between groups exposed or not exposed to the GoodReports tool. We also conducted a study comparing completeness of reporting of manuscript text before an author received a reporting guideline recommendation from GoodReports.org with the completeness of the text subsequently submitted to a partner journal.

RESULTS

Seventy percent (423/599) of survey respondents rated GoodReports 8 or more out of 10 for usefulness, and 74% (198/267) said they had made changes to their manuscript after using the website. We agreed with the GoodReports reporting guideline recommendation in 84% (72/86) of cases. Of authors who completed the guideline finder questionnaire, 14% (10/69) failed to submit a completed checklist compared to 30% (41/136) who did not use the tool. Of the 69 authors who received a GoodReports reporting guideline recommendation, 20 manuscript pairs could be reviewed before and after use of GoodReports. Five included more information in their methods section after exposure to GoodReports. On average, authors reported 57% of necessary reporting items before completing a checklist on GoodReports.org and 60% after.

CONCLUSION

The data suggest that reporting guidance is needed early in the writing process, not at submission stage. We are developing GoodReports by adding more reporting guidelines and by creating editable article templates. We will test whether GoodReports users write more complete study reports in a randomised trial targeting researchers starting to write health research articles.

Meaningful associations in the adolescent brain cognitive development study

The Adolescent Brain Cognitive Development (ABCD) Study is the largest single-cohort prospective longitudinal study of neurodevelopment and children's health in the United States. A cohort of n = 11,880 children aged 9-10 years (and their parents/guardians) were recruited across 22 sites and are being followed with in-person visits on an annual basis for at least 10 years. The study approximates the US population on several key sociodemographic variables, including sex, race, ethnicity, household income, and parental education. Data collected include assessments of health, mental health, substance use, culture and environment and neurocognition, as well as geocoded exposures, structural and functional magnetic resonance imaging (MRI), and whole-genome genotyping. Here, we describe the ABCD Study aims and design, as well as issues surrounding estimation of meaningful associations using its data, including population inferences, hypothesis testing, power and precision, control of covariates, interpretation of associations, and recommended best practices for reproducible research, analytical procedures and reporting of results.

Quality of reporting in chiropractic mixed methods research: a methodological review protocol

Background

Mixed methods designs are increasingly used in health care research to enrich findings. However, little is known about the frequency of use of this methodology in chiropractic research, or the quality of reporting among chiropractic studies using mixed methods.

Objective

To quantify the use and quality of mixed methods in chiropractic research, and explore the association of study characteristics (e.g., authorship, expertise, journal impact factor, country and year of publication) with reporting quality.

Methods

We will conduct a systematic search of MEDLINE, EMBASE, CINAHL, and the Index to Chiropractic Literature to identify all chiropractic mixed methods studies published from inception of each database to December 31, 2020. Articles reporting the use of both qualitative and quantitative methods, or mixed qualitative methods, will be included. Pairs of reviewers will perform article screening, data extraction, risk of bias with the Mixed Methods Appraisal Tool (MMAT), and appraisal of reporting quality using the Good Reporting of A Mixed Methods Study (GRAMMS) guideline. We will explore the correlation between GRAMMS and MMAT scores, and construct generalized estimating equations to explore factors associated with reporting quality.

Discussion

This will be the first methodological review to examine the reporting quality of published mixed methods studies involving chiropractic research. The results of our review will inform opportunities to improve reporting in chiropractic mixed methods studies. Our results will be disseminated in a peer-reviewed publication and presented publicly at conferences and as part of a doctoral thesis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12998-021-00395-0.

How to Choose a Survival Period? The Impact of Antibiotic Use on OS or PFS in NSCLC Patients Treated With Immune Checkpoint Inhibitors: A Systematic Review and Meta-Analysis

Background:

The development of immunotherapy has dramatically changed the treatment of non-small-cell lung cancer. The negative association of antibiotics on the clinical activity of immune checkpoint inhibitors in patients with NSCLC is well known.

Methods:

PubMed, Embase, and Medline databases were searched until January 11, 2020. We included retrospective studies of ICIs (e.g., PD-1, PD-L1, and CTLA-4). The clinical outcomes were progression-free survival (PFS) and overall survival (OS). Hazard ratios (HRs) and 95% confidence intervals (CIs) were calculated, and subgroup and sensitivity analyses were performed.

Results:

Our results indicated that the use of antibiotics reduced the survival of NSCLC patients treated with ICIs. The pooled HRs of PFS and OS were HR = 1.41 (95% CI = 1.23-1.61; P < 0.001) and HR = 2.16 (95% CI = 1.79-2.60; P < 0.001). We divided the studies into 5 subgroups according to antibiotic exposure time. Subgroup analysis showed that the patients that were administered antibiotics [−60 days; 0 days] or [−30 days; 0 days] before the initiation of ICIs treatment had a poorer OS rate, whereas those patients that were administered antibiotics [0 days; 30 days] after the initiation of ICIs treatment had a poorer PFS rate. In summary, ATB treatment in patients [−60 days; +30 days] near the initiation of ICIs treatment significantly reduced the survival in NSCLC patients.

Conclusion:

Our results indicated that ATB use is negatively associated with survival in NSCLC patients treated with ICIs immunotherapy. Similar studies involving a larger sample of cases are still being published. This meta-analysis identified that the timing of ATB treatment in NSCLC patients receiving ICIs immunotherapy has different effects on the OS and PFS of these patients. ATB treatment prior to the initiation of ICIs treatment affects OS, whereas ATB treatment after the initiation of ICIs treatment affects PFS.

Adherence to participant flow diagrams in trials on postoperative pain management after total hip and knee arthroplasty: a methodological review

BACKGROUND

The Consolidated Standards of Reporting Trials (CONSORT) statement aims to improve transparent reporting of randomised clinical trials. It comprises a participant flow diagram with the reporting of essential numbers for enrolment, allocation and analyses. We aimed to quantify the use of participant flow diagrams in randomised clinical trials on postoperative pain management after total hip and knee arthroplasty.

METHODS

We searched PubMed, Embase and CENTRAL up till January 2020. The primary outcome was the proportion of trials with adequate reporting of participant flow diagrams, defined as reporting of number of participants screened for eligibility, randomised and included in the primary analysis. Secondary outcomes were recruitment (randomised:screened) and retention (analysed:randomised) rates, reporting of a statistical strategy, reasons for exclusion from the primary analysis and handling of missing outcome data. Trends over time were assessed with statistical process control.

RESULTS

Of the 570 included trials, we found adequate reporting in 240 (42%). Reporting with participant flow diagram increased significantly over time. Median recruitment was 73% (IQR 44-91%), and retention was 97% (IQR 93-100%). These rates did not change over time. Trials with adequate reporting of participant flow were more likely to report a statistical strategy (41% vs 8%), reasons for post-randomisation exclusions (100% vs 55%) and handling of missing outcome data (14% vs 6%).

CONCLUSIONS

Adherence to participant flow diagrams for RCTs has increased significantly over time. Still, there is room for improvement of adequate reporting of flow diagrams, to increase transparency of trials details.

The Use of Music in the Treatment and Management of Serious Mental Illness: A Global Scoping Review of the Literature

Mental and substance use disorders have been identified as the leading cause of global disability, and the global burden of mental illness is concentrated among those experiencing disability due to serious mental illness (SMI). Music has been studied as a support for SMIs for decades, with promising results; however, a lack of synthesized evidence has precluded increased uptake of and access to music-based approaches. The purpose of this scoping review was to identify the types and quantity of research at intersections of music and SMIs, document evidentiary gaps and opportunities, and generate recommendations for improving research and practice. Studies were included if they reported on music's utilization in treating or mitigating symptoms related to five SMIs: schizophrenia, bipolar disorder, generalized anxiety disorder, major depressive disorder, or post-traumatic stress disorder. Eight databases were searched; screening resulted in 349 included studies for data extraction. Schizophrenia was the most studied SMI, with bipolar disorder studied the least. Demographics, settings, and activity details were found to be inconsistently and insufficiently reported; however, listening to recorded music emerged as the most common musical activity, and activity details appeared to have been affected by the conditions under study. RCTs were the predominant study design, and 271 unique measures were utilized across 289 primary studies. Over two-thirds of primary studies (68.5%) reported positive results, with 2.8% reporting worse results than the comparator, and 12% producing indeterminate results. A key finding is that evidence synthesis is precluded by insufficient reporting, widely varied outcomes and measures, and intervention complexity; as a result, widespread changes are necessary to reduce heterogeneity (as feasible), increase replicability and transferability, and improve understandings of mechanisms and causal pathways. To that end, five detailed recommendations are offered to support the sharing and development of information across disciplines.

Inexpensive Systemic Inflammatory Biomarkers in Ovarian Cancer: An Umbrella Systematic Review of 17 Prognostic Meta-Analyses

The association of several inflammation-based biomarkers [lymphocyte-to-monocyte, neutrophil-to-lymphocyte, and platelet-to-lymphocyte ratios (LMR, NLR, and PLR, respectively)] with the survival of epithelial ovarian cancer (EOC) patients has been extensively investigated in several systematic reviews and meta-analyses (MAs) of observational studies. The aim of this umbrella systematic review is to appraise all available results in published MAs that explored the association between these biomarkers and EOC outcomes. An umbrella systematic review of the current evidence for systemic inflammatory biomarkers in the peripheral blood of EOC patients was performed by searching several databases including PubMed/Medline and Web of Science. The quality of the MAs was appraised using the AMSTAR-2 tool as well as other qualitative criteria. The evidence was graded from convincing (Class I) to weak (Class IV). Our umbrella review appraised 17 MAs of retrospective studies (range: 7-16) with a number of enrolled patients ranging from 1,636 to 4,910 patients in each MA. All these MAs demonstrated that pretreatment high NLR and PLR, as well as low LMR, were independent predictors of poor overall survival and progression-free survival in EOC. Nearly all published MAs were conducted by Chinese researchers (16/17) and were redundant in their character. Another issue in these MAs is the absence of prior PROSPERO database registration as well as the earlier exclusion of the gray literature. On the other hand, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and Meta-analyses Of Observational Studies in Epidemiology (MOOSE)-based reporting guidelines were used in nine out of the 17 MAs. A good number of MAs have transparently provided funding acknowledgment. The AMSTAR-2-based assessment showed low quality in 11 out of the 17 reviewed MAs. This negative rating was largely due to the absence of critical domains. Finally, all evaluated MAs were rated as Class III or IV (suggestive and weak, respectively). Despite the power of MAs in increasing sampling and precision, the quality of the current non-randomized evidence on this topic is still weak.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO, identifier CRD42020201493.

Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC)

INTRODUCTION

Methodological studies (ie, studies that evaluate the design, conduct, analysis or reporting of other studies in health research) address various facets of health research including, for instance, data collection techniques, differences in approaches to analyses, reporting quality, adherence to guidelines or publication bias. As a result, methodological studies can help to identify knowledge gaps in the methodology of health research and strategies for improvement in research practices. Differences in methodological study names and a lack of reporting guidance contribute to lack of comparability across studies and difficulties in identifying relevant previous methodological studies. This paper outlines the methods we will use to develop an evidence-based tool-the MethodologIcal STudy reportIng Checklist-to harmonise naming conventions and improve the reporting of methodological studies.

METHODS AND ANALYSIS

We will search for methodological studies in the Cumulative Index to Nursing and Allied Health Literature, Cochrane Library, Embase, MEDLINE, Web of Science, check reference lists and contact experts in the field. We will extract and summarise data on the study names, design and reporting features of the included methodological studies. Consensus on study terms and recommended reporting items will be achieved via video conference meetings with a panel of experts including researchers who have published methodological studies.

ETHICS AND DISSEMINATION

The consensus study has been exempt from ethics review by the Hamilton Integrated Research Ethics Board. The results of the review and the reporting guideline will be disseminated in stakeholder meetings, conferences, peer-reviewed publications, in requests to journal editors (to endorse or make the guideline a requirement for authors), and on the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network and reporting guideline websites.

REGISTRATION

We have registered the development of the reporting guideline with the EQUATOR Network and publicly posted this project on the Open Science Framework (www.osf.io/9hgbq).

Declaration of use and appropriate use of reporting guidelines in high-impact rehabilitation journals is limited: a meta-research study

OBJECTIVES

The main aims of this metaresearch study conducted among high-impact rehabilitation journals were: 1) to evaluate if the use of reporting guidelines (RGs) was declared and 2) to categorize the declared use as appropriate or inappropriate.

STUDY DESIGN AND SETTING

Cross-sectional analysis of a random sample of 200 studies published in the period 2010-2019 in five generic rehabilitation journals with the highest 5-year impact factor. Randomized controlled trials, systematic reviews, observational studies, and diagnostic studies were included. Prevalence with 95% confidence intervals (CIs) was estimated for the main outcomes.

RESULTS

Among the 200 selected studies, 17.5% (95% CI: 12.2-22.8%) declared using RGs. Among these studies, 48.6% (95% CI: 32-65.1%) declared an appropriate use. There was an increasing trend over time for authors to report the use of RGs (OR: 1.31; 95% CI: 1.13-1.53). Systematic reviews (n = 54) reported more frequently the use of RGs than other study designs (35.2%).

CONCLUSION

In high-impact rehabilitation journals, a small minority of article authors declared the use of RGs. In approximately half of these studies, RGs were used inappropriately. There is an urgent need to improve the use of RGs in this field of research.

Criteria to Qualify Microorganisms as "Probiotic" in Foods and Dietary Supplements

Still relevant after 19 years, the FAO/WHO definition of probiotics can be translated into four simple and pragmatic criteria allowing one to conclude if specific strains of microorganisms qualify as a probiotic for use in foods and dietary supplements. Probiotic strains must be (i) sufficiently characterized; (ii) safe for the intended use; (iii) supported by at least one positive human clinical trial conducted according to generally accepted scientific standards or as per recommendations and provisions of local/national authorities when applicable; and (iv) alive in the product at an efficacious dose throughout shelf life. We provide clarity and detail how each of these four criteria can be assessed. The wide adoption of these criteria is necessary to ensure the proper use of the word probiotic in scientific publications, on product labels, and in communications with regulators and the general public.

The effect of the CONSORT statement on the amount of "unclear" Risk of Bias reporting in Cochrane Systematic Reviews

Background

The Consolidated Standards of Reporting Trials (CONSORT) statement aims to improve clarity and consistency of transparency of reporting in Randomized Controlled Trials (RCTs). The Cochrane Risk of Bias (RoB) tool for RCTs helps authors to judge the RoB. as ‘‘low”, “high” or “unclear”.

Objective

In this study we aimed to assess whether the implementation and updates of the CONSORT statement influenced the trend of “unclear” RoB scores of RCTs included in Cochrane systematic reviews.

Methods

All Cochrane reviews published in December to October 2016 were retrieved. The publication year of RCTS included in the reviews were sorted into time frames (≤1995, 1996–2000, 2001–2009 and ≥2010) based on the release- and updates of the CONSORT statement (1996, 2001 and 2010). The association between “unclear” RoB versus “low or high” RoB and the year of publication in different time frames were calculated using a binary logistic regression.

Results

Data was extracted from 64 Cochrane reviews, with 989 RCTS (6471 items). The logistic regression showed that the odds of RCTs published ≥2010, compared to ≤1995 were more likely not to report an “unclear” RoB for the total data (Odds Ratio (OR) 0.69 (95% Confidence interval: 0.59–0.80)), random sequence generation (OR 0.32 (0.22–0.47), allocation concealment (0.64 (0.43–0.95)) and incomplete outcome data (OR 0.60 (0.39–0.91)).

Conclusion

A slight decrease of “unclear” RoB reporting over time was found. To improve quality of reporting authors are encouraged to adhere to reporting guidelines.

Quality of reporting for randomised clinical trials published in Latin American and Spanish journals: A protocol for a systematic survey of three clinical specialities

INTRODUCTION

Quality of reporting refers to how published articles communicate how the research was done and what was found. Gaps and imprecisions of reporting hamper the assessment of the methodological quality and internal and external validity. The CONsolidated Standards of Reporting Trials (CONSORT) are a set of evidence-based recommendations of the minimum elements to be included in the reporting of randomised controlled trials (RCTs) to ensure a complete and transparent account of what was done, how it was done and what was found. Few studies have been conducted on the impact of CONSORT on RCTs published in Latin American and Spanish journals. We aim to assess the reporting quality of RCTs of three clinical specialities published in Spanish and Latin American journals, as well as to assess changes over time and associations of quality with journal and country indicators.

METHODS AND ANALYSIS

We will conduct a systematic survey of all RCTs published in Spanish-language journals in three clinical fields (dentistry, neurology and geriatrics) from 1990 to 2018. We will include RCTs from previous work that has identified all RCTs on these medical fields published in Spain and Latin America. We will update this work via handsearching of relevant journals. Assessment of quality of reporting will be conducted independently and in duplicate using the CONSORT 2010 Statement. We will also extract journal and country indicators. We will conduct descriptive statistics and secondary analyses considering the year, country, and journal of publication, among others.

ETHICS AND DISSEMINATION

The Universidad de Santiago de Chile's ethics committee approved the protocol. We will disseminate the results of this work in peer-reviewed scientific journals and conference proceedings. We expect to raise awareness among researchers, journal editors and funders on the importance of training in reporting guidelines and using them from the inception of RCT protocols.

The adaptive designs CONSORT extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits. In order to encourage its wide dissemination this article is freely accessible on the BMJ and Trials journal websites."To maximise the benefit to society, you need to not just do research but do it well" Douglas G Altman.

The Adaptive designs CONSORT Extension (ACE) statement: a checklist with explanation and elaboration guideline for reporting randomised trials that use an adaptive design

Adaptive designs (ADs) allow pre-planned changes to an ongoing trial without compromising the validity of conclusions and it is essential to distinguish pre-planned from unplanned changes that may also occur. The reporting of ADs in randomised trials is inconsistent and needs improving. Incompletely reported AD randomised trials are difficult to reproduce and are hard to interpret and synthesise. This consequently hampers their ability to inform practice as well as future research and contributes to research waste. Better transparency and adequate reporting will enable the potential benefits of ADs to be realised.This extension to the Consolidated Standards Of Reporting Trials (CONSORT) 2010 statement was developed to enhance the reporting of randomised AD clinical trials. We developed an Adaptive designs CONSORT Extension (ACE) guideline through a two-stage Delphi process with input from multidisciplinary key stakeholders in clinical trials research in the public and private sectors from 21 countries, followed by a consensus meeting. Members of the CONSORT Group were involved during the development process.The paper presents the ACE checklists for AD randomised trial reports and abstracts, as well as an explanation with examples to aid the application of the guideline. The ACE checklist comprises seven new items, nine modified items, six unchanged items for which additional explanatory text clarifies further considerations for ADs, and 20 unchanged items not requiring further explanatory text. The ACE abstract checklist has one new item, one modified item, one unchanged item with additional explanatory text for ADs, and 15 unchanged items not requiring further explanatory text.The intention is to enhance transparency and improve reporting of AD randomised trials to improve the interpretability of their results and reproducibility of their methods, results and inference. We also hope indirectly to facilitate the much-needed knowledge transfer of innovative trial designs to maximise their potential benefits.

Quality, scope and reporting standards of randomised controlled trials in Irish Health Research: an observational study

Background

Despite efforts to improve the accuracy and transparency of the design, conduct, and reporting of randomised controlled trials (RCTs), deficiencies remain. Such deficiencies contribute to significant, avoidable waste of health research investment and impede reproducibility. This study aimed to synthesise and critically analyse changes over time in the conduct and reporting of internationally published evidence on patient and/or population health-oriented RCTs conducted in one country.

Methods

This observational study drew on systematic review methods. We searched six databases for published RCTs (database inception to December 2018) where ≥ 80% of participants were recruited in the Republic of Ireland. RCTs of interventions targeted at patients, providers and/or policy makers intended to improve health, healthcare or health research were included. For each study, screening, data extraction and methodological quality appraisal were conducted by one member of the author team.

Results

From 17,560 titles and abstracts, 752 unique RCTs were published in 745 papers between 1968 and 2018, with a steady year-on-year increase since 1968. The number of participants was in the range of 2–8628. The majority were parallel design (86%) and classified as treatment evaluation. Of the 418 RCTs published since the introduction of mandatory clinical trial registration by the International Committee of Medical Journal Editors in 2005, 32% (n = 134) provided a trial registration number. This increased to 47% when taking studies published between 2013 and 2018 (n = 232). Since the 1996 publication of the CONSORT statement, 16% of included RCTs made specific reference to a standardised reporting guideline and this increased to 31% for more recent studies published between 2013 and 2018. Overall, 7% (n = 53) of studies referred to a published study protocol, increasing to 20% for studies published between 2013 and 2018.

Conclusion

Evidence from this single-country study of RCTs published in the international literature suggests that both the number overall, the number registered and the number referencing reporting guidelines have increased steadily over time. Despite widespread endorsement of reporting standards, reporting of RCTs remains suboptimal in domains such as compliance with the CONSORT statement and prospective trial registration. Researchers, funders and journal editors, nationally and internationally, should continue to focus on improving reporting and examining avoidable waste of health research investment.