A cross-sectional study of the reporting quality of pilot or feasibility trials in high-impact anesthesia journals

Completeness of reporting of quality improvement studies in neonatology is inadequate: a systematic literature survey

INTRODUCTION

Quality improvement (QI) is a growing field of inquiry in healthcare, but the reporting quality of QI studies in neonatology remains unclear. We conducted a systematic survey of the literature to assess the reporting quality of QI studies and factors associated with reporting quality.

METHODS

We searched Medline for publications of QI studies from 2016 to 16 April 2020. Pairs of reviewers independently screened citations and assessed reporting quality using a 31-item modified Standards for Quality Improvement Reporting Excellence, 2nd edition (SQUIRE 2.0) checklist. We reported the number (percentage) of studies that reported each item and their corresponding 95% CIs. We used Poisson regression to explore factors associated with reporting quality, namely, journal endorsement of SQUIRE 2.0, declaration of funding sources, year of publication and number of authors. The results were reported as incidence rate ratio (IRR) and 95% CI.

RESULTS

Of 1921 citations, 336 were eligible; among them, we randomly selected 100 articles to assess reporting quality. The mean (standard deviation) number of SQUIRE 2.0 items adhered to was 22.0 (4.5). Percentage of articles reporting each item varied from 26% to 100%. Journal endorsement of SQUIRE 2.0 (IRR=1.11, 95% CI 1.02 to 1.21, p=0.015), declaration of funding sources and increasing number of authors were significantly associated with better reporting.

CONCLUSIONS

Reporting quality of QI studies in neonatology is inadequate. Endorsing the SQUIRE 2.0 guideline is a step that journals can implement to enhance the completeness of reporting.

A systematic review of reporting quality for anaesthetic interventions in randomised controlled trials

Interventions from randomised controlled trials can only be replicated if they are reported in sufficient detail. The results of trials can only be confidently interpreted if the delivery of the intervention was systematic and the protocol adhered to. We systematically reviewed trials of anaesthetic interventions published in 12 journals from January 2016 to September 2019. We assessed the detail with which interventions were reported, using the Consolidated Standards of Reporting Trials statement for non-pharmacological treatments. We analysed 162 interventions reported by 78 trials in 18,675 participants. Detail sufficiently precise to replicate the intervention was reported for 111 (69%) interventions. Intervention standardisation was reported for 135 (83%) out of the 162 interventions, and protocol adherence was reported for 20 (12%) interventions. Sixty (77%) out of the 78 trials reported the administrative context in which interventions were delivered and 36 (46%) trials detailed the expertise of the practitioners. We conclude that bespoke reporting tools should be developed for anaesthetic interventions and interventions in other areas such as critical care.

A tutorial on methodological studies: the what, when, how and why

BACKGROUND

Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

MAIN BODY

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

CONCLUSION

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Reporting quality and spin in abstracts of randomized clinical trials of periodontal therapy and cardiovascular disease outcomes

OBJECTIVE

Poor reporting in randomized clinical trial (RCT) abstracts reduces quality and misinforms readers. Spin, a biased presentation of findings, could frequently mislead clinicians to accept a clinical intervention despite non-significant primary outcome. Therefore, good reporting practices and absence of spin enhances research quality. We aim to assess the reporting quality and spin in abstracts of RCTs evaluating the effect of periodontal therapy on cardiovascular (CVD) outcomes.

METHODS

PubMed, Scopus, the Cochrane Central Register of Controlled Trials (CENTRAL), and 17 trial registration platforms were searched. Cohort, non-randomized, non-English studies, and pediatric studies were excluded. RCT abstracts were reviewed by 2 authors using the CONSORT for abstracts and spin checklists for data extraction. Cohen's Kappa statistic was used to assess inter-rater agreement. Data on the selected RCT publication metrics were collected. Descriptive analysis was performed with non-parametric methods. Correlation analysis between quality, spin and bibliometric parameters was conducted.

RESULTS

24 RCTs were selected for CONSORT analysis and 14 fulfilled the criteria for spin analysis. Several important RCT elements per CONSORT were neglected in the abstract including description of the study population (100%), explicitly stated primary outcome (87%), methods of randomization and blinding (100%), trial registration (87%). No RCT examined true outcomes (CVD events). A significant fraction of the abstracts appeared with at least one form of spin in the results and conclusions (86%) and claimed some treatment benefit in spite of non-significant primary outcome (64%). High-quality reporting had a significant positive correlation with reporting of trial registration (p = 0.04) and funding (p = 0.009). Spinning showed marginal negative correlation with reporting quality (p = 0.059).

CONCLUSION

Poor adherence to the CONSORT guidelines and high levels of data spin were found in abstracts of RCTs exploring the effects of periodontal therapy on CVD outcomes. Our findings indicate that journal editors and reviewers should consider strict adherence to proper reporting guidelines to improve reporting quality and reduce waste.

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

BACKGROUND

A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs.

OBJECTIVES

This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews.

METHODS

Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval).

RESULTS

Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely.

CONCLUSIONS

Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.

Protocol for a systematic review of reporting standards of anaesthetic interventions in randomised controlled trials

INTRODUCTION

There is significant variation in how anaesthesia is defined and reported in clinical research. This lack of standardisation complicates the interpretation of published evidence and planning of future clinical trials. This systematic review will assess the reporting of anaesthesia as an intervention in randomised controlled trials (RCT) against the Consolidated Standards of Reporting Trials for Non-Pharmacological Treatments (CONSORT-NPT) framework.

METHODS AND ANALYSIS

Online archives of the top six journals ranked by impact factor for anaesthesia and the top three general medicine and general surgery journals will be systematically hand searched over a 42-month time period to identify RCTs describing the use of anaesthetic interventions for any invasive procedure. All modes of anaesthesia and anaesthesia techniques will be included. All study data, including the type of anaesthetic intervention described, will be extracted in keeping with the CONSORT-NPT checklist. Descriptive statistics will be used to summarise general study details including types/modes of anaesthetic interventions, and reporting standards of the trials.

ETHICS AND DISSEMINATION

No ethical approval is required. The results will be used to inform a funding application to formally standardise general, local, regional anaesthesia and sedation for use in clinical research. The systematic review will be disseminated via peer-reviewed manuscript and conferences.

PROSPERO REGISTRATION NUMBER

CRD42019141670.

The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study

Introduction

Pilot and feasibility trials are conducted to determine feasibility or to collect information that would inform the design of a larger definitive trial. Clear progression criteria are required to determine if a definitive or main trial is feasible and how it should be designed. We sought to determine how often progression criteria are reported and the associated factors.

Methods

We conducted a methodological review of protocols for pilot randomised trials published in three journals that publish research protocols (BMJ Open, Trials, Pilot and Feasibility Studies), using a PubMed search (2013–2017). We extracted bibliometric information including the country in which the study was conducted, source of funding, type of intervention, use of a primary feasibility outcome, sample size reporting, and justification. We used generalised linear models to determine the factors associated with reporting progression criteria.

Results

Our search retrieved 276 articles, of which 49 were not eligible. We included 227 articles. Overall, 45/227 (19.8%; 95% confidence interval [CI] 14.8–25.6) reported progression criteria. Protocols published in more recent years were significantly associated with higher odds of reporting progression criteria (adjusted odds ratio [aOR] 1.40; 95% CI 1.03–1.92; p = 0.034). Pilot trials from Europe (aOR 0.19; 95% CI 0.08–0.48; p < 0.001) and the rest of the world (aOR 0.05; 95% CI 0.01–0.18; p < 0.003) compared to North America were significantly associated with lower odds of reporting progression criteria. Journal, source of funding, sample size, intervention type, and having a primary outcome related to feasibility were not significantly associated with reporting progression criteria.

Conclusion

Progression criteria are not often explicitly stated in protocols of pilot trials leaving room for varied interpretation of findings. The development of formal guidance for progression criteria in protocols of pilot trials is warranted.