Pilot randomized controlled trials in the orthopaedic surgery literature: a systematic review

REporting quality of PilOt randomised controlled trials in surgery (REPORTS): a methodological survey protocol

INTRODUCTION

The aim of this methodological review is to evaluate the completeness of reporting of surgical pilot and feasibility randomised trials as per the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials. Moreover, we aim to assess for the presence of spin reporting and inconsistency between abstract and main text reporting in surgical pilot and feasibility randomised trials.

METHODS AND ANALYSIS

A comprehensive, electronic search strategy will be used to identify studies indexed in Medline, Embase and Cochrane Central Register of Controlled Trials (CENTRAL) databases. Studies will be included if they are pilot or feasibility randomised trials of surgical interventions. The primary outcome will be overall CONSORT statement extension to randomised pilot and feasibility trials checklist completeness. This will be defined as trials reporting each of the 40 items in the CONSORT statement extension to randomised pilot and feasibility trials checklist. Secondary outcomes will include the reporting of individual studies as per the CONSORT extension to randomised pilot and feasibility trials, the use of spin reporting strategies, trial factors associated with reporting quality and spin strategy use, and consistency between abstract and main text reporting. Poisson and logistic regressions will be performed to explore the association between trial factors and completeness of reporting as measured by the number of reported CONSORT items.

ETHICS AND DISSEMINATION

This is a methodological survey that has been registered a priori on the International Prospective Register for Systematic Reviews (PROSPERO) (CRD42023475512). Local ethics approval is not required. We plan to disseminate study results through peer-reviewed publication and conference presentations.

How do pilot and feasibility studies inform randomised placebo-controlled trials in surgery? A systematic review

INTRODUCTION

Randomised controlled trials (RCTs) with a placebo comparator are considered the gold standard study design when evaluating healthcare interventions. These are challenging to design and deliver in surgery. Guidance recommends pilot and feasibility work to optimise main trial design and conduct; however, the extent to which this occurs in surgery is unknown.

METHOD

A systematic review identified randomised placebo-controlled surgical trials. Articles published from database inception to 31 December 2020 were retrieved from Ovid-MEDLINE, Ovid-EMBASE and CENTRAL electronic databases, hand-searching and expert knowledge. Pilot/feasibility work conducted prior to the RCTs was then identified from examining citations and reference lists. Where studies explicitly stated their intent to inform the design and/or conduct of the future main placebo-controlled surgical trial, they were included. Publication type, clinical area, treatment intervention, number of centres, sample size, comparators, aims and text about the invasive placebo intervention were extracted.

RESULTS

From 131 placebo surgical RCTs included in the systematic review, 47 potentially eligible pilot/feasibility studies were identified. Of these, four were included as true pilot/feasibility work. Three were original articles, one a conference abstract; three were conducted in orthopaedic surgery and one in oral and maxillofacial surgery. All four included pilot RCTs, with an invasive surgical placebo intervention, randomising 9-49 participants in 1 or 2 centres. They explored the acceptability of recruitment and the invasive placebo intervention to patients and trial personnel, and whether blinding was possible. One study examined the characteristics of the proposed invasive placebo intervention using in-depth interviews.

CONCLUSION

Published studies reporting feasibility/pilot work undertaken to inform main placebo surgical trials are scarce. In view of the difficulties of undertaking placebo surgical trials, it is recommended that pilot/feasibility studies are conducted, and more are reported to share key findings and optimise the design of main RCTs.

PROSPERO REGISTRATION NUMBER

CRD42021287371.

From Protocol to Definitive Study-The State of Randomized Controlled Trial Evidence in Sports Medicine Research: A Systematic Review and Survey Study

OBJECTIVE

To evaluate the progression, quality, and challenges associated with conducting and publishing randomized controlled trials (RCTs) in sports medicine.

DESIGN

Systematic review and survey.

SETTING

MEDLINE and Embase were searched for all publications before September 17, 2021. A targeted search of clinicaltrials.gov , BMC Musculoskeletal Disorders, PubMed, and Google Scholar were also conducted. The survey was administered to authors using REDCap.

PARTICIPANTS

Where the systematic search revealed no corresponding published definitive trial, authors of the published pilots were surveyed.

INTERVENTIONS

Survey assessing limitations to definitive trials.

MAIN OUTCOME MEASURES

Protocol/method articles, pilot articles, and relevant clinical trial registry records with corresponding definitive trials were pooled.

RESULTS

Our literature search yielded 27 006 studies; of which, we included 208 studies (60 (28.8%) pilot RCTs, 84 (40.4%) protocol/method articles, and 64 (30.8%) trial registry records). From these, 44 corresponding definitive RCTs were identified. Pilot study and definitive RCT methodological quality increased on average most significantly during the duration of this review (30.6% and 8.2%). Of the 176 authors surveyed, 59 (33.5%) responded; 24.6% (14/57) stated that they completed an unpublished definitive trial, while 52.6% (30/57) reported having one underway.

CONCLUSIONS

The quality and number of RCT publications within the field of sports medicine has been increasing since 1999. The number of sports medicine-related protocol and pilot articles preceding a definitive trial publication showed a sharp increase over the past 10 years, although only 5 pilot studies have progressed to a definitive RCT.

Randomized controlled trials in neurosurgery

Randomized controlled trials (RCTs) have become the standard method of evaluating new interventions (whether medical or surgical), and the best evidence used to inform the development of new practice guidelines. When we review the history of medical versus surgical trials, surgical RCTs usually face more challenges and difficulties when conducted. These challenges can be in blinding, recruiting, funding, and even in certain ethical issues. Moreover, to add to the complexity, the field of neurosurgery has its own unique challenges when it comes to conducting an RCT. This paper aims to provide a comprehensive review of the history of neurosurgical RCTs, focusing on some of the most critical challenges and obstacles that face investigators. The main domains this review will address are: (1) Trial design: equipoise, blinding, sham surgery, expertise-based trials, reporting of outcomes, and pilot trials, (2) trial implementation: funding, recruitment, and retention, and (3) trial analysis: intention-to-treat versus as-treated and learning curve effect.

Status, reporting completeness and methodological quality of pilot randomised controlled trials in acupuncture: protocol for a systematic review

INTRODUCTION

To date, there has been a lack of knowledge about the status, reporting completeness and methodological quality of pilot trials in the acupuncture field. Thus, this systematic review protocol aims to: (1) investigate publication trends and aspects of feasibility evaluated in acupuncture pilot trials; (2) identify the proportion of acupuncture pilot trials that lead to definitive trials and (3) assess the reporting completeness and methodological quality of pilot trials in acupuncture.

METHODS AND ANALYSIS

Studies of acupuncture pilot randomised controlled trials published from 2011 to 2021 will be retrieved in seven databases in January 2022, including PubMed, Web of Science, EMBASE, Cochrane Library, Chinese National Knowledge Infrastructure, Wanfang Database and Chinese Biomedical Literature Database. The methodological quality and reporting completeness of all included studies will be assessed using the risk of bias 2.0 tool (RoB 2) and the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials, respectively. For the primary analysis, publication trends, aspects of feasibility and the proportion of pilot trials that lead to definitive trials will be analysed. A quantitative analysis of the methodological quality and reporting completeness of the included trials will be implemented by calculating the percentage of items reported in each domain of RoB 2 and CONSORT. The secondary analysis will adopt a regression analysis to identify factors associated with the reporting completeness.

ETHICS AND DISSEMINATION

Ethical approval is not required for this study. This study is planned to be submitted to a peer-reviewed academic journal.

Quality of reporting for pilot randomized controlled trials in the pediatric urology literature-A systematic review

BACKGROUND

The conduct and reporting of pilot studies is important to assess the feasibility of future randomized controlled trials (RCT). The Consolidated Standards of Reporting Trials (CONSORT) statement extension to pilot/feasibility studies addresses the reporting quality of the pilot studies (Summary Table 1). The aims of this systematic review are (1) to assess the reporting quality of pilot studies in pediatric urology and (2) to explore the factors that are associated with the reporting quality of these studies.

METHODS

A comprehensive search was conducted through MEDLINE® and EMBASE® to identify pilot RCTs from 2005 to 2018. Two reviewers independently performed title and abstract screening and full text review, with discrepancies resolved by consensus. CONSORT extension reported items were summarized and overall proportion of reported items for each article was estimated. A linear regression model was conducted to determine factors associated with higher reporting quality. Publication year, biostatistician/epidemiologist support, sample size justification and journal impact factor were collected.

RESULTS

Of the 1463 titles duplicates were removed and 1347 were screened, 36 studies were included. Overall, 36 pilot studies reported about 8-9 of 17 items [51% (95% CI: 46 - 56%)]. The most reported items were contact details for the corresponding author (97%), title identification of study as randomised pilot or feasibility trial (95%), eligibility criteria and setting (81%), both interventions (78%), and specific objectives of the pilot trial (75%). Less fulfilled items were blinding (11%), registration of the trial (11%), randomization details (28%), detailing recruitment status in the pilot study (19%), trial design (31%), and source of funding for pilot trial (34%). Interpretation of the results of pilot trial and their implications for the future definitive trial was reported by 34% of the studies. Factors associated with higher reporting quality were the presence of biostatistician or epidemiologist (P = 0.004), and if the sample size for the pilot study was justified (P = 0.002).

DISCUSSION

Overall reporting quality of pilot studies in pediatric urology literature from 2005-2018 was suboptimal. The quality of pilot RCTs included in the present review were lower than that observed in the orthopedic literature, however, it appears to be consistent with the trends regarding OQS in chronic kidney disease and allopathic medicine. While we endeavoured to maintain utmost rigidity of this systematic review, there are inherent limitations. The CONSORT 2010 extension for pilot RCTs was published in 2016. Clinical trials can take several years, many pilot studies published pre-2016 would not have had the guidance of the extension during designing phases. Not all pilot RCTs are published, so this could potentially reduce the generalizability of the findings from this review. Only studies in English, published in full peer-reviewed journals were included, and this review only addressed the reporting quality of pilot studies in pediatric urology.

CONCLUSION

This review demonstrated that reporting quality of pilot studies in pediatric urology is currently suboptimal. Including biostatistician and/or epidemiologist, can ameliorate the quality of future pilot studies. Implementing CONSORT 2010 extension by journals as a prerequisite for submission of pilot or feasibility trials is recommended to improve the robustness and transparency of future pilot studies.