Adherence of Randomized Trials Within Children's Surgical Specialties Published During 2000 to 2009 to Standard Reporting Guidelines

Addressing barriers to evidence-based medicine in pediatric surgery: an introduction to the Canadian Association of Paediatric Surgeons Evidence-Based Resource

Background

Pediatric surgical practice lags behind medicine in presence and use of evidence, primarily due to time constraints of using existing tools that are not specific to pediatric surgery, lack of sufficient patient data and unstructured pediatric surgery training methods.

Method

We developed, disseminated and tested the effectiveness of an evidence-based resource for pediatric surgeons and researchers that provides brief, informative summaries of quality-assessed systematic reviews and meta-analyses on conflicting pediatric surgery topics.

Results

Responses of 91 actively practicing surgeons who used the resource were analysed. The majority of participants found the resource useful (75%), improved their patient care (66.6%), and more than half (54.2%) found it useful in identifying research gaps. Almost all participants reported that the resource could be used as a teaching tool (93%).

Conclusion

Lack of awareness of the resource is the primary barrier to its routine use, leading to potential calls for more active dissemination worldwide. Users of the Canadian Association of Paediatric Surgeons Evidence-Based Resource find that the summaries are useful, identify research gaps, help mitigate multiple barriers to evidence-based medicine, and may improve patient care.

Adherence to the CONSORT statement and extension for nonpharmacological treatments in randomized controlled trials of bariatric surgery: A systematic survey

Reporting is critical for establishing the value of randomized controlled trials (RCTs). This study evaluated the adherence of bariatric surgery RCT reporting to the CONsolidated Standards Of Reporting Trials (CONSORT) statement 2010 and its 2017 extension for non-pharmacologic treatments (NPT extension). We identified all RCTs comparing bariatric surgery with conservational therapy or alternative bariatric surgery up to June 30, 2020. Reporting quality was assessed using criteria developed from the CONSORT statement and the NPT extension and scored as a percentage. The factors associated with reporting quality were explored by univariate and multivariate analysis. In total, 102 RCTs of bariatric surgery were included. The median scores according to the CONSORT statement and NPT extension were 63.3 and 26.8 of a maximum possible 100, respectively. Two-thirds of NPT extension items were reported in less than 25% of the RCTs. The median score improved over time for the CONSORT statement but not the NPT extension. A higher CONSORT score was associated with publication in core clinical journals, protocol registration, and funding. No factors associated with the NPT extension score were identified. Substantial efforts are warranted from authors, journals, registration platforms, and funders to overcome the flaws in the reporting of bariatric surgery RCTs.

Assessing the methodological and reporting quality of clinical systematic reviews and meta-analyses in paediatric urology: can practices on contemporary highest levels of evidence be built?

INTRODUCTION

Systematic reviews and meta-analyses provide a comprehensive summary of research studies and are used to assess clinical evidence, form policy and construct guidelines. This is pertinent to childhood surgery with issues of consent and condition prevalence. The aims of this study were to evaluate the methodological and reporting quality of these reviews and to identify how these reviews might guide clinical practice amongst those conditions most commonly encountered and managed by practicing paediatric urologists.

METHODS

A systematic search of the English literature was performed to identify systematic reviews and meta-analyses focusing on clinical paediatric urology (1/1/1992-1/12/2018) to include common paediatric urological conditions managed by paediatric urology residents/fellows. To these reviews, Assessing the Methodological Quality of Systematic Reviews (AMSTAR)-2 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores were applied. Univariate linear regression and descriptive statistical methods were performed.

RESULTS

From an initial literature review of 1723 articles, 227 were included in the analysis. Inter-reviewer agreement was high amongst 3 independent reviewers (κ = 0.92). Eighty-four percent of systematic reviews and meta-analyses were published since 2009 following publication of the PRISMA guidelines. The overall impact factor was 3.38 (0.83-17.58), with adherence to AMSTAR-2 criteria 48.46% and PRISMA criteria 70.1%. From a methodological perspective, 15% of reviews were of moderate quality, 65% were of low quality and 20% reviews were of critically low quality, with none found to have good quality reporting.

CONCLUSIONS

Despite the continued increase of systematic reviews and meta-analyses in paediatric urology from which many guidelines are based, a significant number of reviews contain poor methodology and, to a lesser extent, poor reporting quality. Journals should consider having specific 'a priori' criteria based on checklists before publication of manuscripts to ensure the highest possible reporting quality.

Enrollment and reporting practices in pediatric general surgical randomized clinical trials: A systematic review and observational analysis

BACKGROUND

Pediatric surgical randomized clinical trials (RCTs) are labor-intensive and costly. This systematic review investigated patient accrual and estimates of study duration in RCTs by interrogating enrollment and registration practices.

METHODS

We performed a peer-review search of multiple databases from 2000 to 2016 evaluating RCTs salient to the field with inclusion mandated that a self-identified pediatric surgeon be listed as an author. Trial registries were also searched. RCTs were appraised, and predictors of success were evaluated using multivariate logistic regression, with success defined as achievement of recruitment objectives.

RESULTS

After screening, 137 RCTs were analyzed. Mean Jadad score was 1.80 (median=2). CONSORT scores ranged between 17% and 97% (median=58%). Sixty-seven studies described sample-size determination, 49 reported projected enrollment, and 26 were successful. Among 26 registered RCTs, 15 disclosed their expected completion date, which was achieved by 8. On average, protocols underwent 3.42 iterations. 9% of trials were terminated before completion, most commonly owing to poor recruitment. Trial registration and urgent cases significantly predicted success on multivariable analysis (p<0.05).

CONCLUSION

Overall quality of reporting in pediatric surgical trials is poor. Sample-size calculation and patient accrual are frequently poorly performed or underestimated, resulting in trial overrun and/or premature termination. These data may help inform subsequent study design and facilitate successful completion.

LEVEL OF EVIDENCE

Level III-Systematic Review and Observational (Case-Control) Analysis.

Primary outcomes reporting in trials of paediatric type 1 diabetes mellitus: a systematic review

OBJECTIVE

Our objective was to systematically review randomised clinical trials (RCTs) of paediatric type 1 diabetes mellitus (T1DM) to assess reporting of (1) primary outcome, (2) outcome measurement properties and (3) presence or absence of adverse events.

METHODS

Electronic searches in MEDLINE, EMBASE, CINAHL, Cochrane SR and the Cochrane Central Register of Controlled Trials (CENTRAL) databases were undertaken. The search period was between 2001 and 2017. English-language RCTs on children younger than 21 years with T1DM were selected. We excluded studies of diagnostic or screening tools, multiple phase studies, protocols, and follow-up or secondary analysis of data.

RESULTS

Of 11 816 unique references, 231 T1DM RCTs were included. Of total 231 included studies, 117 (50.6%) trials failed to report what their primary outcome was. Of 114 (49.4%) studies that reported primary outcome, 88 (77.2%) reported one and 26 (22.8%) more than one primary outcomes. Of 114 studies that clearly stated their primary outcome, 101 (88.6%) used biological/physiological measurements and 13 (11.4%) used instruments (eg, questionnaires, scales, etc) to measure their primary outcome; of these, 12 (92.3%) provided measurement properties or related citation. Of the 231 included studies, 105 (45.5%) reported that adverse events occurred, 39 (16.9%) reported that no adverse events were identified and 87 (37.7%) did not report on the presence or absence of adverse events.

CONCLUSION

Despite tremendous efforts to improve reporting of clinical trials, clear reporting of primary outcomes of RCTs for paediatric T1DM is still lacking. Adverse events due to DM interventions were often not reported in the included trials. Transparent reporting of primary outcome, validity of measurement tools and adverse events need to be improved in paediatric T1DM trials.

A systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery

OBJECTIVE

Our objective was to evaluate quality of conduct and reporting of published systematic reviews and meta-analyses in paediatric surgery. We also aimed to identify characteristics predictive of review quality.

BACKGROUND

Systematic reviews summarise evidence by combining sources, but are potentially prone to bias. To counter this, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was published to aid in reporting. Similarly, the Assessing the Methodological Quality of Systematic Reviews (AMSTAR) measurement tool was designed to appraise methodology. The paediatric surgical literature has seen an increasing number of reviews over the past decade, but quality has not been evaluated.

METHODS

Adhering to PRISMA guidelines, we performed a systematic review with a priori design to identify systematic reviews and meta-analyses of interventions in paediatric surgery. From 01/2010 to 06/2016, we searched: MEDLINE, EMBASE, Cochrane, Centre for Reviews and Dissemination, Web of Science, Google Scholar, reference lists and journals. Two reviewers independently selected studies and extracted data. We assessed conduct and reporting using AMSTAR and PRISMA. Scores were calculated as the sum of reported items. We also extracted author, journal and article characteristics, and used them in exploratory analysis to determine which variables predict quality.

RESULTS

112 articles fulfilled eligibility criteria (53 systematic reviews; 59 meta-analyses). Overall, 68% AMSTAR and 56.8% PRISMA items were reported adequately. Poorest scores were identified with regards a priori design, inclusion of structured summaries, including the grey literature, citing excluded articles and evaluating bias. 13 reviews were pre-registered and 6 in PRISMA-endorsing journals. The following predicted quality in univariate analysis:, word count, Cochrane review, journal h-index, impact factor, journal endorses PRISMA, PRISMA adherence suggested in author guidance, article mentions PRISMA, review includes comparison of interventions and review registration. The latter three variables were significant in multivariate regression.

CONCLUSIONS

There are gaps in the conduct and reporting of systematic reviews in paediatric surgery. More endorsement by journals of the PRISMA guideline may improve review quality, and the dissemination of reliable evidence to paediatric clinicians.

Reporting of Randomized Controlled Trials in Cleft Lip and Palate: A 10-Year Review

Objectives

Reviews of the quality of reporting of randomized controlled trials (RCTs) have recently been conducted in different surgical specialties. In this review of RCTs relating to cleft lip, cleft palate, and cleft lip and palate (CL/P), we investigate the quality of reporting against the Consolidated Standards of Reporting Trials (CONSORT) checklist.

Design

A systematic review of CL/P RCTs published from 2004 to 2013, with the included articles scored against the CONSORT checklist.

Patients, Participants

The literature search identified 174 articles. Studies were selected for participants with CL/P who were involved in an RCT with prospective data collection and reported in a full journal article. A total of 6352 participants were included from 65 CUP RCTs during the study period.

Main Outcome Measures

The methodological quality of RCTs was assessed using the CONSORT checklist and Jadad scale.

Results

The mean CONSORT score was 15.8, and the mean Jadad score was 3.3. There was a significant positive correlation between the CONSORT and Jadad score ( P < .0001, ρ = .47). The only significant correlation showed that with an increasing number of authors, both the CONSORT and the Jadad score increased.

Conclusion

This analysis has shown that that there are deficiencies in the transparent reporting of factors such as randomization implementation, blinding, and participant flow. Interventions, outcomes, and the interpretation of results are well presented. We would recommend that RCTs are conceived and undertaken using the CONSORT checklist and reported in a clear and reproducible manner.

Recommendations and evidence for reporting items in pediatric clinical trial protocols and reports: two systematic reviews

BACKGROUND

Complete and transparent reporting of clinical trial protocols and reports ensures that these documents are useful to all stakeholders, that bias is minimized, and that the research is not wasted. However, current studies repeatedly conclude that pediatric trial protocols and reports are not appropriately reported. Guidelines like SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) may improve reporting, but do not offer guidance on issues unique to pediatric trials. This paper reports two systematic reviews conducted to build the evidence base for the development of pediatric reporting guideline extensions: 1) SPIRIT-Children (SPIRIT-C) for pediatric trial protocols, and 2) CONSORT-Children (CONSORT-C) for pediatric trial reports.

METHOD

MEDLINE, the Cochrane Methodology Register, and reference lists of included studies were searched. Publications of any type were eligible if they included explicit recommendations or empirical evidence for the reporting of potential items in a pediatric protocol (SPIRIT-C systematic review) or trial report (CONSORT-C systematic review). Study characteristics, recommendations and evidence for pediatric extension items were extracted. Recurrent themes in the recommendations and evidence were identified and synthesized. All steps were conducted by two reviewers.

RESULTS

For the SPIRIT-C and CONSORT-C systematic reviews 366 and 429 publications were included, respectively. Recommendations were identified for 48 of 50 original reporting items and sub-items from SPIRIT, 15 of 20 potential SPIRIT-C reporting items, all 37 original CONSORT items and sub-items, and 16 of 22 potential CONSORT-C reporting items. The following overarching themes of evidence to support or refute the utility of reporting items were identified: transparency; reproducibility; interpretability; usefulness; internal validity; external validity; reporting bias; publication bias; accountability; scientific soundness; and research ethics.

CONCLUSION

These systematic reviews are the first to systematically gather evidence and recommendations for the reporting of specific items in pediatric protocols and trials. They provide useful and translatable evidence on which to build pediatric extensions to the SPIRIT and CONSORT reporting guidelines. The resulting SPIRIT-C and CONSORT-C will provide guidance to the authors of pediatric protocols and reports, respectively, helping to alleviate concerns of inappropriate and inconsistent reporting, and reduce research waste.

The explicit mentioning of reporting guidelines in urogynecology journals in 2013: A bibliometric study

AIMS

Poor reporting of research may limit critical appraisal and reproducibility, whereas adherence to reporting guidelines (RG) can guarantee completeness and transparency. We aimed to determine the explicit citing of RGs (CONSORT, PRISMA, STROBE) in urogynecology articles in 2013, the requirements of relevant journals and a potential difference between urogynecology and general gynecology journals.

METHODS

All urogynecologic articles published between January and December 2013 in the journals NAU, IUJ, FPMRS, GREEN, AJOG, and BJOG were included. Issues were searched for systematic reviews, RCTs, cohort studies, case-control studies and cross-sectional studies. Each electronic article was searched for the term PRISMA, CONSORT, or STROBE according to the study design. Instructions to Authors of the six journals were screened for requirement of using RGs.

RESULTS

We included 296 articles (243 observational studies, 40 RCTs, and 13 systematic reviews). The use of PRISMA guidelines was explicitly declared in 54% of systematic reviews, CONSORT guidelines were referenced in 25% of RCTs and STROBE in 1.2% of observational studies. The use of CONSORT is required by all journals except FPMRS. PRISMA and STROBE are only compulsory in the journals GREEN, AJOG, and BJOG. The overall rate of explicit mentioning of RGs comparing urogynecology and general gynecology journals was 6.7% versus 7.1%, respectively.

CONCLUSIONS

The explicit mentioning of RGs was on a relatively low level. A slightly higher adherence was recognized among general gynecology journals compared to urogynecology journals. Stronger efforts should be taken to further promote the use of RGs in urogynecology.

Impact of a randomized clinical trial on children with perforated appendicitis

BACKGROUND

We previously conducted a randomized, clinical trial comparing early appendectomy with interval appendectomy for perforated appendicitis. The purpose of the present study was to evaluate the effect this clinical trial had on subsequent practice patterns and outcomes for patients with perforated appendicitis at the free-standing children's hospital conducting the trial.

METHODS

A retrospective study was conducted comparing children with perforated appendicitis treated before the trial (2005-2006) and after the trial (2009-2011). Early appendectomy was performed within 24 hours of diagnosis; interval appendectomy occurred 4-6 weeks after initial treatment with antibiotics. Patient characteristics, treatment variables, and outcomes were collected and compared.

RESULTS

The pretrial group consisted of 92 patients-62 (67%) underwent early appendectomy, and 30 (33%) patients had interval appendectomy. The posttrial group was composed of 103 patients, with 87 (84%) undergoing early appendectomy and 16 (16%) interval appendectomy (P = .005). The groups were similar in patient and admission characteristics, although the posttrial group had a lower percentage of self-pay patients and fewer computed tomography scans; health care use was similar between groups. Overall, the posttrial group had fewer adverse events (18% vs 34%, P = .02), specifically fewer wound infections (2% vs 14%, P = .001) and fewer unplanned readmissions (7% vs 16%, P = .04) than the pretrial group. In the posttrial group, those patients selected for interval appendectomy were more likely to complete the planned course of therapy than in the pretrial group.

CONCLUSION

A clinical trial conducted at our institution to evaluate currently available treatment options for perforated appendicitis did change practice patterns at our hospital. After the trial, there was an increase in the use of early appendectomy, a decrease in the number of computed tomography scans performed per patient, and a reduction in the overall adverse event rate.