Randomized controlled trials in pediatric surgery: could we do better?

Tissue Engineering of the Urethra: From Bench to Bedside

Tissue engineering (TE) is a promising approach for repair/substitution of damaged tissues and organs. Urethral strictures are common and serious health conditions that impair quality of life and may lead to serious organ damage. The search for ideal materials for urethral repair has led to interest of scientists and surgeons in urethral TE. Over the last decades, a significant amount of preclinical studies and considerable progress have been observed. In contrast, urethral TE has made slow progress in clinical practice so far. To address this, we conducted a systematic review of the literature on clinical applications of TE constructs for urethral repair in the last three decades. In summary, the TE approach is promising and effective, but many issues remain that need to be addressed for broader adoption of TE in urethral repair. Better design of trials, better cooperation of research groups and centralization could lead to reduction of costs and slowly proceed to commercialization and routine use of TE products for urethral reconstruction.

Meta-analyses in paediatric surgery are often fragile: implications and consequences

PURPOSE

Meta-analyses occupy the highest level of evidence and thereby guide clinical decision-making. Recently, randomised-controlled trials were evaluated for the robustness of their findings by calculating the fragility index. The fragility index is the number of events that needs to be added to one treatment arm until the statistical significance collapses. We, therefore, aimed to evaluate the robustness of paediatric surgical meta-analyses.

METHODS

We searched MEDLINE for paediatric surgical meta-analyses in the last decade. All meta-analyses on a paediatric surgical condition were eligible for analysis if they based their conclusion on a statistically significant meta-analysis.

RESULTS

We screened 303 records and conducted a full-text evaluation of 60 manuscripts. Of them, 39 were included in our analysis that conducted 79 individual meta-analyses with significant results. Median fragility index was 5 (Q25-Q75% 2-11). Median fragility in relation to included patients was 0.77% (Q25-Q75% 0.29-1.87%).

CONCLUSION

Paediatric surgical meta-analyses are often fragile. In almost 60% of results, the statistical significance depends on less than 1% of the included population. However, as the fragility index is just a transformation of the P value, it basically conveys the same information in a different format. It therefore should be avoided.

Evidence-based Plastic Surgery: Assessing Progress over Two 5-year Periods from 2009 to 2019

Publications on evidence-based medicine have increased. Previous articles have examined evidence-based plastic surgery, but the latest was published in 2013. The aim of this study was to examine the trend in the number of high-evidence publications over two 5-year periods across 3 main plastic surgery journals. Further, this study aimed to quality-assess randomized controlled trials (RCTs) published in the latter period.

Methods

All articles were identified using PubMed Search Tools and Single Citation Matcher. Three journals were manually screened from May 15, 2009, to May 15, 2014, and from May 16, 2014, to May 16, 2019. The reporting of RCTs was assessed using a modified Consolidated Standards of Reporting Trials (CONSORT) checklist.

Results

Of a total of 17,334 publications, 6 were meta-analyses of RCTs, 120 were other meta-analyses, and 247 were initially identified as RCTs. Although a significant increase in the number of higher-evidence publications is observed, these represent 2.09% (n = 363) of the total. An estimated 86 RCTs were eligible for quality-assessment, with the most popular sub-specialty being breast surgery (n = 30). The most highly reported criteria were inclusion/exclusion criteria and blinding (both n = 67; 77.91%), and the least reported criterion was allocation concealment (n = 21; 24.42%).

Conclusions

This study observes a positive trend in high-evidence publications. The number of RCTs published has increased significantly over a breadth of sub-specialties. The reporting of several CONSORT criteria in RCTs remains poor. Observation to standard reporting guidelines is advocated to improve the quality of reporting.

Full publication of results initially presented in abstracts

BACKGROUND

Abstracts of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of results reported in these abstracts is based on the magnitude or direction of the results, publication bias may result. Publication bias creates problems for those conducting systematic reviews or relying on the published literature for evidence about health and social care.

OBJECTIVES

To systematically review reports of studies that have examined the proportion of meeting abstracts and other summaries that are subsequently published in full, the time between meeting presentation and full publication, and factors associated with full publication.

SEARCH METHODS

We searched MEDLINE, Embase, the Cochrane Library, Science Citation Index, reference lists, and author files. The most recent search was done in February 2016 for this substantial update to our earlier Cochrane Methodology Review (published in 2007).

SELECTION CRITERIA

We included reports of methodology research that examined the proportion of biomedical results initially presented as abstracts or in summary form that were subsequently published. Searches for full publications had to be at least two years after meeting presentation.

DATA COLLECTION AND ANALYSIS

Two review authors extracted data and assessed risk of bias. We calculated the proportion of abstracts published in full using a random-effects model. Dichotomous variables were analyzed using risk ratio (RR), with multivariable models taking into account various characteristics of the reports. We assessed time to publication using Kaplan-Meier survival analyses.

MAIN RESULTS

Combining data from 425 reports (307,028 abstracts) resulted in an overall full publication proportion of 37.3% (95% confidence interval (CI), 35.3% to 39.3%) with varying lengths of follow-up. This is significantly lower than that found in our 2007 review (44.5%. 95% CI, 43.9% to 45.1%). Using a survival analyses to estimate the proportion of abstracts that would be published in full by 10 years produced proportions of 46.4% for all studies; 68.7% for randomized and controlled trials and 44.9% for other studies. Three hundred and fifty-three reports were at high risk of bias on one or more items, but only 32 reports were considered at high risk of bias overall.Forty-five reports (15,783 abstracts) with 'positive' results (defined as any 'significant' result) showed an association with full publication (RR = 1.31; 95% CI 1.23 to 1.40), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; 95% CI 1.07 to 1.28) in 34 reports (8794 abstracts). Results emanating from randomized or controlled trials showed the same pattern for both definitions (RR = 1.21; 95% CI 1.10 to 1.32 (15 reports and 2616 abstracts) and RR = 1.17; 95% CI, 1.04 to 1.32 (13 reports and 2307 abstracts), respectively.Other factors associated with full publication include oral presentation (RR = 1.46; 95% CI 1.40 to 1.52; studied in 143 reports with 115,910 abstracts); acceptance for meeting presentation (RR = 1.65; 95% CI 1.48 to 1.85; 22 reports with 22,319 abstracts); randomized trial design (RR = 1.51; 95% CI 1.36 to 1.67; 47 reports with 28,928 abstracts); and basic research (RR = 0.78; 95% CI 0.74 to 0.82; 92 reports with 97,372 abstracts). Abstracts originating at an academic setting were associated with full publication (RR = 1.60; 95% CI 1.34 to 1.92; 34 reports with 16,913 abstracts), as were those considered to be of higher quality (RR = 1.46; 95% CI 1.23 to 1.73; 12 reports with 3364 abstracts), or having high impact (RR = 1.60; 95% CI 1.41 to 1.82; 11 reports with 6982 abstracts). Sensitivity analyses excluding reports that were abstracts themselves or classified as having a high risk of bias did not change these findings in any important way.In considering the reports of the methodology research that we included in this review, we found that reports published in English or from a native English-speaking country found significantly higher proportions of studies published in full, but that there was no association with year of report publication. The findings correspond to a proportion of abstracts published in full of 31.9% for all reports, 40.5% for reports in English, 42.9% for reports from native English-speaking countries, and 52.2% for both these covariates combined.

AUTHORS' CONCLUSIONS

More than half of results from abstracts, and almost a third of randomized trial results initially presented as abstracts fail to be published in full and this problem does not appear to be decreasing over time. Publication bias is present in that 'positive' results were more frequently published than 'not positive' results. Reports of methodology research written in English showed that a higher proportion of abstracts had been published in full, as did those from native English-speaking countries, suggesting that studies from non-native English-speaking countries may be underrepresented in the scientific literature. After the considerable work involved in adding in the more than 300 additional studies found by the February 2016 searches, we chose not to update the search again because additional searches are unlikely to change these overall conclusions in any important way.

Enrollment and reporting practices in pediatric general surgical randomized clinical trials: A systematic review and observational analysis

BACKGROUND

Pediatric surgical randomized clinical trials (RCTs) are labor-intensive and costly. This systematic review investigated patient accrual and estimates of study duration in RCTs by interrogating enrollment and registration practices.

METHODS

We performed a peer-review search of multiple databases from 2000 to 2016 evaluating RCTs salient to the field with inclusion mandated that a self-identified pediatric surgeon be listed as an author. Trial registries were also searched. RCTs were appraised, and predictors of success were evaluated using multivariate logistic regression, with success defined as achievement of recruitment objectives.

RESULTS

After screening, 137 RCTs were analyzed. Mean Jadad score was 1.80 (median=2). CONSORT scores ranged between 17% and 97% (median=58%). Sixty-seven studies described sample-size determination, 49 reported projected enrollment, and 26 were successful. Among 26 registered RCTs, 15 disclosed their expected completion date, which was achieved by 8. On average, protocols underwent 3.42 iterations. 9% of trials were terminated before completion, most commonly owing to poor recruitment. Trial registration and urgent cases significantly predicted success on multivariable analysis (p<0.05).

CONCLUSION

Overall quality of reporting in pediatric surgical trials is poor. Sample-size calculation and patient accrual are frequently poorly performed or underestimated, resulting in trial overrun and/or premature termination. These data may help inform subsequent study design and facilitate successful completion.

LEVEL OF EVIDENCE

Level III-Systematic Review and Observational (Case-Control) Analysis.

Study design in evidence-based surgery: What is the role of case-control studies?

Randomized controlled trials (RCTs) are the gold standard in terms of study design, however, in the surgical setting conducting RCTs can often be unethical or logistically impossible. Case-control studies should become the major study design used in surgical research when RCTs are unable to be conducted and definitely replacing case series which offer little insight into surgical outcomes and disease processes.

Partial versus complete fundoplication for the correction of pediatric GERD: a systematic review and meta-analysis

BACKGROUND

There is no consensus as to what extent of "wrap" is required in a fundoplication for correction of gastroesophageal reflux disease (GERD).

OBJECTIVE

To evaluate if a complete (360 degree) or partial fundoplication gives better control of GERD.

METHODS

A systematic search of MEDLINE and Scopus identified interventional and observational studies of fundoplication in children. Screening identified those comparing techniques. The primary outcome was recurrence of GERD following surgery. Dysphagia and complications were secondary outcomes of interest. Meta-analysis was performed when appropriate. Study quality was assessed using the Cochrane Risk of Bias Tool.

RESULTS

2289 abstracts were screened, yielding 2 randomized controlled trials (RCTs) and 12 retrospective cohort studies. The RCTs were pooled. There was no difference in surgical success between partial and complete fundoplication, OR 1.33 [0.67,2.66]. In the 12 cohort studies, 3 (25%) used an objective assessment of the surgery, one of which showed improved outcomes with complete fundoplication. Twenty-five different complications were reported; common were dysphagia and gas-bloat syndrome. Overall study quality was poor.

CONCLUSIONS

The comparison of partial fundoplication with complete fundoplication warrants further study. The evidence does not demonstrate superiority of one technique. The lack of high quality RCTs and the methodological heterogeneity of observational studies limits a powerful meta-analysis.

Surgical treatment of neurogenic stress urinary incontinence: A systematic review of quality assessment and surgical outcomes

BACKGROUND

There are many opinions but little firm knowledge about the optimal treatment of neurogenic stress urinary incontinence (NSUI).

OBJECTIVE

To scrutinize the quality and surgical outcomes of the available treatment modalities in the published literature.

EVIDENCE ACQUISITION

A systematic review of the published literature from Pub Med and Web of Science was undertaken for studies describing surgical treatment of NSUI between 1990 and 2013. A checklist of criteria of methodological and reporting quality of interventions in urological publications was applied to assess quality of the retrieved publications. Surgical outcomes of success, failure, and reoperation were calculated. Statistical analyses included one-way ANOVA and post-hoc tests to determine significant differences between groups.

EVIDENCE SYNTHESIS

Thirty studies were identified with Level 3 evidence. The quality of reporting was 43-81%, with significantly higher quality noted in studies published after 2002 (64% vs. 45%, P < 0.0001). None of the studies followed a randomized controlled trial (RCT) design. Three primary surgical procedures were used in 29 of 30 studies: artificial urinary sphincter (AUS), urethral slings, and urethral bulking agents. One study used a ProACT device. AUS was considered more successful than urethral bulking agents (77 ± 15% vs. 27 ± 20%, P = 0.002). Urethral bulking agents reported higher failures than urethral sling procedures (49 ± 16% vs. 21 ± 19%, P = 0.016) and AUS (21 ± 19% vs. 10 ± 11%, P < 0.002).

CONCLUSIONS

The quality of evidence obtained from non-RCTs is modest. Surgeries for NSUI have relatively high success rates but also high complication rates in this highly heterogeneous population. More studies using modern techniques are required to update our knowledge.

A pilot investigation of a Pediatric Surgery Journal Club

BACKGROUND

The CanMEDS competency "scholar" encompasses the creation, dissemination, application, and translation of medical knowledge. We hypothesize that a structured journal club (JC) for pediatric surgical trainees would meet these objectives in an enjoyable and long-lasting manner.

METHODS

A JC involving two pediatric surgery training programs was created with each session focusing on a specific study design. Pre-tests/post-tests were administered before/after each session with durability of learning assessed during the following session. Metrics analyzed included participant satisfaction and an appraisal of evidence-based medicine (EBM) principals. Test results were analyzed using the paired T-test with p<0.05 considered significant.

RESULTS

On average, 14 participants attended each session, with all trainees present (4). While participants believed they understood EBM principles, 40% were unfamiliar with question formulation, 48% were unfamiliar with critical appraisal tools, and 60% had not appraised an article within the previous year. Pre-test to post-test comparison yielded an improvement in mean score (20=perfect score): 10.8 to 16.9, p<0.01. Measures of participant satisfaction were uniformly positive.

CONCLUSION

A structured Pediatric Surgery Journal Club addresses scholarly training objectives in a highly satisfactory manner and yields durable learning. A web-based curriculum based on this model could serve as an important educational tool for trainees and attending staff alike.

Meta analysis of robot-assisted versus conventional laparoscopic fundoplication in children

BACKGROUND

Minimally invasive fundoplication may be performed using either a robot-assisted (RF) or conventional laparoscopic (LF) technique. Evidence comparing RF and LF in children remains unclear. This study aims to elucidate the comparative safety and efficacy of RF versus LF by systematic review and meta-analysis.

METHODS

Comparative studies investigating RF versus LF in children were identified from multiple electronic literature databases. Meta-analysis was performed using random effects modeling. Safety parameters investigated were post-operative morbidity and intra-operative conversions. Efficacy outcomes of interest were operative success, re-operation, post-operative complications, length of hospital stay (LOS), total operating time (OT), analgesia requirement, and cost.

RESULTS

Six observational studies met inclusion criteria, reporting outcomes of 297 children. No randomized controlled trials were identified. Pooled analysis determined no statistically significant differences between RF and LF for conversions, OT, LOS, and post-operative complications. There was no standardized follow up beyond the early post-operative period to enable data synthesis for remaining outcomes of interest. Limited evidence indicates higher costs with RF.

CONCLUSIONS

Safety and short-term efficacy seem comparable between RF and LF in children. There is insufficient evidence to assess comparative effectiveness for many important procedure specific outcome measures. Higher quality and longer follow-up studies are required.

The timing of ostomy closure in infants with necrotizing enterocolitis: a systematic review

PURPOSE

The optimal timing of ostomy closure is a matter of debate. We performed a systematic review of outcomes of early ostomy closure (EC, within 8 weeks) and late ostomy closure (LC, after 8 weeks) in infants with necrotizing enterocolitis.

METHODS

PubMed, EMbase, Web-of-Science, and Cinahl were searched for studies that detailed time to ostomy closure, and time to full enteral nutrition (FEN) or complications after ostomy closure. Patients with Hirschsprung's disease or anorectal malformations were excluded. Analysis was performed using SPSS 17 and RevMan 5.

RESULTS

Of 778 retrieved articles, 5 met the inclusion criteria. The median score for study quality was 9 [range 8-14 on a scale of 0 to 32 points (Downs and Black, J Epidemiol Community Health 52:377-384, 1998)]. One study described mean time to FEN: 19.1 days after EC (n = 13) versus 7.2 days after LC (n = 24; P = 0.027). Four studies reported complication rates after ostomy closure, complications occurred in 27% of the EC group versus 23% of the LC group. The combined odds ratio (LC vs. EC) was 1.1 [95% CI 0.5, 2.5].

CONCLUSION

Evidence that supports early or late closure is scarce and the published articles are of poor quality. There is no significant difference between EC versus LC in the complication rate. This systematic review supports neither early nor late ostomy closure.

The levels of evidence in pediatric orthopaedic journals: where are we now?

BACKGROUND

In recent years, it has become common to publish a level of evidence grading for orthopaedic journal publications. Our primary research question is: is there an improvement in levels of evidence of articles published in pediatric orthopaedic journals over time? In addition, what is the current status of levels of evidence in pediatric orthopaedic journals?

METHODS

All articles in Journal of Pediatric Orthopaedics-A (JPO-A) and Journal of Pediatric Orthopaedics-B (JPO-B) for 2001, 2002, 2007, and 2008 and those in Journal of Children's Orthopaedics (JCO) for 2007 and 2008, were collected by an independent reviewer. Of the 1,039 articles identified, animal, cadaveric and basic science studies, expert opinion and review articles were excluded. Seven hundred fifty remaining articles were blinded and randomized with respect to journal, title, publication date, author, and institution. According to the currently accepted grading system, study type and level of evidence was assigned to each article. Interobserver and intraobserver reliability were investigated. Statistical analysis was carried out using SPSS software.

RESULTS

There were no statistically significant differences in study type or levels of evidence in articles published before and after 2003. Of articles published during 2007/2008, 3.0% were graded as level I, 5.0% as level II, 24.1% as level III, and 58.0% as level IV. Analysis of the separate journals for all 4 years revealed that JPO-A published 2.6% (13 of 503) level I studies, whereas JPO-B published 4.3% (7 of 163) and JCO published 1.2% (1 of 84). The intraobserver reliability was high for study type (κ, 0.842) and substantial for level of evidence (κ, 0.613). The interobserver reliability for study type and level of evidence was high (κ 0.921 and 0.860, respectively).

CONCLUSIONS

Since the introduction of levels of evidence to orthopaedic journals in 2003, there has been minimal change in the quality of evidence in pediatric orthopaedic publications. We note a modest increase in level III articles and a corresponding decrease in level IV articles. Articles can be reliably graded by nonepidemiologically trained individuals.

LEVEL OF EVIDENCE

Not applicable.

The academic value of rehabilitation medicine meetings

PURPOSE

Twice-yearly meetings of The British Society of Rehabilitation Medicine (BSRM) take place at which posters and free papers are generated, as abstracts, to present novel research findings, audits and case reports. The aim of this study was to evaluate the academic value of these meetings, by determining the subsequent rate of publication in peer-reviewed journals of abstracts presented. This was compared to the publication rate of other European medical specialist society meetings.

METHODS

The authors used MEDLINE, PubMed and Google Scholar search engines to look for publication of abstracts presented at BSRM meetings within peer-reviewed journals over a 7-year period (2000-2006). The abstracts were categorised into sub-groups (original study, audit, review, case report and service description) to determine which type was more likely to be published. The above databases were used also to extract studies on publication rate of other medical specialties in Europe.

RESULTS

In 7 years, a total of 251 abstracts (of which 152 are original studies) have been presented as free papers or posters in a total of 13 meetings. The publication rate for the described study categories were: total 34%, original study 52%, review 50%, case report 5%, audit 0% and service description 0%. Publication rates from other specialist meetings in Europe range from 10% to 70%.

CONCLUSION

The average publication rate for an abstract submitted to a BSRM meeting is 34% for any abstract and 52% for an original study suggesting that the meeting is generating abstracts of comparable academic interest to other specialist societies.

"Evidence-based dentistry in oral surgery: could we do better?"

Evidence-based Dentistry (EBD), like Evidence-based Medicine (EBM), was born in order to seek the “best available research evidence” in the field of dentistry both in research and clinical routine.

But evidence is not clearly measurable in all fields of healthcare: in particular, while drug effect is rather independent from clinician’s characteristics, the effectiveness of surgical procedures is strictly related to surgeon’s expertise, which is difficult to quantify. The research problems of dentistry have a lot in common with other surgical fields, where at the moment the best therapeutic recommendations and guidelines originates from an integration of evidence-based medicine and data from consensus conferences.

To cope with these problems, new instruments have been developed, aimed at standardizing clinical procedures (CAD-CAM technology) and at integrating EBM achievements with the opinions of expert clinicians (GRADE System).

One thing we have to remember however: it is necessary to use the instruments developed by evidence-based medicine but is impossible to produce sound knowledge without considering clinical expertise and quality of surgical procedures simultaneously. Only in this way we will obtain an evidence-based dentistry both in dental research and clinical practice, which is up to third millennium standards.

“Evidence-Based Dentistry in Oral Surgery: Could We Do Better?”

Evidence-based Dentistry (EBD), like Evidence-based Medicine (EBM), was born in order to seek the “best available research evidence” in the field of dentistry both in research and clinical routine.

But evidence is not clearly measurable in all fields of healthcare: in particular, while drug effect is rather independent from clinician’s characteristics, the effectiveness of surgical procedures is strictly related to surgeon’s expertise, which is difficult to quantify. The research problems of dentistry have a lot in common with other surgical fields, where at the moment the best therapeutic recommendations and guidelines originates from an integration of evidence-based medicine and data from consensus conferences.

To cope with these problems, new instruments have been developed, aimed at standardizing clinical procedures (CAD-CAM technology) and at integrating EBM achievements with the opinions of expert clinicians (GRADE System).

One thing we have to remember however: it is necessary to use the instruments developed by evidence-based medicine but is impossible to produce sound knowledge without considering clinical expertise and quality of surgical procedures simultaneously. Only in this way we will obtain an evidence-based dentistry both in dental research and clinical practice, which is up to third millennium standards.

Challenges of randomized controlled surgical trials

The concept of evidence-based medicine has gained broad support in the medical community, because clinical decisions based on information from rigorous scientific study are most likely to provide optimal care. Researchers attempt to answer clinical questions using either observational studies or randomized controlled trials (RCTs). Observational studies currently dominate the surgical literature but provide a level of evidence inferior to RCTs. RCTs are ethically grounded in clinical equipoise and may further reduce the potential for bias or other confounding factors by blinding. This article discusses the barriers to implementation of surgical RCTs.

Randomized controlled trials in neurosurgery--how good are we?

BACKGROUND

The strongest evidence in medical clinical literature is represented by randomized controlled trials (RCTs). This study was designed to evaluate neurosurgically relevant RCTs published recently by neurosurgeons.

METHOD

A literature search in MEDLINE and EMBASE included all clinical studies published up to 30 June 2006. RCTs with neurosurgical relevance published by at least one author with affiliation to a neurosurgical department were selected. The number and characteristics of individual trials were recorded, and the quality of the trials with regard to study design, quality of reporting, and relevance for clinical practice was assessed by two different investigators using a modification of the Scottish Intercollegiate Guidelines Network methodology checklist. Changes of RCT quality over time as well as factors influencing the quality were analyzed.

FINDINGS

From the initial search results (MEDLINE n = 3,860, EMBASE n = 3,113 articles), 159 RCTs published by neurosurgeons were extracted for final evaluation. Of the RCTs, 62% have been published since 1995; 52% came from the USA, UK, and Germany. The median RCT sample size was 78 patients and the median follow-up 35.7 weeks. Fifty-two percent of all RCTs were of good, 37% of moderate, and 11% of bad quality, with an improvement over time. RCTs with financial funding and RCTs with a sample size of >78 patients were of significantly better quality. There were no major differences in the rating of the studies between the two investigators.

CONCLUSIONS

Only a fraction of neurosurgically relevant literature consists of RCTs, but the quality is satisfying and has significantly improved over the last years. An adequate sample size and sufficient financial support seem to be of substantial importance with regard to the quality of the study. Our data also show that by using a standardized checklist, the quality of trials can be reliably assessed by observers of different experience and educational levels.

Blinding of outcomes in trials of orthopaedic trauma: an opportunity to enhance the validity of clinical trials

BACKGROUND

Blinding personnel in randomized controlled trials is an important strategy to minimize bias and increase the validity of the results. Trials of surgical interventions present blinding challenges not seen in drug trials. How often orthopaedic trauma investigators undertake blinding, and the frequency with which they could potentially utilize blinding, remains uncertain.

METHODS

We conducted a systematic review of all randomized controlled trials of orthopaedic trauma published from 1995 to 2004. Two reviewers assessed each trial for eligibility and extracted data regarding its characteristics, outcomes, reporting of blinding, and feasibility of blinding.

RESULTS

We included 171 unique randomized controlled trials spanning a variety of body regions and interventions. The most commonly reported outcomes were clinical (e.g., mortality or wound infection; 91% of trials), radiographic (83%), patient-reported (66%), and physiological results (e.g., range of motion; 56%). Less than 10% of the trials in each category reported the use of blinded outcome assessors. This contrasted with blinding that investigators could have accomplished: blinding was feasible with use of simple methods such as independent assessors, concealed incisions, and masked radiographs for 89% of clinical assessors, 89% of radiographic assessors, 96% of physiological assessors, and 35% of patient-reported assessors.

CONCLUSIONS

Trials in orthopaedic trauma typically measure many outcomes requiring judgment, but the individuals assessing those outcomes are seldom blinded. Investigators have the opportunity to enhance the validity of future clinical trials by incorporating simple blinding techniques.

A guide to planning and executing a surgical randomized controlled trial

Evidence-based medicine requires that treatments given to patients demonstrate effectiveness. The randomized controlled trial (RCT) has become the preeminent study design to assess the efficacy of treatments. Randomized controlled trials are frequently used to evaluate pharmaceutical treatments but are less often used in surgery. The lack of surgical RCTs is partly due to ethical and methodological concerns associated with surgical interventions. We provide a guide to planning and conducting a surgical RCT.

A quality assessment of randomized clinical trials in pediatric orthopaedics

The promotion and practice of evidence-based medicine necessitates a critical evaluation of medical literature, including the criterion standard of randomized clinical trials (RCTs). Recent studies have examined the quality of RCTs in various surgical specialties, but no study has focused on pediatric orthopaedics. The purpose of this study was to assess and describe the quality of RCTs published in the last 10 years in journals with high clinical impact in pediatric orthopaedics. All of the RCTs in pediatric orthopaedics published in 5 well-recognized journals between 1995 and 2005 were reviewed using the Detsky Quality Assessment Scale. The mean percentage score on the Detsky scale was 53% (95% confidence interval, 46%-60%). Only 7 (19%) of the articles satisfied the threshold for a satisfactory level of methodological quality (Detsky >75%). Most RCTs in pediatric orthopaedics that are published in well-recognized peer-reviewed journals demonstrate substantial deficiencies in methodological quality. Particular areas of weakness include inadequate rigor and reporting of randomization methods, use of inappropriate or poorly described outcome measures, inadequate description of inclusion and exclusion criteria, and inappropriate statistical analysis. Further efforts are necessary to improve the conduct and reporting of clinical trials in this field to avoid inadvertent misinformation of the clinical community.

Full publication of results initially presented in abstracts

BACKGROUND

Abstracts of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of abstract results is based on the magnitude or direction of study results, publication bias may result. Publication bias, in turn, creates problems for those conducting systematic reviews or relying on the published literature for evidence.

OBJECTIVES

To determine the rate at which abstract results are subsequently published in full, and the time between meeting presentation and full publication. To assess the association between study characteristics and full publication.

SEARCH STRATEGY

We searched MEDLINE, EMBASE, The Cochrane Library, Science Citation Index, reference lists, and author files. Date of most recent search: June 2003.

SELECTION CRITERIA

We included all reports that examined the subsequent full publication rate of biomedical results initially presented as abstracts or in summary form. Follow-up of abstracts had to be at least two years.

DATA COLLECTION AND ANALYSIS

Two reviewers extracted data. We calculated the weighted mean full publication rate and time to full publication. Dichotomous variables were analyzed using relative risk and random effects models. We assessed time to publication using Kaplan-Meier survival analyses.

MAIN RESULTS

Combining data from 79 reports (29,729 abstracts) resulted in a weighted mean full publication rate of 44.5% (95% confidence interval (CI) 43.9 to 45.1). Survival analyses resulted in an estimated publication rate at 9 years of 52.6% for all studies, 63.1% for randomized or controlled clinical trials, and 49.3% for other types of study designs.'Positive' results defined as any 'significant' result showed an association with full publication (RR = 1.30; CI 1.14 to 1.47), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; CI 1.02 to 1.35), and 'positive' results emanating from randomized or controlled clinical trials (RR = 1.18, CI 1.07 to 1.30). Other factors associated with full publication include oral presentation (RR = 1.28; CI 1.09 to 1.49); acceptance for meeting presentation (RR = 1.78; CI 1.50 to 2.12); randomized trial study design (RR = 1.24; CI 1.14 to 1.36); and basic research (RR = 0.79; CI 0.70 to 0.89). Higher quality of abstracts describing randomized or controlled clinical trials was also associated with full publication (RR = 1.30, CI 1.00 to 1.71).

AUTHORS' CONCLUSIONS

Only 63% of results from abstracts describing randomized or controlled clinical trials are published in full. 'Positive' results were more frequently published than not 'positive' results.

Does it matter if clinicians recruiting for a trial don't understand what the trial is really about? Qualitative study of surgeons' experiences of participation in a pragmatic multi-centre RCT

Background

Qualitative methods are increasingly used to study the process of clinical trials and patients understanding of the rationale for trials, randomisation and reasons for taking part or refusing. Patients' understandings are inevitably influenced by the recruiting clinician's understanding of the trial, yet relatively little qualitative work has explored clinicians' perceptions and understandings of trials. This study interviewed surgeons shortly after the multi-centre, pragmatic RCT in which they had participated had been completed.

Methods

We used in-depth interviews with surgeons who participated in the Spine Stabilisation Trial (a pragmatic RCT) to explore their understanding of the trial purpose and how this understanding had influenced their recruitment procedures and interpretation of the results. A purposive sample of eleven participating surgeons was chosen from 8 of the 15 UK trial centres.

Results

Although the surgeons thought that the trial was addressing an important question there was little agreement about what this question was: although it was a trial of 'equivalent' treatments, some thought that it was a trial of surgery, others a trial of rehabilitation and others that it was exploring what to do with patients in whom all other treatment options had been unsuccessful. The surgeons we interviewed were not aware of the rationale for the pragmatic inclusion criteria and nearly all were completely baffled about the meaning of 'equipoise'. Misunderstandings about the entry criteria were an important source of confusion about the results and led to reluctance to apply the results to their own practice.

Conclusion

The study suggests several lessons for the conduct of future multi-centre trials. Recruiting surgeons (and other clinicians) may not be familiar with the rationale for pragmatic designs and may need to be regularly reminded about the purpose during the study. Reassurance may be necessary that a pragmatic design is not considered a design fault. We conclude that it does matter if clinicians do not understand the rationale for the trial if, as we have shown here, their perception of the trial aims and methods adversely affects who they recruit; if their views affect what the patients are told; and if they mistakenly view the results as unscientific, unreliable and ultimately irrelevant to their practice.

An absence of pediatric randomized controlled trials in general medical journals, 1985-2004

OBJECTIVE

There are numerous potential barriers to conducting randomized controlled trials (RCTs) in children. The purpose of this study was to compare the quantity, trends over time, characteristics, and quality of pediatric RCTs published in general medical journals (GMJs) with adult RCTs.

STUDY DESIGN AND SETTING

We conducted an electronic search of adult and pediatric RCTs from 1985-2004 and a manual search of published RCTs in the year 2000 in five high-impact GMJs (New England Journal of Medicine, Journal of the American Medical Association [JAMA], the Lancet, British Medical Journal [BMJ], Canadian Medical Association Journal [CMAJ]). Linear trends were identified and the 1-year sample was analyzed for publication characteristics (location of recruitment, sample size, number of centers, funding sources, and results) and quality scoring (Jadad score, intention-to-treat analysis, and citation frequency since publication).

RESULTS

Adult RCTs increased by 4.71 RCTs/year (95% confidence interval (CI) 3.62-5.80; P<0.001), which was significantly higher (P<0.0001) than pediatric RCTs, which increased by 0.4 RCTs/year (95% CI -0.02 to 0.9; P=0.06). Adult RCTs were more likely to be hospital-based (P=.001) and to involve more centers in multicenter studies (P=0.02). Quality scores were similar, although adult RCTs were cited more frequently (P=0.003).

CONCLUSION

There may be significant barriers to the publication of high-quality pediatric RCTs in GMJs.

Randomized Controlled Trials in Pediatric Urology: Room for Improvement

PURPOSE

The primary purpose of this study was to ascertain what proportion of the body of published literature in pediatric urology is represented by randomized controlled trials. The secondary purpose was to assess the quality of these trials.

MATERIALS AND METHODS

Using a predefined strategy, we conducted systematic computerized searches of the MEDLINE (years 1966 to 2004) and EMBASE (1980 to 2004) databases to identify all English language randomized controlled trials related to pediatric urology. Full text versions of identified studies were reviewed in blinded fashion for key demographic, methodological and statistical characteristics. Trial quality was assessed with the previously validated Jadad tool.

RESULTS

The 77 identified randomized controlled trials represented only 0.4% to 0.9% of the indexed pediatric urology literature. The origins of these trials were Europe (40%), North America (26%) and a variety of other geographic centers (34%). A primarily surgical focus was present in 43% of the studies. Trials with negative results represented only 19% of the total randomized controlled trials. Generally, the trials were of low to fair quality (median Jadad score 3), with substandard methodological reporting and planning. There was not a significant trend toward improved quality in recent years. Trials from North America and Europe had higher quality (p = 0.007), as did those reporting negative results (p = 0.0001).

CONCLUSIONS

Randomized controlled trials in pediatric urology constitute only a small proportion of the body of published literature in the field. High quality studies are uncommon. Efforts should be made to increase the number of well designed, randomized controlled trials in pediatric urology.

La chirurgie factuelle et ses difficultés

Evidence based medicine can be defined as the application of the best evidence in the care of a given patient. When applied to surgical practice, it appears that this concept has some limitations. To discuss these limitations, the authors made the choice to discuss the terms the original definition. Some factors are related to the paucity and the poor quality of randomized controlled trials and meta-analyses in surgery, to the difficulties to appraise the surgical publications and apply the results of randomized trials to a given patient, and to bring the surgeons more willing to endorse the principles of evidence-based medicine. But all these limitations could be overcome making evidence-based surgery not to be a simple passing fad but a formal paradigm.

Limits of Evidence-based Surgery

Evidence-based medicine can be summarized as the use of current best evidence in the care of individual patients. When applied to surgical practice, it appears clearly that the concept of evidence-based medicine involves some limitations. To discuss these limits, the author went back over the terms of the evidence-based medicine definition. Limits are related to the low quantity and quality of randomized controlled trials and meta-analyses in surgery, the difficulties when critically appraising the literature and applying the results of evidence to individual patients, and bringing surgeons more willing to endorse the principles of evidence-based medicine. However all these limits can be overcome in the future, with the aim that evidence-based surgery will not be a passing fad.

Randomized Trials of Neurosurgical Interventions: A Systematic Appraisal

OBJECTIVE

To systematically appraise the study design and quality of reporting of randomized controlled trials (RCTs) on neurosurgical procedures and to identify potential defects and biases.

METHODS

Randomized controlled trials with at least five patients comparing any neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, EMBASE, and the Cochrane Library. We analyzed study design, quality of reporting, and trial results.

RESULTS

The median sample size in the 108 eligible reports was 68 patients. Ninety-nine trials (91.7%) reported inclusion and exclusion criteria, 55 (50.9%) mentioned the randomization mode, and 87 (80.6%) adequately described withdrawals, but only 31 (28.7%) described allocation concealment, only 23 (21.3%) gave power calculations, and only 20 (18.5%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 46 reports (42.6%), and no difference between the compared procedures was found in 60 trials (55.6%). Trials with a larger sample size were more likely to report withdrawals (P = 0.02) and power calculations (P = 0.006). Only 14 trials (13.6%) were double-blind, and this was less frequent in longer trials (P = 0.02). Among quality criteria, only the reporting of randomization mode improved significantly over time (P = 0.015).

CONCLUSION

Several aspects of the design and reporting of randomized controlled trials on neurosurgical procedures can be improved. Larger, adequately powered, and accurately reported trials are needed.