Surgery and the randomised controlled trial: past, present and future

Global trends and research status in ankylosing spondylitis clinical trials: a bibliometric analysis of the last 20 years

Background

Ankylosing spondylitis (AS) is a rheumatic and autoimmune disease associated with a chronic inflammatory response, mainly characterized by pain, stiffness, or limited mobility of the spine and sacroiliac joints. Severe symptoms can lead to joint deformity, destruction, and even lifelong disability, causing a serious burden on families and society as a whole. A large number of clinical studies have been published on AS over the past 20 years. This study aimed to summarize the current research status and global trends relating to AS clinical trials through a bibliometric analysis.

Methods

The Web of Science Core Collection database was searched for publications related to AS clinical trials published between January 2003 and June 2023. Bibliometric analysis and web visualization were performed using CiteSpace, VOSviewer, and a bibliometric online analysis platform (https://bibliometric.com), which included the number of publications, citations, countries, institutions, journals, authors, references, and keywords.

Results

1,212 articles published in 201 journals from 65 countries were included in this study. The number of publications related to AS clinical trials is increasing annually. The United States and the Free University of Berlin, the countries and institutions, respectively, that have published the most articles on AS, have made outstanding contributions to this field. The author with the most published papers and co-citations over the period covered by the study was Desiree Van Der Heijde. The journal with the most published and cited articles was Annals of the Rheumatic Diseases. The keywords: "double-blind," "rheumatoid arthritis," "efficacy," "placebo-controlled trial," "infliximab," "etanercept," "psoriatic arthritis" and "therapy" represent the current research hotspots regarding AS.

Discussion

This is the first study to perform a bibliometric analysis and visualization of AS clinical trial publications, providing a reliable research focus and direction for clinicians. Future studies in the field of AS clinical trials should focus on placebo-controlled trials of targeted therapeutic drugs.

Characteristics of baseline frequency data in spinal RCTs do not suggest widespread non-random allocation

BACKGROUND

Recent signs of fraudulent behaviour in spine RCTs have queried the integrity of trials in the field. RCTs are particularly important due to the weight they are accorded in guiding treatment decisions, and thus, ensuring RCTs' reliability is crucial. This study investigates the presence of non-random baseline frequency data in purported RCTs published in spine journals.

METHODS

A PubMed search was performed to obtain all RCTs published in four spine journals (Spine, The Spine Journal, the Journal of Neurosurgery Spine, and European Spine Journal) between Jan-2016 and Dec-2020. Baseline frequency data were extracted, and variable-wise p values were calculated using the Pearson Chi-squared test. These p values were combined for each study into study-wise p values using the Stouffer method. Studies with p values below 0.01 and 0.05 and those above 0.95 and 0.99 were reviewed. Results were compared to Carlisle's 2017 survey of anaesthesia and critical care medicine RCTs.

RESULTS

One hundred sixty-seven of the 228 studies identified were included. Study-wise p values were largely consistent with expected genuine randomized experiments. Slightly more study-wise p values above 0.99 were observed than expected, but a number of these had good explanations to account for that excess. The distribution of observed study-wise p values was more closely matched to the expected distribution than those in a similar survey of the anaesthesia and critical care medicine literature.

CONCLUSION

The data surveyed do not show evidence of systemic fraudulent behaviour. Spine RCTs in major spine journals were found to be consistent with genuine random allocation and experimentally derived data.

A mixed-methods systematic review of patients' experience of being invited to participate in surgical randomised controlled trials

RATIONALE

Randomised controlled trials (RCTs) of surgical interventions are increasing. Such trials encounter challenges that are not present in RCTs of non-surgical interventions because of the nature of the intervention. Several studies have explored patients' experiences of surgical trials to improve recruitment or identify barriers and facilitators to research in this setting. Synthesizing these studies may reveal further insights or confirm whether saturation of relevant themes has been achieved.

OBJECTIVE

This review aimed to understand the experiences of adults who are invited to participate in surgical RCTs.

METHOD

MEDLINE, Web of Science, and CINAHL were searched to identify articles meeting the inclusion criteria. Assessment of quality was conducted with studies given an overall quality rating of good, fair, or poor. A segregated approach was used to synthesize the data. This method included a thematic synthesis of the qualitative data and a narrative review of the quantitative data. The findings of both syntheses were then integrated.

RESULTS

Thirty-four articles reporting 28 trials were included. This review found that the decision to participate in a surgical trial is influenced by multiple factors including patients' individual circumstances and attitudes, and the characteristics of the trial itself. The study identified three themes which encompass both qualitative and quantitative findings. These themes reveal it was important for patients to i) make sense of the trial and trial processes, ii) weigh up the risks and benefits of their different treatment options and participation, and iii) trust the trial and staff.

CONCLUSIONS

A patient-centred approach to trial recruitment may help staff build trusting relationships with patients and address their individual concerns about the trial and the risks and benefits of participation.

Norwegian moratorium on transanal total mesorectal excision

A national decision

Introduction of Novel Surgical Techniques: A Survey on Knowledge, Attitude, and Practice of Surgeons

Purpose. To investigate the knowledge, attitude, and practice of surgeons toward introducing novel surgical techniques in Egypt, Palestine, and Vietnam. Summary Background Data. Despite the recent advances in modern surgical care and its role in advancing the quality and the length of lives, surgery in the developing world has stagnated or even regressed. Methods. A survey was undertaken among the surgeons in 9 hospitals belonging to the 3 countries. Questions were categorized into knowledge, attitude, and practice questions. Meta-analyses were performed to estimate the event rate and compare between knowledge and practice, senior and junior surgeons. Results. A total of 244 responses, with a response rate of 79.7%, were included in the analysis. Regarding knowledge and attitude, the results were satisfactory except that only 55.8% of surgeons appraised their level of education and 43.3% wanted to earn money from the novel procedure. There was a significant difference between knowledge and practice regarding getting informed consent from the patients (P = .024), discussing the novelty of the procedure (P < .001), discussing the alternative procedures (P < .001), discussing the surgeons' experience and level of skills (P < .001), discussing the risk of the new procedure (P < .001), and monitoring the outcomes after the new procedure (P < .001). Conclusions. Most surgeons have sufficient knowledge and are motivated regarding adopting novel surgical techniques in order to provide the best care for the patients. However, there was a gap between knowledge and practice. Training programs and evidence-based guidelines regarding the introduction of novel surgical techniques are needed to overcome these challenges.

A protocol for the development of reporting guidelines for IDEAL stage studies

•

The IDEAL Framework and Recommendations guide evaluation of surgical innovations.

•

Current reporting guidelines do not cover every stage of surgical innovation.

•

We propose to develop guidance using a Delphi consensus process to fill these gaps.

•

Guidelines will facilitate the use of IDEAL for evaluating surgical innovations.

Background

New surgical procedures, devices and other complex interventions need robust evaluation for safety, efficacy and effectiveness. The IDEAL Framework and Recommendations lay out a pathway to achieve this and offer general guidance on how studies at each stage should be reported. However, researchers require some assistance in translating theory into practice. We will develop a set of reporting guidelines for each IDEAL stage where deemed necessary through Delphi consensus methodology.

Methods

For each IDEAL stage requiring a new set of reporting guidelines, we will use the following process. We will search for the relevant reporting guidelines already in existence and use principles developed by the IDEAL Collaboration to compile the initial long list of potential checklist items. In each round, the participants will rate the importance of reporting each element on a nine-point Likert scale as proposed by the GRADE group. Sequential rounds and questionnaire administration and completion will take place until a final set of items is produced. There will then be a final consensus meeting of a working group to condense and refine the final recommendations for the reporting guidelines.

Clinical trials in valvular surgery: a 2018 update

PURPOSE OF REVIEW

There is a growing emphasis on the conduct of large-scale, multicenter randomized controlled trials (RCTs) to guide decision-making in cardiac surgery. Here we review recent landmark RCTs in cardiac valvular surgery.

RECENT FINDINGS

RCTs are the gold-standard level of data in medicine. However, there are unique challenges of conducting large-scale surgical trials including funding, blinding, generalizability, nonstandardization of the surgical technique, crossover, among others. Thus, the vast majority of clinical outcomes data in cardiac surgery are mainly from observational studies and most prospective data are small, single-center trials. The Cardiothoracic Surgery Network is the largest platform focused on the conduct of high-quality, multicenter cardiac surgical trials, which has already produced several seminal guideline-changing and practice-changing contributions to the surgical approach to functional mitral regurgitation, aortic stenosis, atrial fibrillation, and neuroprotective surgical adjuncts.

SUMMARY

There continues to be great interest in the conduct of high-quality, RCTs to help guide surgical management of patients with valvular heart disease.

The reporting quality of parallel randomised controlled trials in ophthalmic surgery in 2011: a systematic review

PURPOSE

Randomised controlled trials (RCTs) represent a gold standard for evaluating therapeutic interventions. However, poor reporting clarity can prevent readers from assessing potential bias that can arise from a lack of methodological rigour. The Consolidated Standards of Reporting Trials statement for non-pharmacological interventions 2008 (CONSORT NPT) was developed to aid reporting. RCTs in ophthalmic surgery pose particular challenges in study design and implementation. We aim to provide the first assessment of the compliance of RCTs in ophthalmic surgery to the CONSORT NPT statement.

METHOD

In August 2012, the Medline database was searched for RCTs in ophthalmic surgery reported between 1 January 2011 and 31 December 2011. Results were searched by two authors and relevant papers selected. Papers were scored against the 23-item CONSORT NPT checklist and compared against surrogate markers of paper quality. The CONSORT score was also compared between different RCT designs.

RESULTS

In all, 186 papers were retrieved. Sixty-five RCTs, involving 5803 patients, met the inclusion criteria. The mean CONSORT score was 8.9 out of 23 (39%, range 3.0-14.7, SD 2.49). The least reported items related to the title and abstract (1.6%), reporting intervention adherence (3.1%), and interpretation of results (4.7%). No significant correlation was found between CONSORT score and journal impact factor (R=0.14, P=0.29), number of authors (R=0.01, P=0.93), or whether the RCT used paired-eye, one-eye, or two-eye designs in their randomisation (P=0.97).

CONCLUSIONS

The reporting of RCTs in ophthalmic surgery is suboptimal. Further work is needed by trial groups, funding agencies, authors, and journals to improve reporting clarity.

A systematic review of the quality of randomized controlled trials in head and neck oncology surgery

OBJECTIVES/HYPOTHESIS

To determine the quality of randomized controlled trials (RCTs) in head and neck surgery in which surgery was a primary intervention.

DATA SOURCES

Potential articles were identified in PubMed without publication date restrictions.

REVIEW METHODS

Articles were scored using the CONSORT checklist and the relationship between the checklist score and whether the first and/or last authors were surgeons was investigated. Differences in the checklist score based on how many surgeons were among the first and last authors of the study were analyzed using the Kruskal-Wallis test. Fisher's exact test was used to examine if there was a significant difference of the reporting of individual items from the checklist between surgeons and nonsurgeons. A nonparametric trend test was used to determine whether there was a difference in the reporting of individual items based on whether there were none, one, or two surgeons among first and last authors.

RESULTS

A total of 38 publications satisfied the inclusion criteria. There was a trend toward lower quality for studies in which surgeons were either first, last, or both first and last authors compared to studies that were first-authored and last-authored by nonsurgeons (P = 0.068). Nonsurgeons were more likely to report on critical elements regarding hypothesis, sample size determination, randomization, and eligibility of centers (P = 0.023-0.058).

CONCLUSION

The quality of RCTs in head and neck surgery is poor. Improved training in conducting and reporting clinical research is needed in otolaryngology residencies.

Time to be BRAVE: is educating surgeons the key to unlocking the potential of randomised clinical trials in surgery? A qualitative study

BACKGROUND

Well-designed randomised clinical trials (RCTs) provide the best evidence to inform decision-making and should be the default option for evaluating surgical procedures. Such trials can be challenging, and surgeons' preferences may influence whether trials are initiated and successfully conducted and their results accepted. Preferences are particularly problematic when surgeons' views play a key role in procedure selection and patient eligibility. The bases of such preferences have rarely been explored. Our aim in this qualitative study was to investigate surgeons' preferences regarding the feasibility of surgical RCTs and their understanding of study design issues using breast reconstruction surgery as a case study.

METHODS

Semistructured qualitative interviews were undertaken with a purposive sample of 35 professionals practicing at 15 centres across the United Kingdom. Interviews were transcribed verbatim and analysed thematically using constant comparative techniques. Sampling, data collection and analysis were conducted concurrently and iteratively until data saturation was achieved.

RESULTS

Surgeons often struggle with the concept of equipoise. We found that if surgeons did not feel 'in equipoise', they did not accept randomisation as a method of treatment allocation. The underlying reasons for limited equipoise were limited appreciation of the methodological weaknesses of data derived from nonrandomised studies and little understanding of pragmatic trial design. Their belief in the value of RCTs for generating high-quality data to change or inform practice was not widely held.

CONCLUSION

There is a need to help surgeons understand evidence, equipoise and bias. Current National Institute of Health Research/Medical Research Council investment into education and infrastructure for RCTs, combined with strong leadership, may begin to address these issues or more specific interventions may be required.

Randomised controlled trials in plastic surgery: a systematic review of reporting quality

BACKGROUND

We recently conducted a systematic review of the methodological quality of randomised controlled trials (RCTs) in plastic surgery. In accordance with convention, we are here separately reporting a systematic review of the reporting quality of the same RCTs.

METHODS

MEDLINE® and the Cochrane Database of Systematic Reviews were searched by an information specialist from 1 January 2009 to 30 June 2011 for the MESH heading 'Surgery, Plastic'. Limitations were entered for English language, human studies and randomised controlled trials. Manual searching for RCTs involving surgical techniques was performed within the results. Scoring of the eligible papers was performed against the 23-item CONSORT Statement checklist. Independent secondary scoring was then performed and discrepancies resolved through consensus.

RESULTS

Fifty-seven papers met the inclusion criteria. The median CONSORT score was 11.5 out of 23 items (range 5.3-21.0). Items where compliance was poorest included intervention/comparator details (7 %), randomisation implementation (11 %) and blinding (26 %). Journal 2010 impact factor or number of authors did not significantly correlate with CONSORT score (Spearman rho = 0.25 and 0.12, respectively). Only 61 % declared conflicts of interest, 75 % permission from an ethics review committee, 47 % declared sources of funding and 16 % stated a trial registry number. There was no correlation between the volume of RCTs performed in a particular country and reporting quality.

CONCLUSIONS

The reporting quality of RCTs in plastic surgery needs improvement. Better education, awareness amongst all stakeholders and hard-wiring compliance through electronic journal submission systems could be the way forward. We call for the international plastic surgical community to work together on these long-standing problems.

Evaluating the extent of clinical uncertainty among treatment options for patients with early-onset scoliosis

BACKGROUND

Literature guiding the management of early-onset scoliosis consists primarily of studies with a low level of evidence. Evaluation of clinical equipoise (i.e., when there is no known superiority among treatment modalities) allows for prioritization of research efforts. The objective of this study was to evaluate areas of clinical uncertainty among pediatric spine surgeons regarding the treatment of early-onset scoliosis.

METHODS

Fourteen experienced pediatric spine surgeons participated in semistructured interviews to identify clinical variables that influence decision making in the treatment of early-onset scoliosis. A series of case scenarios of 315 patients with idiopathic and neuromuscular early-onset scoliosis was then developed to be representative of those encountered in clinical practice. Using an online survey, eleven surgeons selected their choice of eight treatment options for each case scenario. Associations between case characteristics and treatment choices were assessed with chi-square and logistic regression analysis. Participants then reviewed the areas of treatment uncertainty identified in the survey, nominated additional research questions of interest, and ranked their interest to further explore the identified research questions.

RESULTS

Collective equipoise was identified in numerous scenarios in the survey spanning a range of ages and magnitudes of scoliosis, and additional questions were identified during the nominal group technique. Areas that had the greatest clinical uncertainty included the management of patients who have finished treatment with a growing-rod, timing of rod-lengthening intervals, and indications for spine-based and rib-based proximal instrumentation anchors. The use of rib anchors compared with spine-based anchors was ranked highly for consideration in future clinical trials.

CONCLUSIONS

Variability in decision making with regard to the optimum treatment of certain subsets of patients with early-onset scoliosis reflects gaps in the available evidence. Structured consensus methods identified priorities for higher levels of research in this area of scoliosis. Higher-level studies, including randomized trials, should focus on answering the questions highlighted in this report.

Laparoscopic repair for perforated peptic ulcer disease

BACKGROUND

Perforated peptic ulcer is a common abdominal disease that is treated by surgery. The development of laparoscopic surgery has changed the way to treat such abdominal surgical emergencies. The results of some clinical trials suggest that laparoscopic surgery could be a better strategy than open surgery in the correction of perforated peptic ulcer but the evidence is not strongly in favour for or against this intervention.

OBJECTIVES

To measure the effect of laparoscopic surgical treatment versus open surgical treatment in patients with a diagnosis of perforated peptic ulcer in relation to abdominal septic complications, surgical wound infection, extra-abdominal complications, hospital length of stay and direct costs.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) on The Cochrane Library (2004, Issue 2), PubMed/MEDLINE (1966 to July 2004), EMBASE (1985 to November 2004) and LILACS (1988 to November 2004) as well as the reference lists of relevant articles. Searches in all databases were updated in December 2009 and January 2012. We did not confine our search to English language publications.

SELECTION CRITERIA

Randomized clinical trials comparing laparoscopic surgery versus open surgery for the repair of perforated peptic ulcer using any mechanical method of closure (suture, omental patch or fibrin sealant).

DATA COLLECTION AND ANALYSIS

Primary outcome measures included proportion of septic and other abdominal complications (surgical site infection, suture leakage, intra-abdominal abscess, postoperative ileus) and extra-abdominal complications (pulmonary). Secondary outcomes included mortality, time to return to normal diet, time of nasogastric aspiration, hospital length-of-stay and costs. Outcomes were summarized by reporting odds ratios (ORs) and 95% confidence intervals (CIs), using the fixed-effect model.

MAIN RESULTS

We included three randomized clinical trials of acceptable quality. We found no statistically significant differences between laparoscopic and open surgery in the proportion of abdominal septic complications (OR 0.66; 95% CI 0.30 to 1.47), pulmonary complications (OR 0.43; 95% CI 0.17 to 1.12) or number of septic abdominal complications (OR 0.60; 95% CI 0.32 to 1.15). Heterogeneity was significant for pulmonary complications and operating time.

AUTHORS' CONCLUSIONS

This review suggests that a decrease in septic abdominal complications may exist when laparoscopic surgery is used to correct perforated peptic ulcer. However, it is necessary to perform more randomized controlled trials with a greater number of patients to confirm such an assumption, guaranteeing a long learning curve for participating surgeons. With the information provided it could be said that laparoscopic surgery results are not clinically different from those of open surgery.

Trials and tribulations: the professional development of surgical trialists

BACKGROUND

Regulatory and professional bodies issue an ever-increasing number of guidance documents on the ethics and methods of clinical trials, but the quality of clinical trials of invasive therapeutic procedures continues to be a concern. We interviewed aspiring and accomplished surgical trialists to understand how they use guidance documents and other resources in their work.

METHODS

We performed a qualitative research study involving semistructured interviews of a diverse sample of 15 surgical trialists.

RESULTS

Professional development as a surgical trialist was haphazard, inefficient, and marked by avoidable mistakes. Four types of resources played constructive roles: formal education; written materials on clinical trials; experience with actual trials; and interpersonal interactions with peers, experts, collaborators, and mentors. Recommendations for improvement centered on education, mentoring, networking, participating in trials, and facilitation by department chairs.

CONCLUSIONS

The haphazard and unstructured nature of the current system is adding unnecessarily to the numerous challenges faced by surgical trialists.

Surgical management for upper urinary tract transitional cell carcinoma (UUT-TCC): a systematic review

Surgical management of upper urinary tract transitional cell carcinoma (UUT-TCC) has significantly changed over the past two decades. Data for several new surgical techniques, including nephron-sparing surgery (NSS), is emerging. The study systematically reviewed the literature comparing (randomised and observational studies) surgical and oncological outcomes for various surgical techniques MEDLINE, EMBASE, Cochrane Library, CINAHL, British Nursing Index, AMED, LILACS, Web of Science, Scopus, Biosis, TRIP, Biomed Central, Dissertation Abstracts, ISI proceedings, and PubMed were searched to identify suitable studies. Data were extracted from each identified paper independently by two reviewers (B.R. and B.S.) and cross checked by a senior member of the team. The data analysis was performed using the Cochrane software Review manager version 5. Comparable data from each study was combined in a meta-analysis where possible. For dichotomous data, odds ratios with 95% confidence intervals (CIs) were estimated based on the fixed-effects model and according to an intention-to-treat analysis. If the data available were deemed not suitable for a meta-analysis it was described in a narrative fashion. One randomised control trial (RCT) and 19 observational studies comparing open nephroureterectomy (ONU) and laparoscopic NU (LNU) were identified. The RCT reported the LNU group to have statistically significantly less blood loss (104 vs 430 mL, P < 0.001) and mean time to discharge (2.30 vs 3.65 days, P < 0.001) than the ONU group. At a median follow-up of 44 months, the overall 5-year cancer-specific survival (CSS; 89.9 vs 79.8%) and 5-year metastasis-free survival rates (77.4 vs 72.5%) for the ONU were better than for LNU, respectively, although not statistically significant. A meta-analysis of the observational studies favoured LNU group for lower urinary recurrence (P < 0.001) and distant metastasis. The meta-analyses for local recurrence for the two groups were comparable. One retrospective study comparing ONU with a percutaneous approach for grade 2 disease reported no significant differences in CSS rates (53.8 vs 53.3 months). Three retrospective studies compared NSS and radical NU, and reported no significant differences in overall CSS and recurrence-free survival between the two approaches. Five retrospective studies compared various techniques of en bloc excision of the lower ureter. No technique was reported to be better (operative and oncological) than any other. This review concludes that there is a paucity of good quality evidence for the various surgical approaches for UUT-TCC. The techniques have been assessed and reported in many retrospective single-centre studies favouring LNU for better perioperative outcomes and comparable oncological safety. The reported observational studies data is further supported by one RCT.

Planned vaginal birth or elective repeat caesarean: patient preference restricted cohort with nested randomised trial

BACKGROUND

Uncertainty exists about benefits and harms of a planned vaginal birth after caesarean (VBAC) compared with elective repeat caesarean (ERC). We conducted a prospective restricted cohort study consisting of a patient preference cohort study, and a small nested randomised trial to compare benefits and risks of a planned ERC with planned VBAC.

METHODS AND FINDINGS

2,345 women with one prior caesarean, eligible for VBAC at term, were recruited from 14 Australian maternity hospitals. Women were assigned by patient preference (n = 2,323) or randomisation (n = 22) to planned VBAC (1,225 patient preference, 12 randomised) or planned ERC (1,098 patient preference, ten randomised). The primary outcome was risk of fetal death or death of liveborn infant before discharge or serious infant outcome. Data were analysed for the 2,345 women (100%) and infants enrolled. The risk of fetal death or liveborn infant death prior to discharge or serious infant outcome was significantly lower for infants born in the planned ERC group compared with infants in the planned VBAC group (0.9% versus 2.4%; relative risk [RR] 0.39; 95% CI 0.19-0.80; number needed to treat to benefit 66; 95% CI 40-200). Fewer women in the planned ERC group compared with women in the planned VBAC had a major haemorrhage (blood loss ≥ 1,500 ml and/or blood transfusion), (0.8% [9/1,108] versus 2.3% [29/1,237]; RR 0.37; 95% CI 0.17-0.80).

CONCLUSIONS

Among women with one prior caesarean, planned ERC compared with planned VBAC was associated with a lower risk of fetal and infant death or serious infant outcome. The risk of major maternal haemorrhage was reduced with no increase in maternal or perinatal complications to time of hospital discharge. Women, clinicians, and policy makers can use this information to develop health advice and make decisions about care for women who have had a previous caesarean.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN53974531

Management of perforated peptic ulcer in a district general hospital

INTRODUCTION

Laparoscopic surgery has become increasingly popular for elective surgery but it has gained slow transference to emergency surgery. The management of perforated peptic ulcers (PPU) laparoscopically is an accepted strategy yet it still remains infrequently used. The purpose of this study was to analyse the utility and outcomes of laparoscopy versus open repair for PPU in a district general hospital. In addition, we evaluated whether the subspecialty of the on-call consultant affected the method of repair performed and the training opportunities for trainee surgeons.

METHODS

Between 2003 and 2009, 53 patients underwent laparoscopic repair, 89 patients underwent open repair and a further 20 patients had laparoscopic repair that was converted to open repair for PPU. The results from a prospectively compiled database were analysed with primary outcome measures including operative time, length of hospital stay and mortality.

RESULTS

The median operating time in the laparoscopic group was 60.0 minutes compared with 50.5 minutes in the open group. Hospital stay in surviving patients was significantly shorter in patients treated completely laparoscopically (5 days) when compared with the open group (6 days) ( p <0.01). There were six deaths in the laparoscopic group (11%) compared with 13 in the open group (15%) and one in the converted group (5%). Trainees performed 53% (47/89) of open repairs and 13% (7/54) of laparoscopic repairs.

CONCLUSIONS

Both laparoscopic and open repair are equally safe in the management of PPU. Our findings support the view that this procedure can be successfully used as a training operation.

Statistics: The stethoscope of a thinking urologist

Understanding statistical terminology and the ability to appraise clinical research findings and statistical tests are critical to the practice of evidence-based medicine. Urologists require statistics in their toolbox of skills in order to successfully sift through increasingly complex studies and realize the drawbacks of statistical tests. Currently, the level of evidence in urology literature is low and the majority of research abstracts published for the American Urological Association (AUA) meetings lag behind for full-text publication because of a lack of statistical reporting. Underlying these issues is a distinct deficiency in solid comprehension of statistics in the literature and a discomfort with the application of statistics for clinical decision-making. This review examines the plight of statistics in urology and investigates the reason behind the white-coat aversion to biostatistics. Resources such as evidence-based medicine websites, primers in statistics, and guidelines for statistical reporting exist for quick reference by urologists. Ultimately, educators should take charge of monitoring statistical knowledge among trainees by bolstering competency requirements and creating sustained opportunities for statistics and methodology exposure.

Consumer input into research: the Australian Cancer Trials website

BACKGROUND

The Australian Cancer Trials website (ACTO) was publicly launched in 2010 to help people search for cancer clinical trials recruiting in Australia, provide information about clinical trials and assist with doctor-patient communication about trials. We describe consumer involvement in the design and development of ACTO and report our preliminary patient evaluation of the website.

METHODS

Consumers, led by Cancer Voices NSW, provided the impetus to develop the website. Consumer representative groups were consulted by the research team during the design and development of ACTO which combines a search engine, trial details, general information about trial participation and question prompt lists. Website use was analysed. A patient evaluation questionnaire was completed at one hospital, one week after exposure to the website.

RESULTS

ACTO's main features and content reflect consumer input. In February 2011, it covered 1, 042 cancer trials. Since ACTO's public launch in November 2010, until the end of February 2011, the website has had 2, 549 new visits and generated 17, 833 page views. In a sub-study of 47 patient users, 89% found the website helpful for learning about clinical trials and all respondents thought patients should have access to ACTO.

CONCLUSIONS

The development of ACTO is an example of consumers working with doctors, researchers and policy makers to improve the information available to people whose lives are affected by cancer and to help them participate in their treatment decisions, including consideration of clinical trial enrolment. Consumer input has ensured that the website is informative, targets consumer priorities and is user-friendly. ACTO serves as a model for other health conditions.

A Comparative Effectiveness Analysis of Alternative Strategies to Assess Hypothalamic-Pituitary-Adrenal Axis Function After Microsurgical Resection of Pituitary Tumors

BACKGROUND:

Observational methods can be used in conjunction with the comparative effectiveness research (CER) paradigm to inform decisions between alternative patient management strategies in real-world clinical settings.

OBJECTIVE:

To present a brief review of current observational research regarding 3 strategies for predicting normal hypothalamic-pituitary-adrenal (HPA) axis function after surgical resection of pituitary tumors and to apply the CER model to compare these management alternatives.

METHODS:

We designed and conducted 2 prospective observational studies involving 183 patients undergoing microsurgical resection for pituitary tumors. These investigations yielded a comprehensive database comprising longitudinal data from multiple clinical domains. We investigated 3 potential strategies to predict normal postoperative HPA axis function in this cohort, including preoperative adrenocorticotrophic hormone stimulation testing and measurement of serum cortisol levels immediately after surgery or on the first postoperative day. We performed a focused comparative effectiveness review to help inform the decision between the 3 potential clinical management strategies.

RESULTS:

This investigation illustrates the use of observational research methods in conjunction with CER methodology as one means of informing clinical management decisions. Of the 3 strategies for assessing postoperative HPA axis function studied, preoperative and immediate postoperative adrenocorticotrophic hormone stimulation testing had the highest sensitivity, accuracy, and positive predictive value for normal HPA axis function postoperatively. The preoperative strategy was also the most cost-effective approach (12% reduction vs benchmark).

CONCLUSION:

The decision to use any of the 3 strategies outlined requires attention to a specific patient's clinical situation, but this decision may be aided by the results of this CER analysis.

The role of observational investigations in comparative effectiveness research

INTRODUCTION

Comparative effectiveness research (CER) seeks to inform clinical decisions between alternate treatment strategies using data that reflects real patient populations and real-world clinical scenarios for the purpose of improving patient outcomes. There are multiple clinical situations where the unique characteristics of observational investigations can inform medical decision-making within the CER paradigm. Accordingly, it is critical for clinicians to appreciate the strengths and limitations of observational research, particularly as they apply to CER.

METHODS

This review focuses on the role of observational research in CER. We discuss the concept of evidence hierarchies as they relate to observational research and CER, review the scope and nature of observational research, present the rationale for its inclusion in CER investigations, discuss potential sources of bias in observational investigations as well as strategies used to compensate for these biases, and discuss a framework to implement observational research in CER.

CONCLUSIONS

The CER paradigm recognizes the limitations of hierarchical models of evidence and favors application of a strength-of-evidence model. In this model, observational research fills gaps in randomized clinical trial data and is particularly valuable to investigate effectiveness, harms, prognosis, and infrequent outcomes as well as in circumstances where randomization is not possible and in studies of many surgical populations. Observational investigations must be designed with careful consideration of potential sources of bias and must incorporate strategies to control such bias prospectively, and their results must be reported in a uniform and transparent fashion. When these conditions can be achieved, observational research represents a valuable and critical component of modern CER.

Evaluation and stages of surgical innovations

Surgical innovation is an important part of surgical practice. Its assessment is complex because of idiosyncrasies related to surgical practice, but necessary so that introduction and adoption of surgical innovations can derive from evidence-based principles rather than trial and error. A regulatory framework is also desirable to protect patients against the potential harms of any novel procedure. In this first of three Series papers on surgical innovation and evaluation, we propose a five-stage paradigm to describe the development of innovative surgical procedures.

Randomized controlled trials in neurosurgery--how good are we?

BACKGROUND

The strongest evidence in medical clinical literature is represented by randomized controlled trials (RCTs). This study was designed to evaluate neurosurgically relevant RCTs published recently by neurosurgeons.

METHOD

A literature search in MEDLINE and EMBASE included all clinical studies published up to 30 June 2006. RCTs with neurosurgical relevance published by at least one author with affiliation to a neurosurgical department were selected. The number and characteristics of individual trials were recorded, and the quality of the trials with regard to study design, quality of reporting, and relevance for clinical practice was assessed by two different investigators using a modification of the Scottish Intercollegiate Guidelines Network methodology checklist. Changes of RCT quality over time as well as factors influencing the quality were analyzed.

FINDINGS

From the initial search results (MEDLINE n = 3,860, EMBASE n = 3,113 articles), 159 RCTs published by neurosurgeons were extracted for final evaluation. Of the RCTs, 62% have been published since 1995; 52% came from the USA, UK, and Germany. The median RCT sample size was 78 patients and the median follow-up 35.7 weeks. Fifty-two percent of all RCTs were of good, 37% of moderate, and 11% of bad quality, with an improvement over time. RCTs with financial funding and RCTs with a sample size of >78 patients were of significantly better quality. There were no major differences in the rating of the studies between the two investigators.

CONCLUSIONS

Only a fraction of neurosurgically relevant literature consists of RCTs, but the quality is satisfying and has significantly improved over the last years. An adequate sample size and sufficient financial support seem to be of substantial importance with regard to the quality of the study. Our data also show that by using a standardized checklist, the quality of trials can be reliably assessed by observers of different experience and educational levels.

Experience with an online prospective database on adolescent idiopathic scoliosis: development and implementation

Considerable variability exists in the surgical treatment and outcomes of adolescent idiopathic scoliosis (AIS). This is due to the lack of evidence-based treatment guidelines and outcome measures. Although clinical trials have been extolled as the highest form of evidence for evaluating treatment efficacy, the disadvantage of cost, time, lack of feasibility, and ethical considerations indicate a need for a new paradigm for evidence based research in this spinal deformity. High quality clinical databases offer an alternative approach for evidence-based research in medicine. So, we developed and established Scolisoft, an international, multidimensional and relational database designed to be a repository of surgical cases for AIS, and an active vehicle for standardized surgical information in a format that would permit qualitative and quantitative research and analysis. Here, we describe and discuss the utility of Scolisoft as a new paradigm for evidence-based research on AIS. Scolisoft was developed using dot.net platform and SQL server from Microsoft. All data is deidentified to protect patient privacy. Scolisoft can be accessed at (www.scolisoft.org). Collection of high quality data on surgical cases of AIS is a priority and processes continue to improve the database quality. The database currently has 67 registered users from 21 countries. To date, Scolisoft has 200 detailed surgical cases with pre, post, and follow up data. Scolisoft provides a structured process and practical information for surgeons to benchmark their treatment methods against other like treatments. Scolisoft is multifaceted and its use extends to education of health care providers in training, patients, ability to mine important data to stimulate research and quality improvement initiatives of healthcare organizations.

Randomized controlled trials in plastic surgery: a 20-year review of reporting standards, methodologic quality, and impact

BACKGROUND

Randomized controlled trials in plastic surgery have not been analyzed comprehensively. We analyzed plastic surgical randomized controlled trials with respect to reporting standards, methodologic quality, and impact on the specialty.

METHODS

Randomized controlled trials published from 1986 to 2006 in three major plastic surgery journals were scored for quality and impact using the Consolidated Standards of Reporting Trials checklist, the Jadad criteria, citation numbers, and other parameters. The associations between the quality scores and multiple independent parameters, including trial impact, were explored. The relative impact of randomized controlled trials in plastic surgery was compared with that in other specialties.

RESULTS

A total of 163 randomized controlled trials were evaluated. The average Consolidated Standards of Reporting Trials and Jadad scores were 49 percent and 2.3, respectively. There were deficiencies in the reporting of parameters that influence bias and statistical significance. Randomized controlled trials with high impact or high methodologic quality had higher reporting scores. However, the quality and impact scores did not correlate with the number of participants, subject category, country of origin, or year or journal of publication. Nonsurgical trials had significantly higher quality and impact than surgical trials. Randomized controlled trials in plastic surgery had relatively lower impact as compared with randomized controlled trials in other specialties.

CONCLUSIONS

The reporting and methodologic standards of randomized controlled trials in plastic surgery need improvement. Standards could be improved if well-accepted reporting and methodologic criteria are considered when designing and evaluating randomized controlled trials. Instituting higher standards may improve the impact of randomized controlled trials and make them more influential in plastic surgery.

The role of evidence-based medicine in neurosurgery

Evidence-based medicine (EBM) has become one of the pillars of modern medicine. There are many myths and misconceptions about EBM that retard its application into neurosurgical practice. We aim to dispel some of these. Neurosurgeons should cultivate sound EBM practice and strive to improve the available evidence base for neurosurgery. Although randomised controlled trials are not suitable to investigate many neurosurgical problems, this does not preclude practising EBM that requires consideration of the strongest available evidence at the time of clinical decision making. Although individualised analysis of primary sources of evidence gives the clearest analysis, the diversity and complexity of modern medicine means that it is difficult for clinicians to stay abreast of all EBM through this mechanism alone. The development of evidence-based practice guidelines by neurosurgeons is a practical alternative. We recommend that neurosurgical organisations and societies take up this challenge and develop EBM education programs, encouraging the practise of EBM by neurosurgeons and neurosurgical trainees.

A quality assessment of randomized clinical trials in pediatric orthopaedics

The promotion and practice of evidence-based medicine necessitates a critical evaluation of medical literature, including the criterion standard of randomized clinical trials (RCTs). Recent studies have examined the quality of RCTs in various surgical specialties, but no study has focused on pediatric orthopaedics. The purpose of this study was to assess and describe the quality of RCTs published in the last 10 years in journals with high clinical impact in pediatric orthopaedics. All of the RCTs in pediatric orthopaedics published in 5 well-recognized journals between 1995 and 2005 were reviewed using the Detsky Quality Assessment Scale. The mean percentage score on the Detsky scale was 53% (95% confidence interval, 46%-60%). Only 7 (19%) of the articles satisfied the threshold for a satisfactory level of methodological quality (Detsky >75%). Most RCTs in pediatric orthopaedics that are published in well-recognized peer-reviewed journals demonstrate substantial deficiencies in methodological quality. Particular areas of weakness include inadequate rigor and reporting of randomization methods, use of inappropriate or poorly described outcome measures, inadequate description of inclusion and exclusion criteria, and inappropriate statistical analysis. Further efforts are necessary to improve the conduct and reporting of clinical trials in this field to avoid inadvertent misinformation of the clinical community.

Birth after caesarean study--planned vaginal birth or planned elective repeat caesarean for women at term with a single previous caesarean birth: protocol for a patient preference study and randomised trial

BACKGROUND

For women who have a caesarean section in their preceding pregnancy, two care policies for birth are considered standard: planned vaginal birth and planned elective repeat caesarean. Currently available information about the benefits and harms of both forms of care are derived from retrospective and prospective cohort studies. There have been no randomised trials, and recognising the deficiencies in the literature, there have been calls for methodologically rigorous studies to assess maternal and infant health outcomes associated with both care policies. The aims of our study are to assess in women with a previous caesarean birth, who are eligible in the subsequent pregnancy for a vaginal birth, whether a policy of planned vaginal birth after caesarean compared with a policy of planned repeat caesarean affects the risk of serious complications for the woman and her infant.

METHODS/DESIGN

DESIGN

Multicentered patient preference study and a randomised clinical trial.

INCLUSION CRITERIA

Women with a single prior caesarean presenting in their next pregnancy with a single, live fetus in cephalic presentation, who have reached 37 weeks gestation, and who do not have a contraindication to a planned VBAC. Trial Entry & Randomisation: Eligible women will be given an information sheet during pregnancy, and will be recruited to the study from 37 weeks gestation after an obstetrician has confirmed eligibility for a planned vaginal birth. Written informed consent will be obtained. Women who consent to the patient preference study will be allocated their preference for either planned VBAC or planned, elective repeat caesarean. Women who consent to the randomised trial will be randomly allocated to either the planned vaginal birth after caesarean or planned elective repeat caesarean group. Treatment Groups: Women in the planned vaginal birth group will await spontaneous onset of labour whilst appropriate. Women in the elective repeat caesarean group will have this scheduled for between 38 and 40 weeks. Primary Study Outcome: Serious adverse infant outcome (death or serious morbidity).

SAMPLE SIZE

2314 women in the patient preference study to show a difference in adverse neonatal outcome from 1.6% to 3.6% (p = 0.05, 80% power).

Evidence-Based Practice in Oral and Maxillofacial Surgery: Audit of 1 Training Center

PURPOSE

To evaluate the proportion of evidence-based interventions in the field of oral and maxillofacial surgery in a regional training center.

PATIENTS AND METHODS

A prospective clinical audit was carried out within the discipline of Oral and Maxillofacial Surgery, University of Hong Kong in February 2004 for a period of 6 months to investigate the extent of evidence-based practice. Consecutive diagnosis and intervention pairs were identified and recorded through standardized charts in randomly selected clinical sessions. A corresponding literature search using Medline and the Cochrane Library was performed to identify best current evidence. Each pair was then analyzed and graded according to the best current evidence.

RESULTS

Of 500 cases, 273 were eligible for evaluation while the rest were excluded based on 4 defined exclusion criteria. A majority of interventions (n = 195, 71.4%) were found to be evidence-based. Seventy-eight (28.6%) interventions were found to be not evidence-based. Among the evidence, a majority (56.1%) was level 5 evidence, which are case series or systematic review/meta-analysis of case series, and 36% were level 3 or above, which are randomized control trial (RCT) (level 3), meta-analysis of RCTs (level 2), or systematic review of RCTs (level 1). There was no statistically significant difference in the proportion of evidence-based practice between specialists and trainees in oral and maxillofacial surgery who saw and treated patients.

CONCLUSION

This study demonstrated that most interventions prescribed in this oral and maxillofacial surgery training center were evidence-based, and the proportion was comparable with that reported by other specialties.

A Critical Assessment of the Quality of Reporting of Randomized, Controlled Trials in the Urology Literature

PURPOSE

Randomized, controlled trials are the gold standard for evidence based assessment of therapeutic interventions. In 1996 the Consolidated Standards of Reporting Trials statement was published in an effort to standardize the reporting of clinical trials. To our knowledge we report the first systematic assessment of randomized, controlled trial quality in the urology literature by Consolidated Standards of Reporting Trials standards.

MATERIALS AND METHODS

All human subject randomized, controlled trials published in 4 leading urology journals in 1996 and 2004 were identified for formal review. A standardized evaluation form was developed based on the Consolidated Standards of Reporting Trials statement. Each article was evaluated by 2 independent reviewers and discrepancies were settled by consensus. A Consolidated Standards of Reporting Trials criteria summary score was calculated on a scale of 0 to 22.

RESULTS

A total of 152 randomized, controlled trials met inclusion criteria. The mean+/-SEM Consolidated Standards of Reporting Trials summary score was 10.2+/-0.3 (median 10.3) and 12.0+/-0.3 (median 12.2) in 1996 and 2004, respectively, with a mean difference of 1.8 (95% CI 1.0, 2.6; p=0.001). Reporting of important methodological criteria, eg sample size justification and randomization implementation, improved from 1996 to 2004. Improvement notwithstanding, reporting of key methodological criteria remained consistently below 50% in 2004.

CONCLUSIONS

This formal review suggests that randomized, controlled trial reporting in the urology literature has improved since the publication of the Consolidated Standards of Reporting Trials statement in 1996. Certain areas, such as reporting of trial methods, continue to meet Consolidated Standards of Reporting Trials criteria in fewer than half of publications. Ongoing graduate and postgraduate education in trial design and evidence based practice may result in further improvement in randomized, controlled trial reporting.

Will the dilemma of evidence-based surgery ever be resolved?

Exponents of evidence-based medicine do not undermine the importance of clinical expertise and skills, but they emphasize that decision-making in medicine should be based on the best available evidence derived from the systematic analysis of observations made in an objective, unbiased and a reproducible fashion. The randomized controlled trial (RCT) is the most scientifically rigorous means of hypothesis testing in epidemiology. Discrepancies between established surgical and other interventions and best available evidence are common. These can be in the form of significant delay in adopting a new intervention despite strong supportive evidence, adopting an intervention before supportive evidence becomes available for reasons of novelty or pear pressure and the lack of supportive evidence for many established common practices. This is compounded further by the paucity of good quality evidence for most surgical procedures. This is arguably because of the inherent difficulties in conducting surgical RCT. The practical, ethical and financial ramifications are complex and the nature of surgical disease often compromise the chances of success or completion of RCT. Carrying out surgical RCT may have more implications on the clinician's authority, autonomy and income and their results are more likely to be influenced by his/her expertise and competence than medical RCT. Furthermore, the success of surgical RCT is often jeopardized by very low recruitment rates. The aim of this study is to discuss the dilemma of producing evidence in surgery.

Impact of Study Design on Outcome after Endovascular Abdominal Aortic Aneurysm Repair. A Comparison between the Randomized Controlled DREAM-trial and the Observational EUROSTAR-registry

BACKGROUND

Patients with abdominal aortic aneurysm (AAA) can be treated by transfemoral endovascular intervention and by conventional open surgery. Level-one evidence of the safety and efficacy of one treatment mode over the other is only provided by a randomised controlled trial (RCT). Results reported by voluntary registries are considered less valid than data from RCTs. On the other hand the outcome of a RCT may not be generalisable to the common practice because of vigorous selection of patients and institutions.

PURPOSE

The outcomes reported by the Dutch Randomized Endovascular Aneurysm Management (DREAM) trial were compared with the results of the EURopean collaborators on Stent-graft techniques for AAA Repair (EUROSTAR) registry.

METHODS

To obtain comparable study groups with regard to risk factors equal proportions of ASA I, II and III patients as observed in the endovascular arm of the DREAMtrial were selected at random from the EUROSTAR-registry. All patients had an aneurysm of at least 50mm. Only patients, who had been enrolled into the registry from 1999, were selected to avoid the influence of first generation endografts which are not longer in use. Patient characteristics and outcomes of endovascular AAA repair (EVAR) of EUROSTAR and DREAM-trial participants were compared. Differences in early findings between study groups were assessed by Chi-Square tests for discrete variables and by Wilcoxon rank sum tests for continuous variables. Follow-up variables were analysed by Kaplan-Meier and Cox proportional hazard models.

RESULTS

Data of 177 patients of the DREAM trial with randomization to EVAR and 856 patients selected in the EUROSTAR-registry were compared. Baseline characteristics were comparable between the EUROSTAR-cohort and EVAR-arm of the DREAM-trial. The 36-month survival-rate was 87.6% for EVAR-arm in the DREAM-trial similar to the 86.8% found in this EUROSTAR-study population. The freedom of secondary procedures reached after 3 years 85.7%, and 86.9% in the DREAM and EUROSTAR-cohort, respectively.

CONCLUSION

We found comparable characteristics and outcomes between patients of comparable risk class of the EUROSTAR-registry and the EVAR-cohort of the DREAM-trial. This demonstrates the following: first, the EUROSTAR-data provide reliable information, and further comparisons of registry data with patients treated by conventional AAA surgery may be justified. Secondly, the various outcomes of the randomised DREAM trial appear generalisable, as it agrees with observations in a broad common practice derived database.

Does it matter if clinicians recruiting for a trial don't understand what the trial is really about? Qualitative study of surgeons' experiences of participation in a pragmatic multi-centre RCT

Background

Qualitative methods are increasingly used to study the process of clinical trials and patients understanding of the rationale for trials, randomisation and reasons for taking part or refusing. Patients' understandings are inevitably influenced by the recruiting clinician's understanding of the trial, yet relatively little qualitative work has explored clinicians' perceptions and understandings of trials. This study interviewed surgeons shortly after the multi-centre, pragmatic RCT in which they had participated had been completed.

Methods

We used in-depth interviews with surgeons who participated in the Spine Stabilisation Trial (a pragmatic RCT) to explore their understanding of the trial purpose and how this understanding had influenced their recruitment procedures and interpretation of the results. A purposive sample of eleven participating surgeons was chosen from 8 of the 15 UK trial centres.

Results

Although the surgeons thought that the trial was addressing an important question there was little agreement about what this question was: although it was a trial of 'equivalent' treatments, some thought that it was a trial of surgery, others a trial of rehabilitation and others that it was exploring what to do with patients in whom all other treatment options had been unsuccessful. The surgeons we interviewed were not aware of the rationale for the pragmatic inclusion criteria and nearly all were completely baffled about the meaning of 'equipoise'. Misunderstandings about the entry criteria were an important source of confusion about the results and led to reluctance to apply the results to their own practice.

Conclusion

The study suggests several lessons for the conduct of future multi-centre trials. Recruiting surgeons (and other clinicians) may not be familiar with the rationale for pragmatic designs and may need to be regularly reminded about the purpose during the study. Reassurance may be necessary that a pragmatic design is not considered a design fault. We conclude that it does matter if clinicians do not understand the rationale for the trial if, as we have shown here, their perception of the trial aims and methods adversely affects who they recruit; if their views affect what the patients are told; and if they mistakenly view the results as unscientific, unreliable and ultimately irrelevant to their practice.

Assuring quality in HPB surgery--efficacy and safety

Surgical innovations have made enormous contributions towards the welfare of patients when they have been appropriate, effective and applied with expertise and overall care. However, the potential for advancement and for harm of new surgical techniques, and the level of expertise necessary for their safe introduction, are not always immediately apparent. Furthermore, it is difficult and time-consuming to assess the efficacy and safety of new procedures in the clinical setting. In 1998 the Royal Australasian College of Surgeons established ASERNIP-S, the Australian Safety and Efficacy Register of New and Interventional Procedures-Surgical, to help ensure that new technologies that are being introduced are well proven in concept, are as safe and effective as possible, and are utilized with high levels of skill underpinned by the level of training.

Levels of evidence and journal impact factor in oral and maxillofacial surgery

The aim of this study was to identify the relationship between quality of research, in terms of levels of evidence, and journal impact factor (IF), and to describe the pattern of publications in oral and maxillofacial surgery. All four major journals in this subject area only, and with a published IF, were included in the study. Latest published IF dated 2004 was chosen, and all articles related to its calculation were accessed and classified into four levels of evidence. Correlation between levels of evidence and IF was investigated and the pattern of publications was described. All eligible 932 published articles were analysed. None (0%) were level I evidence, 20 (2%) were Level II, 70 (8%) level III and 337 (40%) level IV; 465 (50%) articles were classified as non-evidence. IF ranged from 0.689 to 1.154. There were statistically significant correlations between levels of evidence and IF (rho=1.0, P<0.01). Among the 465 non-evidence articles, there were 219 (47%) case reports, 91 (20%) animal studies, 52 (11%) laboratory studies, 35 (8%) technical notes, 24 (5%) tutorial articles, and 16 (3%) reviews articles.

The reporting of randomized clinical trials using a surgical intervention is in need of immediate improvement: a systematic review

OBJECTIVE

To assess the reporting of surgical interventions, care providers, and number of centers in randomized clinical trials.

METHODS

Systematic review was performed to assess reports of randomized controlled trials assessing surgical procedure published in 2004. A standardized abstraction form was used to extract data.

RESULTS

A total of 158 articles were included. Details on the intervention intended, such as the surgical procedure, were reported in 138 (87.3%) articles, anesthetic management in 56 (35.4%), preoperative care in 34 (15.2%), and postoperative care in 78 (49.4%). How the experimental surgical intervention was carried out was reported in 64 articles (40.5%). Most trials were conducted in single centers (n = 109, 69.0%). The setting was reported in only 11 articles, and the volume of interventions performed was only reported in 5. Selection criteria were reported for care providers in 64 articles (40.5%). The number of care providers performing the intervention was reported in 51 articles (32.2%). The quality of reporting was low as assessed by CLEAR NPT (a 10-items checklist specifically developed to assess the reporting quality of RCTs assessing nonpharmacologic treatment).

CONCLUSIONS

Inadequate reporting on the management of the surgical procedure, care providers, and surgery center may introduce bias in RCTs of surgical interventions, making their results questionable. We recommend extending the CONSORT Statement to surgical interventions.

Comparison of effects in randomized controlled trials with observational studies in digestive surgery

OBJECTIVES

To compare the results of randomized controlled trials versus observational studies in meta-analyses of digestive surgical topics.

SUMMARY BACKGROUND DATA

While randomized controlled trials have been recognized as providing the highest standard of evidence, claims have been made that observational studies may overestimate treatment benefits. This debate has recently been renewed, particularly with regard to pharmacotherapies.

METHODS

The PubMed (1966 to April 2004), EMBASE (1986 to April 2004) and Cochrane databases (Issue 2, 2004) were searched to identify meta-analyses of randomized controlled trials in digestive surgery. Fifty-two outcomes of 18 topics were identified from 276 original articles (96 randomized trials, 180 observational studies) and included in meta-analyses. All available binary data and study characteristics were extracted and combined separately for randomized and observational studies. In each selected digestive surgical topic, summary odds ratios or relative risks from randomized controlled trials were compared with observational studies using an equivalent calculation method.

RESULTS

Significant between-study heterogeneity was seen more often among observational studies (5 of 12 topics) than among randomized trials (1 of 9 topics). In 4 of the 16 primary outcomes compared (10 of 52 total outcomes), summary estimates of treatment effects showed significant discrepancies between the two designs.

CONCLUSIONS

One fourth of observational studies gave different results than randomized trials, and between-study heterogeneity was more common in observational studies in the field of digestive surgery.

The stepped wedge trial design: a systematic review

BACKGROUND

Stepped wedge randomised trial designs involve sequential roll-out of an intervention to participants (individuals or clusters) over a number of time periods. By the end of the study, all participants will have received the intervention, although the order in which participants receive the intervention is determined at random. The design is particularly relevant where it is predicted that the intervention will do more good than harm (making a parallel design, in which certain participants do not receive the intervention unethical) and/or where, for logistical, practical or financial reasons, it is impossible to deliver the intervention simultaneously to all participants. Stepped wedge designs offer a number of opportunities for data analysis, particularly for modelling the effect of time on the effectiveness of an intervention. This paper presents a review of 12 studies (or protocols) that use (or plan to use) a stepped wedge design. One aim of the review is to highlight the potential for the stepped wedge design, given its infrequent use to date.

METHODS

Comprehensive literature review of studies or protocols using a stepped wedge design. Data were extracted from the studies in three categories for subsequent consideration: study information (epidemiology, intervention, number of participants), reasons for using a stepped wedge design and methods of data analysis.

RESULTS

The 12 studies included in this review describe evaluations of a wide range of interventions, across different diseases in different settings. However the stepped wedge design appears to have found a niche for evaluating interventions in developing countries, specifically those concerned with HIV. There were few consistent motivations for employing a stepped wedge design or methods of data analysis across studies. The methodological descriptions of stepped wedge studies, including methods of randomisation, sample size calculations and methods of analysis, are not always complete.

CONCLUSION

While the stepped wedge design offers a number of opportunities for use in future evaluations, a more consistent approach to reporting and data analysis is required.