Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation

Laparoscopic surgery vs laparotomy for early stage endometrial cancer: long-term data of a randomized controlled trial

OBJECTIVE

The purpose of the study was to compare the long-term safety and efficacy of laparoscopic surgery and laparotomy approaches to early stage endometrial cancer.

STUDY DESIGN

This was a prospective long-term extension study of a randomized controlled study that included 84 patients with clinical stage I endometrial cancer (laparoscopic surgery group, 40 women; laparotomy group, 38 women). Safety and efficacy data were evaluated and analyzed by the intention-to-treat principle.

RESULTS

After a follow-up period of 78 months (interquartile range, 7; range, 19-84 months) and 79 months (interquartile range, 6; range, 22-84 months) for laparoscopic surgery and laparotomy groups, respectively, no difference in the cumulative recurrence rates (8/40 [20.0%] and 7/38 [18.4%]; P = .860) and deaths (7/40 [17.5%] and 6/38 [15.8%] patients; P = .839) was detected between groups. No significant differences in overall (P = .535) and disease-free (P = .512) survival were observed.

CONCLUSION

The laparoscopic surgery approach to early stage endometrial cancer is as safe and effective a procedure as the laparotomy approach.

Evidence-based nutrition support in the intensive care unit: an update on reported trial quality

PURPOSE OF REVIEW

Meta-epidemiological reviews report that trials of nutritional support in critical illness rarely fulfil basic quality requirements, with overall quality rated as 'worse than poor'. This update reviews recently published trials to determine whether current evidence meets or exceeds basic quality requirements.

RECENT FINDINGS

Although recent trials were significantly more likely to report blinding, there is a concerning trend towards a decrease in overall trial quality. Many recent trials fail to report the use of 'any' of three key validity criteria: use of blinding, presentation of intention-to-treat analysis and the maintenance allocation concealment.

SUMMARY

Future researchers must improve the quality with which trials are conducted and reported. Submitting a clinical trial to an approved registry prior to enrolling patients provides transparency of conduct. Investigators must ensure that an intention-to-treat analysis is reported, especially when a subset efficacy analysis is presented, even if the intention-to-treat analysis requires imputing missing data values. Investigators also need to improve reporting details concerning allocation concealment and blinding. Finally, until investigators, editors and reviewers embrace these measures, we strongly recommend that readers should become familiar with the appropriate evidence-based medicine users' guides so that they can base clinical decisions on valid studies.

Nonrandomized quality improvement intervention trials might overstate the strength of causal inference of their findings

OBJECTIVE

To assess the strength of causal inferences reported in randomized and nonrandomized evaluations of quality improvement (QI) interventions in relation to the study design and the direction of results for the primary outcomes.

STUDY DESIGN AND SETTING

We searched 11 journals for QI intervention studies that aimed to change clinician behavior. Statements that addressed the causal inference between intervention and outcomes were extracted and were rated by 34 researchers for the strength of causality.

RESULTS

We found 38 randomized controlled trials (RCTs) and 35 non-RCTs, and extracted 68 quotes from the abstracts and 139 from the main text. A significant interaction was found between study design and direction of results for the abstract quotes (P=0.022). The ratings for non-RCTs were higher when the results were mixed, but for RCTs, they were higher if the results were positive or no effect, although none of the differences were statistically significant at alpha=0.05 after adjusting for multiple comparisons. For the main text quotes, the causality rating was higher by 0.43 for RCTs than for non-RCTs after adjusting for the direction of results (P<0.001).

CONCLUSION

Authors might have overstated the strength of causal inference in the abstracts of non-RCTs, but appeared to report causality appropriately in the main text.

A randomised controlled trial of the Neuro Emotional Technique (NET) for childhood Attention Deficit Hyperactivity Disorder (ADHD): a protocol

BACKGROUND

An abundance of literature is dedicated to research for the treatment of Attention Deficit Hyperactivity Disorder (ADHD). Most, is in the area of pharmacological therapies with less emphasis in psychotherapy and psychosocial interventions and even less in the area of complementary and alternative medicine (CAM).The use of CAM has increased over the years, especially for developmental and behavioral disorders, such as ADHD. 60-65% of parents with children with ADHD have used CAM. Medical evidence supports a multidisciplinary approach (i.e. pharmacological and psychosocial) for the best clinical outcomes. The Neuro Emotional Technique (NET), a branch of Chiropractic, was designed to address the biopsychosocial aspects of acute and chronic conditions including non-musculoskeletal conditions. Anecdotally, it has been suggested that ADHD may be managed effectively by NET.

DESIGN/METHODS

A placebo controlled, double blind randomised clinical trial was designed to assess the effectiveness of NET on a cohort of children with medically diagnosed ADHD. Children aged 5-12 years who met the inclusion criteria were randomised to one of three groups. The control group continued on their existing medical regimen and the intervention and placebo groups had the addition of the NET and sham NET protocols added to their regimen respectively. These two groups attended a clinical facility twice a week for the first month and then once a month for six months. The Conners' Parent and Teacher Rating Scales (CRS) were used at the start of the study to establish baseline data and then in one month and in seven months time, at the conclusion of the study. The primary outcome measures chosen were the Conners' ADHD Index and Conners' Global Index. The secondary outcome measures chosen were the DSM-IV: Inattentive, the DSM-IV:Hyperactive-Impulsive, and the DSM-IV:Total subscales from the Conners' Rating Scales, monitoring changes in inattention, hyperactivity and impulsivity. Calculations for the sample size were set with a significance level of 0.05 and the power of 80%, yielding a sample size of 93.

DISCUSSION

The present study should provide information as to whether the addition of NET to an existing medical regimen can improve outcomes for children with ADHD.

TRIAL REGISTRATION

Australian New Zealand Clinical Trial Registration Number: ANZCTRN 012606000332527.

Searching for unpublished trials in Cochrane reviews may not be worth the effort

OBJECTIVE

To assess the value of searching for unpublished data by exploring the extent to which Cochrane reviews include unpublished data and by evaluating the quality of unpublished trials.

STUDY DESIGN AND SETTING

We screened all 2,462 completed Cochrane reviews published since 2000 in the Cochrane Database of Systematic Reviews Issue 3, 2006. In a random sample (n=61) of 292 reviews, including unpublished trials, we studied all 116 references.

RESULTS

Unpublished trials make up 8.8% of all included trials in our sample. Thirty-eight percent of the "unpublished" trials have in fact been published. Allocation concealment was "unclear" or not adequate in 54.3% and 61.3% reported blinding. In 47.2% reported withdrawal rates were >20%. Trials that were eventually published had larger mean population sizes (P-value, 0.02). Of the reported sponsors, 87.3% were drug companies. Methodological quality and publication bias are mentioned in half of the reviews and explored in a third. Quality ratings did not have consequences for pooling, because 82.8% was included in the forest plots.

CONCLUSIONS

A minority of Cochrane reviews include "unpublished trials" and many of these are eventually published. Truly unpublished studies have poor or unclear methodological quality. Therefore, it may be better to invest in regular updating of reviews, rather than in extensive searching for unpublished data.

The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry

To determine whether the CONSORT recommendations influenced the quality of reporting of randomized controlled trials (RCTs) in the field of psychiatry, we evaluated the quality of clinical trial reports before and after the introduction of CONSORT statement. We selected seven high impact journals and retrieved the randomized, clinical trials in the field of psychiatry during the period of 1992-1996 (pre-CONSORT) and 2002-2007 (post-CONSORT). Among the total 5201 articles screened, 736 were identified and entered in our database. After critical review of the publications, 442 articles met the inclusion and exclusion criteria. The CONSORT Index (sum of 22 items of the checklist) during the post-CONSORT period was significantly higher than that during the pre-CONSORT period. However, over 40% of post-CONSORT studies did not adhere to CONSORT statement for reporting the process of randomization, and details of the process for obtaining informed consent were still insufficient. Furthermore, adherence to the CONSORT guidelines of reporting how blinding was accomplished and evaluated actually decreased after publication of the CONSORT statement. Although the overall quality of reporting on psychiatric RCTs generally improved after publication of the CONSORT statement, reporting the details of randomization, blinding, and obtaining informed consent remain insufficient.

A systematic review of trends in the methodological quality of randomized controlled trials in various research fields

OBJECTIVES

We sought to evaluate the trends in the methodological quality of randomized controlled trials in various medical fields.

STUDY DESIGN AND SETTING

Relevant studies were retrieved by the PubMed and the ISI Web of science databases.

RESULTS

Thirty-five out of 457 retrieved studies met the inclusion criteria. Twenty-one out of 35 selected studies reported significant improvement in at least one methodological quality factor. Overall quality scores were increased in 13 out of 26 studies providing relevant data. The most commonly separately examined key quality factors were allocation concealment and blinding in 13 out of 21 studies that reported relevant data. Allocation concealment was the quality characteristic most commonly reported as significantly improving during the reviewed period (in five out of eight studies reporting relevant comparative data).

CONCLUSION

Certain aspects of methodological quality have improved significantly over time, but others remain stagnant. Further efforts to improve study design, conduct, and reporting of randomized controlled trials are warranted.

Randomized controlled trials in plastic surgery: a 20-year review of reporting standards, methodologic quality, and impact

BACKGROUND

Randomized controlled trials in plastic surgery have not been analyzed comprehensively. We analyzed plastic surgical randomized controlled trials with respect to reporting standards, methodologic quality, and impact on the specialty.

METHODS

Randomized controlled trials published from 1986 to 2006 in three major plastic surgery journals were scored for quality and impact using the Consolidated Standards of Reporting Trials checklist, the Jadad criteria, citation numbers, and other parameters. The associations between the quality scores and multiple independent parameters, including trial impact, were explored. The relative impact of randomized controlled trials in plastic surgery was compared with that in other specialties.

RESULTS

A total of 163 randomized controlled trials were evaluated. The average Consolidated Standards of Reporting Trials and Jadad scores were 49 percent and 2.3, respectively. There were deficiencies in the reporting of parameters that influence bias and statistical significance. Randomized controlled trials with high impact or high methodologic quality had higher reporting scores. However, the quality and impact scores did not correlate with the number of participants, subject category, country of origin, or year or journal of publication. Nonsurgical trials had significantly higher quality and impact than surgical trials. Randomized controlled trials in plastic surgery had relatively lower impact as compared with randomized controlled trials in other specialties.

CONCLUSIONS

The reporting and methodologic standards of randomized controlled trials in plastic surgery need improvement. Standards could be improved if well-accepted reporting and methodologic criteria are considered when designing and evaluating randomized controlled trials. Instituting higher standards may improve the impact of randomized controlled trials and make them more influential in plastic surgery.

Quality of reporting of randomized controlled trials in general endocrinology literature

CONTEXT

The reporting quality of randomized controlled trials (RCTs) is poor in general medicine and several areas of specialization but unknown in endocrinology.

OBJECTIVE

Our aim was to assess the reporting quality of RCTs in general endocrinology. A secondary objective was to identify predictors for better reporting quality.

DESIGN AND SETTING

We systematically reviewed RCTs published in three general endocrinology journals between January 2005 and December 2006.

PARTICIPANTS

We included parallel-design RCTs that addressed a question of treatment or prevention. Article selection and data abstraction were conducted by two reviewers independently, and disagreements were resolved by consensus.

MAIN OUTCOMES

There were two main outcomes: 1) a 15-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials (CONSORT); and 2) a 3-point key score, based on allocation concealment, blinding, and use of intention-to-treat analysis.

RESULTS

Eighty nine RCTs were included. The median OQS was 10 (interquartile range = 2). Allocation concealment, blinding, and analysis by intention to treat were reported in 10, 20, and 16 of the 89 RCTs, respectively. A multivariable regression analysis showed that complete industrial funding [incidence rate ratio (IRR) = 1.014; 95% confidence interval (CI), 1.010-1.018], journal of publication (IRR = 1.068; 95% CI, 1.007-1.132), and sample size (IRR = 1.048; 95% CI, 1.026-1.070) were significantly associated with a slightly better OQS.

CONCLUSIONS

The quality of RCT reporting in general endocrine literature is suboptimal. We discuss our results, highlight the areas where improvements are needed, and provide some recommendations.

Enhancing access to reports of randomized trials published world-wide--the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library

Background

Randomized trials are essential in assessing the effects of healthcare interventions and are a key component in systematic reviews of effectiveness. Searching for reports of randomized trials in databases is problematic due to the absence of appropriate indexing terms until the 1990s and inconsistent application of these indexing terms thereafter.

Objectives

The objectives of this study are to devise a search strategy for identifying reports of randomized trials in EMBASE which are not already indexed as trials in MEDLINE and to make these reports easily accessible by including them in the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library, with the permission of Elsevier, the publishers of EMBASE.

Methods

A highly sensitive search strategy was designed for EMBASE based on free-text and thesaurus terms which occurred frequently in the titles, abstracts, EMTREE terms (or some combination of these) of reports of trials indexed in EMBASE. This search strategy was run against EMBASE from 1980 to 2005 (1974 to 2005 for four of the terms) and records retrieved by the search, which were not already indexed as randomized trials in MEDLINE, were downloaded from EMBASE, printed and read. An analysis of the language of publication was conducted for the reports of trials published in 2005 (the most recent year completed at the time of this study).

Results

Twenty-two search terms were used (including nine which were later rejected due to poor cumulative precision). More than a third of a million records were downloaded and scanned and approximately 80,000 reports of trials were identified which were not already indexed as randomized trials in MEDLINE. These are now easily identifiable in CENTRAL, in The Cochrane Library. Cumulative sensitivity ranged from 0.1% to 60% and cumulative precision ranged from 8% to 61%. The truncated term 'random$' identified 60% of the total number of reports of trials but only 35% of the more than 130,000 records retrieved by this term were reports of trials. The language analysis for the sample year 2005 indicated that of the 18,427 reports indexed as randomized trials in MEDLINE, 959 (5%) were in languages other than English. The EMBASE search identified an additional 658 reports in languages other than English, of which the highest number were in Chinese (320).

Conclusion

The results of the search to date have greatly increased access to reports of trials in EMBASE, especially in some languages other than English. The search strategy used was subjectively derived from a small 'gold standard' set of test records and was not validated in an independent test set. We intend to design an objectively-derived validated search strategy using logistic regression based on the frequency of occurrence of terms in the approximately 80,000 reports of randomized trials identified compared with the frequency of these terms across the entire EMBASE database.

Publication of clinical trials supporting successful new drug applications: a literature analysis

BACKGROUND

The United States (US) Food and Drug Administration (FDA) approves new drugs based on sponsor-submitted clinical trials. The publication status of these trials in the medical literature and factors associated with publication have not been evaluated. We sought to determine the proportion of trials submitted to the FDA in support of newly approved drugs that are published in biomedical journals that a typical clinician, consumer, or policy maker living in the US would reasonably search.

METHODS AND FINDINGS

We conducted a cohort study of trials supporting new drugs approved between 1998 and 2000, as described in FDA medical and statistical review documents and the FDA approved drug label. We determined publication status and time from approval to full publication in the medical literature at 2 and 5 y by searching PubMed and other databases through 01 August 2006. We then evaluated trial characteristics associated with publication. We identified 909 trials supporting 90 approved drugs in the FDA reviews, of which 43% (394/909) were published. Among the subset of trials described in the FDA-approved drug label and classified as "pivotal trials" for our analysis, 76% (257/340) were published. In multivariable logistic regression for all trials 5 y postapproval, likelihood of publication correlated with statistically significant results (odds ratio [OR] 3.03, 95% confidence interval [CI] 1.78-5.17); larger sample sizes (OR 1.33 per 2-fold increase in sample size, 95% CI 1.17-1.52); and pivotal status (OR 5.31, 95% CI 3.30-8.55). In multivariable logistic regression for only the pivotal trials 5 y postapproval, likelihood of publication correlated with statistically significant results (OR 2.96, 95% CI 1.24-7.06) and larger sample sizes (OR 1.47 per 2-fold increase in sample size, 95% CI 1.15-1.88). Statistically significant results and larger sample sizes were also predictive of publication at 2 y postapproval and in multivariable Cox proportional models for all trials and the subset of pivotal trials.

CONCLUSIONS

Over half of all supporting trials for FDA-approved drugs remained unpublished >/= 5 y after approval. Pivotal trials and trials with statistically significant results and larger sample sizes are more likely to be published. Selective reporting of trial results exists for commonly marketed drugs. Our data provide a baseline for evaluating publication bias as the new FDA Amendments Act comes into force mandating basic results reporting of clinical trials.

Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication--before-and-after study

PURPOSE

To determine the quality of reporting of diagnostic accuracy studies before and after the Standards for Reporting of Diagnostic Accuracy (STARD) statement publication and to determine whether there is a difference in the quality of reporting by comparing STARD (endorsing) and non-STARD (nonendorsing) journals.

MATERIALS AND METHODS

Diagnostic accuracy studies were identified by hand searching six STARD and six non-STARD journals for 2001, 2002, 2004, and 2005. Diagnostic accuracy studies (n = 240) were assessed by using a checklist of 13 of 25 STARD items. The change in the mean total score on the modified STARD checklist was evaluated with analysis of covariance. The change in proportion of times that each individual STARD item was reported before and after STARD statement publication was evaluated (chi(2) tests for linear trend).

RESULTS

With mean total score as dependent factor, analysis of covariance showed that the interaction between the two independent factors (STARD or non-STARD journal and year of publication) was not significant (F = 0.664, df = 3, partial eta(2) = 0.009, P = .58). Additionally, the frequency with which individual items on the STARD checklist were reported before and after STARD statement publication has remained relatively constant, with little difference between STARD and non-STARD journals.

CONCLUSION

After publication of the STARD statement in 2003, the quality of reporting of diagnostic accuracy studies remained similar to pre-STARD statement publication levels, and there was no meaningful difference (ie, one additional item on the checklist of 13 of 25 STARD items being reported) in the quality of reporting between those journals that published the STARD statement and those that did not.

Tracking of childhood overweight into adulthood: a systematic review of the literature

Overweight and obesity in youth are important public health concerns and are of particular interest because of possible long-term associations with adult weight status and morbidity. The aim of this study was to systematically review the literature and update evidence concerning persistence of childhood overweight. A computerized bibliographical search--restricted to studies with a prospective or retrospective longitudinal design--was conducted. Two authors independently extracted data and assessed the methodological quality of the included studies in four dimensions (i) study population and participation rate; (ii) study attrition; (iii) data collection and (iv) data analysis. Conclusions were based on a rating system of three levels of evidence. A total of 25 publications were selected for inclusion in this review. According to a methodological quality assessment, 13 studies were considered to be of high quality. The majority of these high-quality studies were published after 2001, indicating that recently published data, in particular, provide us with reliable information. All included studies consistently report an increased risk of overweight and obese youth becoming overweight adults, suggesting that the likelihood of persistence of overweight into adulthood is moderate for overweight and obese youth. However, predictive values varied considerably. Limiting aspects with respect to generalizability and methodological issues are discussed.

Effects of protein versus simple sugar intake on weight loss in polycystic ovary syndrome (according to the National Institutes of Health criteria)

OBJECTIVE

To compare the effects of protein vs. simple sugars on weight loss, body composition, and metabolic and endocrine parameters in polycystic ovary syndrome (PCOS).

DESIGN

A 2-month, free-living, randomized, single-blinded study.

SETTING

University PCOS clinic.

PATIENT(S)

Thirty-three patients with PCOS.

INTERVENTION(S)

To achieve a final energy reduction of 450 kcal/day, first the daily energy intake was reduced by 700 kcal; then a 240-kcal supplement containing either whey protein or simple sugars was added.

MAIN OUTCOME MEASURE(S)

Changes in weight, fat mass, fasting glucose and insulin, plasma lipoproteins, and sex steroids.

RESULT(S)

Twenty-four subjects (13 in the simple sugars group and 11 in the protein group) completed the study. The protein group lost more weight (-3.3 +/- 0.8 kg vs. -1.1 +/- 0.6 kg) and more fat mass (-3.1 +/- 0.9 kg vs. -0.5 +/- 0.6 kg) and had larger decreases in serum cholesterol (-33.0 +/- 8.4 mg/dL vs. -2.3 +/- 6.8 mg/dL), high-density lipoprotein cholesterol (-4.5 +/- 1.3 mg/dL vs. -0.4 +/- 1.3 mg/dL), and apoprotein B (-20 +/- 5 mg/dL vs. 3 +/- 5 mg/dL).

CONCLUSION(S)

In patients with PCOS, a hypocaloric diet supplemented with protein reduced body weight, fat mass, serum cholesterol, and apoprotein B more than the diet supplemented with simple sugars.

Survival End Point Reporting in Randomized Cancer Clinical Trials: A Review of Major Journals

Purpose

Several publications showed that the standards for reporting randomized clinical trials (RCTs) might not be entirely suitable. Our aim was to evaluate the reporting of survival end points in cancer RCTs.

Methods

A search in MEDLINE databases identified 274 cancer RCTs published in 2004 in four general medical journals and four clinical oncology journals. Eligible articles were those that reported primary analyses of RCT with survival end points. Methodologists reviewed and scored the articles according to seven key points: prevalence of complete definition of survival end points (time of origin, survival events, censoring events) and relevant information about their analyses (estimation or effect size, precision, number of events, patients at risk). Concordance of key points was evaluated from a random subsample.

Results

After screening, 125 articles were selected; 104 trials were phase III (83%) and 98 publications (78%) were obtained from oncology journals. Among these RCTs, a total of 267 survival end points were recorded, and overall survival (OS) was the most frequent outcome (118 terms, 44%). Survival terms were totally defined for 113 end points (42%) in 65 articles (52%). Accurate information about analysis was retrieved for 73 end points (27%) in 40 articles (32%). The less well-defined information was the number of patients at risk (55%). The reliability was good (κ = 0.72). Finally, according to the key points, optimal reporting was found in 33 end points (12%) or 10 publications.

Conclusion

A majority of articles failed to provide a complete reporting of survival end points, thus adding another source of uncontrolled variability.

The Completeness of Reporting (CORE) index identifies important deficiencies in observational study conference abstracts

OBJECTIVE

The aim was to develop, test, and apply an index to assess the completeness of reporting in a cohort of observational studies of conference abstracts.

STUDY DESIGN AND SETTING

Using rigorous methods, we reduced 245 items generated by literature review to 48 candidate items. In a random sample of 30 conference abstracts of rituximab for nonHodgkin lymphoma, we developed an item impact score using a survey of abstract stakeholders combined with the prevalence of each of the 48 items. We retained 14 independent items representing completeness of reporting, the CORE-14. Two raters determined the reliability of the instrument. We then applied the CORE-14 in another 78 studies to determine the prevalence of each feature.

RESULTS

Our survey response rate was 83.9% (47/56). Interrater reliability (95% CI) of the CORE-14 instrument was 0.56 (0.25, 0.77), which improved by averaging across scores provided by two raters (0.72 [0.49, 0.86]). Applying the CORE-14 in an additional set of 78 abstracts, six items occurred > or =85% and four items occurred < or =40% of the time.

CONCLUSION

Opportunities to improve conference abstract reporting exist. This scale could guide future conference abstract submissions and aid individuals considering conference abstract data to inform clinical practice, systematic reviews, guidelines, or policy.

Health-related quality of life assessment and reported outcomes in leukaemia randomised controlled trials - a systematic review to evaluate the added value in supporting clinical decision making

Health-related quality of life (HRQOL) is increasingly reported as an important outcome in cancer clinical trials. However, very little evidence exists on the impact of such evaluation in randomised controlled trials (RCTs) of leukaemia patients. A systematic search of the literature from 1980 to 2007 was undertaken and studies were identified and evaluated independently, according to a pre-defined coding scheme, by three reviewers. Both HRQOL outcomes and traditional clinical reported outcomes were systematically analysed to evaluate their consistency and their relevance for supporting clinical decision making. Nine RCTs were identified, involving 3838 patients overall. There were four RCTs involving acute myeloid leukaemia patients (AML), three with chronic myeloid leukaemia (CML) and two with chronic lymphocytic leukaemia (CLL). Six studies were published after 2000 and provided fairly robust methodological quality. Imatinib greatly improved HRQOL compared to interferon based treatments in CML patients and fludarabine plus cyclophosphamide does not seem to have a deleterious impact on patient's HRQOL when compared to fludarabine alone or chlorambucil in CLL patients. This study revealed the paucity of HRQOL research in leukaemia patients. Nonetheless, HRQOL assessment is feasible in RCTs and has the great potential of providing valuable outcomes to further support clinical decision making.

Quality of surgical randomized controlled trials for acute cholecystitis: assessment based on CONSORT and additional check items

BACKGROUND/PURPOSE

In this study, we conducted a limited survey of reports of surgical randomized controlled trials, using the consolidated standards of reporting trials (CONSORT) statement and additional check items to clarify problems in the evaluation of surgical reports.

METHODS

A total of 13 randomized trials were selected from two latest review articles on biliary surgery. Each randomized trial was evaluated according to 28 quality measures that comprised items from the CONSORT statement plus additional items. Analysis focused on relationships between the quality of each study and the estimated effect gap ("pooled estimate in meta-analysis" -- "estimated effect of each study").

RESULTS

No definite relationships were found between individual study quality and the estimated effect gap. The following items could have been described but were not provided in almost all the surgical RCT reports: "clearly defined outcomes"; "details of randomization"; "participant flow charts"; "intention-to-treat analysis"; "ancillary analyses"; and "financial conflicts of interest". The item, "participation of a trial methodologist in the study" was not found in any of the reports.

CONCLUSIONS

Although the quality of reporting trials is not always related to a biased estimation of treatment effect, the items used for quality measures must be described to enable readers to evaluate the quality and applicability of the reporting. Further development of an assessment tool is needed for items specific to surgical randomized controlled trials.

[Randomised controlled trials]

Nowadays the Randomised Controlled Trial (RCT) is seen as the gold standard for estimating the effectiveness of an observed intervention, achieving the highest hierarchy of evidence of primary research settings. Its study design basically includes two groups of patients, an intervention group and a control group; patients are randomly allocated to these two groups. After intervention or control intervention took place, predefined outcomes are quantified and compared in the two groups. The study design aims at eliminating all confounding and distorting factors (Bias and Confounder), so that different outcomes between the groups can be only explained by the intervention. There is a broad variation of quality of published RCTs. The reliability of results and extent to which findings provide a correct basis for generalisation to other circumstances needs to be validated. As part of a methods series of the Wiener Medizinische Wochenschrift this paper will discuss principles of study design, critical appraisal and limitations of RCTs.

Chinese authors do need CONSORT: reporting quality assessment for five leading Chinese medical journals

OBJECTIVE

Only a few Chinese medical journals have recommended CONSORT in their "Instruction for authors or Guide for authors". This study aims to evaluate the reporting quality of randomized controlled trials (RCTs) published in the five leading Chinese medical journals indexed by MEDLINE.

METHODS

We identified RCTs published from 2004 to January 2007 in five leading Chinese medical journals by searching three important Chinese databases systematically, namely CNKI (China National Knowledge Infrastructure/Chinese Academic Journals full text Database), VIP (a full text database of China) and CBM disc (China Biomedicine Database Disc) and assessed the quality of each RCT by using the Consolidated Standards for Reporting of Trials (CONSORT) and the 5-point Jadad scale.

RESULTS

One hundred and forty two RCTs were included. Based on the items in the revised CONSORT statement, 130 (91.55%) of the 142 RCTs mentioned "randomization" in the title or abstract, but only 38 (26.76%) RCTs described the method to generate the random sequence; only 6 RCTs had adequate allocation concealment; 24 (17.61%) RCTs mentioned "masking", but only 7 described the process of masking. Three out of 40 items were reported clearly in all included trials, while five items were not mentioned at all. The quality of RCTs was low as assessed by the Jadad scale and 22 RCTs were high-quality research (>/=3 points).

CONCLUSIONS

The reporting quality of RCTs published in the five leading Chinese medical journals is low. Chinese journals should adopt the CONSORT statement to improve the reporting quality of Chinese randomized controlled trials.

From cellular to high-throughput predictive assays in radiation oncology: challenges and opportunities

Substantial research efforts into predictive radiation oncology have so far produced very little in terms of clinically applicable assays. This may change with the development of novel high-throughput assays that are of potential interest in a radiation oncology setting. However, it seems that much current research is opportunistic, driven by the available technologies rather than addressing pertinent clinical or biological questions. This review looks at the experience gained from the attempts to develop cellular radiobiology assays. The research process and, in particular, the need for rigorous validation of any promising assay in an independent dataset are stressed. Some common design problems are discussed using examples from radiation oncology. The statistical challenges and some of the key concepts in analyzing dense datasets from high-throughput assays are briefly reviewed. Single nucleotide polymorphisms, immunohistochemical markers, and DNA microarray gene signatures are used as examples of assays that show promise in radiation oncology applications. Some recent studies suggest a differential treatment response between tumor stem cells and other tumor cells. If this is a general pattern, then future predictive assays may have to be performed on stems cells rather than on unselected tumor cells. Advances in radiogenomics or radioproteomics will come from large collaborative research networks, collecting high-quality dosimetric and clinical outcome data and combining state-of-the-art laboratory techniques with appropriate biostatical methods.

Better reporting, better research: guidelines and guidance in PLoS Medicine

PLoS Medicine announces a new section: Guidelines and Guidance.

An assessment of the quality of randomised controlled trials conducted in China

Background

Despite the rapid increase in research in China, little is known about the quality of clinical trials conducted there.

Methods

A systematic review and critical appraisal of randomised controlled trials (RCTs) conducted in China and published in 2004 was undertaken to describe their characteristics, assess the quality of their reporting, and where possible, the quality of their conduct. Randomised controlled trials in all disease areas and types of interventions, which took place in China and included Chinese citizens were identified using PubMed and hand searching the Journal Series of the Chinese Medical Association. Quality was assessed against a subset of criteria adapted from the CONSORT statement.

Results

Three hundred and seven RCTs were included. One hundred and ninety-nine (64.8%) failed to report methods of randomization and 254 (82.4%) did not mention blinding of either participants or investigators. Reporting of baseline characteristics, primary outcome and length of follow-up was inadequate in a substantial proportion of studies. Fewer than 11% of RCTs mentioned ethical approval and only 18.0% adequately discussed informed consent. However, dropout rates were very favourable with nearly 44% of trials reporting a zero dropout rate.

Conclusion

Reporting of RCTs in China requires substantial improvement to meet the targets of the CONSORT statement. The conduct of Chinese RCTs cannot be directly inferred from the standard of reporting; however without good reporting the methods of the trials cannot be clearly ascertained.

Quality of reporting randomized controlled trials (RCTs) in the nursing literature: application of the consolidated standards of reporting trials (CONSORT)

In the era of evidence-based practice (EBP), Randomized Controlled Trials (RCTs) may provide the best evidence of the efficacy of nursing interventions and yet the quality of RCT reporting in nursing literature has not been evaluated. The purposes of this study were to apply the Consolidated Standards of Reporting Trials (CONSORT) statement to published reports of nursing science, examine how adequately the published reports adhere to the statement, and examine the effect of the adoption of CONSORT on the quality of the RCT published reports. One hundred RCTs from 2002-2005 were identified from 4 nursing journals. Articles were randomly assigned to 4 reviewers and the quality of the published reports was evaluated using a modified CONSORT checklist. There was no difference between the 4 journals in the quality of the published reports of RCTs based on the modified CONSORT checklist employed (F = 1.27, P =.29). The quality of reporting of RCTs improved significantly in the only journal, Nursing Research, to adopt the CONSORT statement during the study period (t =-2.70, P =.01). Adoption of CONSORT is recommended as it may lead to an overall improvement in quality of reporting of RCTs in nursing journals. The profession may also wish to explore the use or development of standards similar to CONSORT but ones more appropriate for the types of research typical of that published by nurse scientists.

Neuroprotection for ischemic stroke: past, present and future

Neuroprotection for ischemic stroke refers to strategies, applied singly or in combination, that antagonize the injurious biochemical and molecular events that eventuate in irreversible ischemic injury. There has been a recent explosion of interest in this field, with over 1000 experimental papers and over 400 clinical articles appearing within the past 6 years. These studies, in turn, are the outgrowth of three decades of investigative work to define the multiple mechanisms and mediators of ischemic brain injury, which constitute potential targets of neuroprotection. Rigorously conducted experimental studies in animal models of brain ischemia provide incontrovertible proof-of-principle that high-grade protection of the ischemic brain is an achievable goal. Nonetheless, many agents have been brought to clinical trial without a sufficiently compelling evidence-based pre-clinical foundation. At this writing, around 160 clinical trials of neuroprotection for ischemic stroke have been initiated. Of the approximately 120 completed trials, two-thirds were smaller early-phase safety-feasibility studies. The remaining one-third were typically larger (>200 subjects) phase II or III trials, but, disappointingly, only fewer than one-half of these administered neuroprotective therapy within the 4-6h therapeutic window within which efficacious neuroprotection is considered to be achievable. This fact alone helps to account for the abundance of "failed" trials. This review presents a close survey of the most extensively evaluated neuroprotective agents and classes and considers both the strengths and weakness of the pre-clinical evidence as well as the results and shortcomings of the clinical trials themselves. Among the agent-classes considered are calcium channel blockers; glutamate antagonists; GABA agonists; antioxidants/radical scavengers; phospholipid precursor; nitric oxide signal-transduction down-regulator; leukocyte inhibitors; hemodilution; and a miscellany of other agents. Among promising ongoing efforts, therapeutic hypothermia, high-dose human albumin therapy, and hyperacute magnesium therapy are considered in detail. The potential of combination therapies is highlighted. Issues of clinical-trial funding, the need for improved translational strategies and clinical-trial design, and "thinking outside the box" are emphasized.

Reporting of Randomized Controlled Trials in Occupational Therapy and Speech Therapy: Evaluation Using an Expansion of the Consort Statement

Research in occupational therapy is necessary for the validation of therapeutic interventions. Randomized controlled trials (RCTs) control against potentially confounding variables and provide validating evidence, if properly reported. The purpose of this study was to evaluate the quality of reporting RCTs in occupational therapy and speech therapy by use of an expansion of the Consolidated Standards of Reporting Trials (CONSORT) statement. The Nelson—Moberg—Norton Expanded CONSORT Instrument (NMNECI) was used to evaluate 15 occupational therapy RCTs and 15 speech therapy RCTs. After the principal investigator evaluated all 30 articles, a masked research assistant evaluated 16 randomly selected RCTs to test for inter-rater reliability. An intraclass correlation coefficient of .93 confirmed overall inter-rater agreement. On average, the 30 articles were consistent with 119.5 NMNECI sub-items ( SD = 25.48) of the 212 possible sub-items. Occupational therapy RCTs (mean = 130.07, SD = 20.21) were more consistent with the CONSORT statement than speech therapy RCTs (mean = 108.93, SD = 26.40), t(28) = 2.46, p = .02. RCTs in occupational therapy and other allied health professions could benefit from using the CONSORT statement as a tool for reporting future research studies.

Quality of reporting of key methodological items of randomized controlled trials in clinical ophthalmic journals

PURPOSE

To evaluate the reporting quality of key methodological items in randomized controlled trials (RCTs) in four general clinical ophthalmology journals.

METHODS

The reporting of 11 key methodological items in RCTs published in American Journal of Ophthalmology, Archives of Ophthalmology, British Journal of Ophthalmology and Ophthalmology in the year 2005 was assessed.

RESULTS

Sixty-seven eligible RCTs were assessed and the mean number of items reported was 6.3 per RCT. No significant difference in the mean number of items reported was found between the four journals (P=0.20). The most frequently reported item was ethics approval and informed consent (97.0%), followed by masking status (85.1%), description of withdrawals (76.1%), adverse events (73.1%), and intention-to-treat analysis (71.6%). Details on sequence generation, randomization restriction, allocation concealment, allocation implementation, patient flow diagrams, and sample size calculation were reported in <50% of the RCTs assessed. Both sample size and page length of the RCTs correlated with the number of methodological items reported (P=0.024 and P=0.008, respectively).

CONCLUSIONS

Similar to other specialties, rooms for improvement exist in the reporting of key methodological items of RCTs in clinical ophthalmic journals. Stricter adoption of the CONSORT statement might enhance the reporting quality of RCTs in ophthalmic journals.

A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials

BACKGROUND

We investigated whether there had been an improvement in quality of reporting for randomised controlled trials of acupuncture since the publication of the STRICTA and CONSORT statements. We conducted a before-and-after study, comparing ratings for quality of reporting following the publication of both STRICTA and CONSORT recommendations.

METHODOLOGY AND PRINCIPAL FINDINGS

Ninety peer reviewed journal articles reporting the results of acupuncture trials were selected at random from a wider sample frame of 266 papers. Papers published in three distinct time periods (1994-1995, 1999-2000 and 2004-2005) were compared. Assessment criteria were developed directly from CONSORT and STRICTA checklists. Papers were independently assessed for quality of reporting by two assessors, one of whom was blind to information which could have introduced systematic bias (e.g. date of publication). We detected a statistically significant increase in the reporting of CONSORT items for papers published in each time period measured. We did not, however, find a difference between the number of STRICTA items reported in journal articles published before and 3 to 4 years following the introduction of STRICTA recommendations.

CONCLUSIONS AND SIGNIFICANCE

The results of this study suggest that general standards of reporting for acupuncture trials have significantly improved since the introduction of the CONSORT statement in 1996, but that quality in reporting details specific to acupuncture interventions has yet to change following the more recent introduction of STRICTA recommendations. Wider targeting and revision of the guidelines is recommended.

Scales to assess the quality of randomized controlled trials: a systematic review

BACKGROUND AND PURPOSE

The methodological quality of randomized controlled trials (RCTs) is commonly evaluated in order to assess the risk of biased estimates of treatment effects. The purpose of this systematic review was to identify scales used to evaluate the methodological quality of RCTs in health care research and summarize the content, construction, development, and psychometric properties of these scales.

METHODS

Extensive electronic database searches, along with a manual search, were performed.

RESULTS

One hundred five relevant studies were identified. They accounted for 21 scales and their modifications. The majority of scales had not been rigorously developed or tested for validity and reliability. The Jadad Scale presented the best validity and reliability evidence; however, its validity for physical therapy trials has not been supported.

DISCUSSION AND CONCLUSION

Many scales are used to evaluate the methodological quality of RCTs, but most of these scales have not been adequately developed and have not been adequately tested for validity and reliability. A valid and reliable scale for the assessment of the methodological quality of physical therapy trials needs to be developed.