Standards for reporting randomized controlled trials in neurosurgery

False claims of equivalence in the neurosurgical trauma literature: prevalence and associated factors-a systematic review protocol

INTRODUCTION

Research quality within the neurosurgical field remains suboptimal. Therefore, many studies published in the neurosurgical literature lack enough statistical power to establish the presence or absence of clinically important differences between treatment arms. The field of neurotrauma deals with additional challenges, with fewer financial incentives and restricted resources in low-income and middle-income countries with the highest burden of neurotrauma diseases. In this systematic review, we aim to estimate the prevalence of false claims of equivalence in the neurosurgical trauma literature and identify its predictive factors.

METHODS AND ANALYSIS

The Preferred Reporting Items for Systematic Review and Meta-Analyses recommendations were followed. Randomised clinical trials that enrolled only traumatic brain injury patients and investigated any type of intervention (surgical or non-surgical) will be eligible for inclusion. The MEDLINE/PubMed database will be searched for articles in English published from January 1960 to July 2020 in 15 top-ranked journals. A false claim of equivalence will be identified by insufficient power to detect a clinically meaningful effect: for categorical outcomes, a difference of at least 25% and 50%, and for continuous outcomes, a Cohen's d of at least 0.5 and 0.8. Using the number of patients in each treatment arm and the minimum effect sizes to be detected, the power of each study will be calculated with the assumption of a two-tailed alpha that equals 0.05. Standardised differences between the groups with and without a false claim of equivalence will be calculated, and the variables with a standardised difference equal or above 0.2 and 0.5 will be considered weakly and strongly associated with false claims of equivalence, respectively. The data analysis will be blinded to the authors and institutions of the studies.

ETHICS AND DISSEMINATION

This study will not involve primary data collection. Therefore, formal ethical approval will not be required. The final systematic review will be published in a peer-reviewed journal and presented at appropriate conferences.

A Systematic Review of Registered Clinical Trials for Peripheral Nerve Injuries

ABSTRACT

Upper extremity peripheral nerve injuries (PNIs) significantly impact daily functionality and necessitate effective treatment strategies. Clinical trials play a crucial role in developing these strategies. However, challenges like retrospective data collection, reporting biases, inconsistent outcome measures, and inadequate data sharing practices hinder effective research and treatment advancements. This review aims to analyze the landscape of reporting, methodological design, outcome measures, and data sharing practices in registered clinical trials concerning upper extremity PNIs. It seeks to guide future research in this vital area by identifying current trends and gaps.A systematic search was conducted on ClinicalTrials.gov and WHO International Clinical Trials Registry Platform up to November 10, 2023, using a combination of MeSH terms and keywords related to upper extremity nerve injury. The PRISMA 2020 guidelines were followed, and the studies were selected based on predefined inclusion and exclusion criteria. A narrative synthesis of findings was performed, with statistical analysis for associations and completion rates.Of 3051 identified studies, 96 met the inclusion criteria. These included 47 randomized controlled trials, 27 nonrandomized trials, and others. Sensory objective measures were the most common primary outcomes. Only 13 studies had a data sharing plan. The analysis revealed varied intervention methods and inconsistencies in outcome measures. There was a significant association between study funding, design, and completion status, but no association between enrollment numbers and completion.This review highlights the need for standardized outcome measures, patient-centered assessments, and improved data sharing in upper extremity PNI trials. The varied nature of interventions and inconsistency in outcome measures indicate the necessity for more rigorous and transparent research practices to strengthen the evidence base for managing these injuries.

The Rising Quality of Randomized Controlled Trials in The Journal of Bone & Joint Surgery: An Updated Analysis from 2014 to 2022

Background

Previous reports found that 40% of randomized controlled trials (RCTs) published in The Journal of Bone & Joint Surgery (JBJS) from 1988 to 2000 and 47% of those published from 2001 to 2013 were of high quality. The purpose of this study was to assess the quality of RCTs published from 2014 to 2022 in JBJS and to compare these findings with those of prior analyses in order to identify trends over time and areas for continued improvement.

Methods

PubMed was searched for the term "randomized controlled trial" to identify studies published in JBJS from 2014 to 2022. Each included RCT was evaluated with use of the Detsky score and a risk-of-bias assessment modified from the Cochrane tool. These evaluations were then compared with previous evaluations of RCTs from the 1988 to 2000 and 2001 to 2013 periods with use of independent-sample t tests. A transformed Detsky score of >75% and a modified risk-of-bias score of ≥8 were defined as being indicative of high quality.

Results

A total of 218 RCTs were published in JBJS from 2014 to 2022. An a priori sample size was calculated in 183 studies (83.9%). A total of 152 (83.1%) of the 183 studies enrolled the calculated number of patients, of which 126 (82.9%) maintained an adequate number at the time of final follow-up. Most RCTs were conducted at a single center (146 of 218; 67%), evaluated a surgical intervention (162 of 218; 74%), and reported positive results (142 of 218; 65%). The mean transformed Detsky score was 85% ± 10% (95% confidence interval, 83.7% to 86.3%), with 82% of trials (179 of 218) scored as high quality. The mean transformed Detsky score from 2014 to 2022 was higher than that from 1988 to 2000 and that from 2001 to 2013 (85% versus 76% and 68%, respectively; p < 0.001). The mean modified risk-of-bias score was 7 ± 1, with 42% of trials (92 of 218) scored as high quality. RCTs published from 2014 to 2022 had a higher mean modified risk-of-bias score than those published from 2001 to 2013 (7 ± 1 versus 6 ± 1; p < 0.001). Compared with the 2001 to 2013 and 2014 to 2022 periods, the 1988 to 2000 period had a greater proportion of trials that reported positive results (51% and 65% versus 82%, respectively; p < 0.001) and that included data from multiple centers (31% and 33% versus 67%; p < 0.001).

Conclusions

The quality of RCTs published in JBJS from 2014 to 2022 has improved from that reported previously, as demonstrated by the increases in the modified risk-of-bias score and transformed Detsky score from prior periods. This may be the result of journal policies such as the requirements of CONSORT adherence and prospective trial registration. Investigators should focus on improving the clarity of reporting, limiting attrition bias, and making efforts to blind support staff in order to increase the quality of future RCTs.

Clinical Relevance

Improving the quality of RCTs is crucial given their potential to influence current clinical practice.

Posters Presented at North American Skull Base Society 2016-2018: What Factors Influence Their Publication?

Objective  Research productivity impacts an individual's academic credentials and serves to advance the field of neurosurgery at large. Poster presentations allow researchers to share preliminary results with respected colleagues; however, more critical is the ability to publish peer-reviewed articles. Key factors that lead posters to journal publication are not well understood and difficult to quantify. This study investigates the association between bibliometrics of authors who presented posters at the North American Skull Base Society (NASBS) meeting and odds of journal publication. Methods  Posters from the 2016 to 2018 NASBS archive were reviewed. Hirsch-index (h-index) of first (FH) and senior (SH) authors, research type, research topic, and number of poster authors (nAuthPost) were collected. For posters published as journal articles, number of days from poster presentation to publication (nDays), number of authors in published articles (nAuthArt), and journal impact factor (JIF) were recorded. Results  One-hundred sixty-nine of 481 posters (35.1%) were published as articles. Median FH and SH for published versus unpublished posters were 7 versus 5 ( p  = 0.01) and 29 versus 19 ( p  < 0.001), respectively. When adjusted with multivariate regression, only SH ( p  < 0.001) and nAuthPost ( p  = 0.001) were significantly associated with odds of publication. Median (interquartile range [IQR]) nDays was 361 (394). Increased authors from poster to article ( p  = 0.017) and lower FH ( p  = 0.08) were correlated with increased time to publication. Median (IQR) JIF for all publications was 1.723 (1.068). Conclusions  Bibliometrics such as h-index and number of authors from posters can help objectively characterize and predict future success in research productivity.

Robustness of Randomized Control Trials Supporting Current Neurosurgery Guidelines

BACKGROUND

Treatment guidelines in neurosurgery are often based on evidence obtained from randomized controlled trials (RCTs).

OBJECTIVE

To evaluate the robustness of RCTs supporting current central nervous tumor and cerebrovascular disease guidelines by calculating their fragility index (FI)-the minimum number of patients needed to switch from an event to nonevent outcome to change significant trial primary outcome.

METHODS

We analyzed RCTs referenced in the Congress of Neurological Surgeons and American Association of Neurological Surgeons guidelines on central nervous tumor and cerebrovascular disease management. Trial characteristics, finding of a statistically significant difference in the primary endpoint favoring the experimental intervention, the FI, and FI minus number lost to follow-up were assessed.

RESULTS

Of 312 RCTs identified, 158 (50.6%) were published from 2000 to 2010 and 106 (34%) after 2010. Sixty-three trials (19.2%) were categorized as surgical trials, and the rest studied medical treatment (82.0%) or percutaneous intervention (8.33%). The trials had a median power of 80.0% (IQR 80.0-90.0). Of these, 120 trials were eligible for FI calculation. The median FI was 7.0 (IQR 2.0-16.25). Forty-four (36.6%) trials had FI ≤ 3 indicating very low robustness. After adjusting for covariates, recently published trials and trials studying percutaneous interventions were associated with significantly higher FI compared with older trials and trials comparing surgical approaches, respectively. Trials limited to single centers were associated with significantly lower FI.

CONCLUSION

Trials supporting current guidelines on neuro-oncological and neurovascular surgical interventions have low robustness. While the robustness of trials has improved over time, future guidelines must take into consideration this metric in their recommendations.

Systematic review of sample size calculations and reporting in randomized controlled trials in ophthalmology over a 20-year period

PURPOSE

Randomized Controlled Trials (RCTs) are considered the gold standard for the practice of evidence-based medicine. The purpose of this study is to systematically assess the reporting of sample size calculations in ophthalmology RCTs in 5 leading journals over a 20-year period. Reviewing sample size calculations in ophthalmology RCTs will shed light on the methodological quality of RCTs and, by extension, on the validity of published results.

METHODS

The MEDLINE database was searched to identify full reports of RCTs in the journals Ophthalmology, JAMA Ophthalmology, American Journal of Ophthalmology, Investigative Ophthalmology and Visual Science, and British Journal of Ophthalmology between January and December of the years 2000, 2010 and 2020. Screening identified 559 articles out of which 289 met the inclusion criteria for this systematic review. Data regarding sample size calculation reporting and trial characteristics was extracted for each trial by independent investigators.

RESULTS

In 2020, 77.9% of the RCTs reported sample size calculations as compared with 37% in 2000 (p < 0.001) and 60.7% in 2010 (p = 0.012). Studies reporting all necessary parameters for sample size recalculation increased significantly from 17.2% in 2000 to 39.3% in 2010 and 43.0% in 2020 (p < 0.001). Reporting of funding was greater in 2020 (98.8%) compared with 2010 (89.3%) and 2000 (53.1%). Registration in a clinical trials database occurred more frequently in 2020 (94.2%) compared to 2000 (1.2%; p < 0.001) and 2010 (68%; p < 0.001). In 2020, 38.4% of studies reported different sample sizes in the online registry from the published article. Overall, the most studied area in 2000 was glaucoma (29.6% of RCTs), whereas in 2010 and 2020, it was retina (40.2 and 37.2% of the RCTs, respectively). The number of patients enrolled in a study and the number of eyes studied was significantly greater in 2020 compared to 2000 and 2010 (p < 0.001).

CONCLUSION

Sample size calculation reporting in ophthalmology RCTs has improved significantly between the years 2000 and 2020 and is comparable to other fields in medicine. However, reporting of certain parameters remains inconsistent with current publication guidelines.

Randomized controlled trials in neurosurgery

Randomized controlled trials (RCTs) have become the standard method of evaluating new interventions (whether medical or surgical), and the best evidence used to inform the development of new practice guidelines. When we review the history of medical versus surgical trials, surgical RCTs usually face more challenges and difficulties when conducted. These challenges can be in blinding, recruiting, funding, and even in certain ethical issues. Moreover, to add to the complexity, the field of neurosurgery has its own unique challenges when it comes to conducting an RCT. This paper aims to provide a comprehensive review of the history of neurosurgical RCTs, focusing on some of the most critical challenges and obstacles that face investigators. The main domains this review will address are: (1) Trial design: equipoise, blinding, sham surgery, expertise-based trials, reporting of outcomes, and pilot trials, (2) trial implementation: funding, recruitment, and retention, and (3) trial analysis: intention-to-treat versus as-treated and learning curve effect.

Spin in the neurosurgical trauma literature: prevalence and associated factors - a systematic review protocol

INTRODUCTION

Spin is defined as an inaccurate interpretation of results, intentionally or not, leading to equivocal conclusions and misdirecting readers to look at the data in an overly optimistic way. Previous studies have shown a high prevalence of spin in scientific papers and this systematic review aims to investigate the nature and prevalence of spin in the neurosurgical trauma literature. Any associated factors will be identified to guide future research practice recommendations.

METHODS AND ANALYSIS

The Preferred Reporting Item for Systematic Reviews and Meta-Analyses recommendations will be followed. Randomised clinical trials (RCTs) that enrolled only patients with traumatic brain injury and investigated any type of intervention (surgical or non-surgical) will be eligible for inclusion. The MEDLINE/PubMed database will be searched for articles in English published in 15 top-ranked journals. Spin will be defined as (1) a focus on statistically significant results not based on the primary outcome; (2) interpreting statistically non-significant results for a superiority analysis of the primary outcome; (3) claiming or emphasising the beneficial effect of the treatment despite statistically non-significant results; (4) conclusion focused in the per-protocol or as-treated analysis instead of the intention-to-treat results; (5) incorrect statistical analysis; (6) republication of a significant secondary analysis without proper acknowledgement of the primary outcome analysis result. Traditional descriptive statistics will be used to present RCT characteristics. Standardised differences between the groups with or without spin will be calculated. The variables with a standardised difference equal or above 0.2 and 0.5 will be considered weakly and strongly associated with spin, respectively.

ETHICS AND DISSEMINATION

This study will not involve primary data collection and patients will not be involved.

TRIAL REGISTRATION NUMBER

10.17605/OSF.IO/H3FGY.

A Retrospective Analysis of Randomized Controlled Trials on Traumatic Brain Injury: Evaluation of CONSORT Item Adherence

Traumatic brain injury (TBI) contributes to death and disability, resulting in an enormous individual and socio-economic challenges. Despite huge efforts, there are still controversies on treatment strategies and early outcome estimation. We evaluate current randomized controlled trials (RCTs) on TBI according to their fulfillment of the CONSORT (Consolidated Statement of Reporting Trials) statement’s criteria as a marker of transparency and the quality of study planning and realization. A PubMed search for RCTs on TBI (January 2014–December 2019) was carried out. After screening of the abstracts (n = 1.926), the suitable full text manuscripts (n = 72) were assessed for the fulfillment of the CONSORT criteria. The mean ratio of consort statement fulfillment was 59% (±13%), 31% of the included studies (n = 22) complied with less than 50% of the CONSORT criteria. Citation frequency was moderately related to ratio of CONSORT item fulfillment (r = 0.4877; p < 0.0001) and citation frequency per year (r = 0.5249; p < 0.0001). The ratio of CONSORT criteria fulfillment was associated with the impact factor of the publishing journal (r = 0.6428; p < 0.0001). Essential data for study interpretation, such as sample size determination (item 7a), participant flow (item 13a) as well as losses and exclusions (item 13b), were only reported in 53%, 60% and 63%, respectively. Reporting and methodological aspects in RCTs on TBI still may be improved. Thus, the interpretation of study results may be hampered due to methodological weaknesses.

A multiyear systematic survey of the quality of reporting for randomised trials in dentistry, neurology and geriatrics published in journals of Spain and Latin America

BACKGROUND

The Iberoamerican Cochrane Network is currently developing an extensive project to identify Spanish-language journals that publish original clinical research in Spain and Latin America. The project is called BADERI (Database of Iberoamerican Essays and Journal) and feeds the research articles, mainly randomised clinical trials (RCTs), into CENTRAL (Cochrane Collaboration Central Register of Controlled Trials). This study aims to assess the quality of reporting of RCTs published in Spanish and Latin American journals for three clinical fields and assess changes over time.

METHODS

We did a systematic survey with time trend analysis of RCTs for dentistry, geriatrics, and neurology. These fields were chosen for pragmatic reasons as they had not yet been completed in BADERI. After screening RCTs from 1990 to 2018 for randomised or quasi-randomised clinical trials, we extracted data for 23 CONSORT items. The primary outcome was the total score of the 23 predefined CONSORT 2010 items for each RCT (score range from 0 to 34). The secondary outcome measure was the score for each one of these 23 items.

RESULTS

A total of 392 articles from 1990 to 2018 were included as follows: dentistry (282), neurology (80), and geriatrics (30). We found that the overall compliance score for the CONSORT items included in this study for all 392 RCTs analysed was 12.6 on a scale with a maximum score of 34. With time, the quality of reporting improved slightly for all RCTs. None of the articles achieved the complete individual CONSORT item compliance score. The lowest overall compliance percentage was for item 10 (Randomisation implementation) and item 24 (Protocol registration), with a dismal 1% compliance across all included RCTs, regardless of country.

CONCLUSIONS

CONSORT compliance is very poor in the 392 analysed RCTs. The impact of the CONSORT statement on improving the completeness of RCT reporting in Latin America and Spain is not clear. Iberoamerican journals should become more involved in endorsing and enforcing adherence to the CONSORT guidelines.

European Guidelines for Obesity Management in Adults with a Very Low-Calorie Ketogenic Diet: A Systematic Review and Meta-Analysis

BACKGROUND

The very low-calorie ketogenic diet (VLCKD) has been recently proposed as an appealing nutritional strategy for obesity management. The VLCKD is characterized by a low carbohydrate content (<50 g/day), 1-1.5 g of protein/kg of ideal body weight, 15-30 g of fat/day, and a daily intake of about 500-800 calories.

OBJECTIVES

The aim of the current document is to suggest a common protocol for VLCKD and to summarize the existing literature on its efficacy in weight management and weight-related comorbidities, as well as the possible side effects.

METHODS

This document has been prepared in adherence with Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines. Literature searches, study selection, methodology development, and quality appraisal were performed independently by 2 authors and the data were collated by means of a meta-analysis and narrative synthesis.

RESULTS

Of the 645 articles retrieved, 15 studies met the inclusion criteria and were reviewed, revealing 4 main findings. First, the VLCKD was shown to result in a significant weight loss in the short, intermediate, and long terms and improvement in body composition parameters as well as glycemic and lipid profiles. Second, when compared with other weight loss interventions of the same duration, the VLCKD showed a major effect on reduction of body weight, fat mass, waist circumference, total cholesterol and triglyceridemia as well as improved insulin resistance. Third, although the VLCKD also resulted in a significant reduction of glycemia, HbA1c, and LDL cholesterol, these changes were similar to those obtained with other weight loss interventions. Finally, the VLCKD can be considered a safe nutritional approach under a health professional's supervision since the most common side effects are usually clinically mild and easily to manage and recovery is often spontaneous.

CONCLUSIONS

The VLCKD can be recommended as an effective dietary treatment for individuals with obesity after considering potential contra-indications and keeping in mind that any dietary treatment has to be personalized. Prospero Registry: The assessment of the efficacy of VLCKD on body weight, body composition, glycemic and lipid parameters in overweight and obese subjects: a meta-analysis (CRD42020205189).

TRIPOD statement: a preliminary pre-post analysis of reporting and methods of prediction models

OBJECTIVES

To assess the difference in completeness of reporting and methodological conduct of published prediction models before and after publication of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.

METHODS

In the seven general medicine journals with the highest impact factor, we compared the completeness of the reporting and the quality of the methodology of prediction model studies published between 2012 and 2014 (pre-TRIPOD) with studies published between 2016 and 2017 (post-TRIPOD). For articles published in the post-TRIPOD period, we examined whether there was improved reporting for articles (1) citing the TRIPOD statement, and (2) published in journals that published the TRIPOD statement.

RESULTS

A total of 70 articles was included (pre-TRIPOD: 32, post-TRIPOD: 38). No improvement was seen for the overall percentage of reported items after the publication of the TRIPOD statement (pre-TRIPOD 74%, post-TRIPOD 76%, 95% CI of absolute difference: -4% to 7%). For the individual TRIPOD items, an improvement was seen for 16 (44%) items, while 3 (8%) items showed no improvement and 17 (47%) items showed a deterioration. Post-TRIPOD, there was no improved reporting for articles citing the TRIPOD statement, nor for articles published in journals that published the TRIPOD statement. The methodological quality improved in the post-TRIPOD period. More models were externally validated in the same article (absolute difference 8%, post-TRIPOD: 39%), used measures of calibration (21%, post-TRIPOD: 87%) and discrimination (9%, post-TRIPOD: 100%), and used multiple imputation for handling missing data (12%, post-TRIPOD: 50%).

CONCLUSIONS

Since the publication of the TRIPOD statement, some reporting and methodological aspects have improved. Prediction models are still often poorly developed and validated and many aspects remain poorly reported, hindering optimal clinical application of these models. Long-term effects of the TRIPOD statement publication should be evaluated in future studies.

A tutorial on methodological studies: the what, when, how and why

BACKGROUND

Methodological studies - studies that evaluate the design, analysis or reporting of other research-related reports - play an important role in health research. They help to highlight issues in the conduct of research with the aim of improving health research methodology, and ultimately reducing research waste.

MAIN BODY

We provide an overview of some of the key aspects of methodological studies such as what they are, and when, how and why they are done. We adopt a "frequently asked questions" format to facilitate reading this paper and provide multiple examples to help guide researchers interested in conducting methodological studies. Some of the topics addressed include: is it necessary to publish a study protocol? How to select relevant research reports and databases for a methodological study? What approaches to data extraction and statistical analysis should be considered when conducting a methodological study? What are potential threats to validity and is there a way to appraise the quality of methodological studies?

CONCLUSION

Appropriate reflection and application of basic principles of epidemiology and biostatistics are required in the design and analysis of methodological studies. This paper provides an introduction for further discussion about the conduct of methodological studies.

Conflicts of interest in randomized controlled trials reported in neurosurgical journals

OBJECTIVE

Randomized controlled trials (RCTs) form the basis of today's evidence-based approach to medicine, and play a critical role in guidelines and the drug and device approval process. Conflicts of interest (COIs) are commonplace in medical research, but little is known about their influence. The authors aimed to evaluate the extent and influence of COIs in recent RCTs published in core neurosurgical journals using a cross-sectional analysis.

METHODS

Through review of 6 general neurosurgical journals, all interventional RCTs published from January 2009 to January 2019 were identified. Because it is difficult to objectively assess study outcome, the authors opted for a strict rating approach based on the statistical significance of unambiguously reported primary endpoints, and the reported statistical protocol.

RESULTS

A total of 129 RCTs met the inclusion criteria. During the study period, the Journal of Neurosurgery published the largest number of RCTs (n = 40, 31%). Any potential COI was disclosed by 57%, and a mean of 12% of authors had a personal COI. Nonfinancial industry involvement was reported in 10%, while 31% and 20% received external support and sponsoring, respectively. Study registration was reported by 56%, while 51% of studies were blinded. Registration showed an increasing trend from 17% to 76% (p < 0.001). The median randomized sample size was 92 (interquartile range 50-153), and 8% were designed to investigate noninferiority or equality. Sixty-three RCTs (49%) unambiguously reported a primary endpoint, of which 13% were composite primary endpoints. In 43%, study outcome was positive, which was associated with a noninferiority design (31% vs 3%, p = 0.007) and a composite primary endpoint (46% vs 9%, p = 0.002). Potential COIs were not significantly associated with study positivity (69% vs 59%, p = 0.433). In the multivariate analysis, only a composite primary endpoint remained predictive of a positive study outcome (odds ratio 6.34, 95% confidence interval 1.51-33.61, p = 0.017).

CONCLUSIONS

This analysis provides an overview of COIs and their potential influence on recent trials published in core neurosurgical journals. Reporting of primary endpoints, study registration, and uniform disclosure of COIs are crucial to ensure the quality of future neurosurgical randomized trials. COIs do not appear to significantly influence the outcome of randomized neurosurgical trials.

Randomized Controlled Trials on Intracerebral Hemorrhage: A Cross Sectional Retrospective Analysis of CONSORT Item Adherence

Object: Intracranial hemorrhage (ICH) is the second most common cause of stroke but still there is little consolidated knowledge about the optimal treatment strategies (e.g., the benefit of surgical evacuation). We evaluated the current randomized controlled trials (RCTs) on primary ICH (01.2013–03.2017) according to their fulfillment of the CONSORT statement's criteria (published in 2010) –as a marker of transparency and quality of study planning and realization.

Methods: A Pubmed and a Cochrane database (including clinicaltrials.gov) search was carried out (01.2014–3.2017, respectively 01.2013–12.2013). Abstracts were screened for inclusion. Eligible full text manuscripts were assessed for the implementation of the CONSORT criteria. Citation frequencies and impact factors of the journals were related to ratio of CONSORT criteria fulfillment. Further, the risk of bias according to the Risk of bias tool 2 (RoB 2) was assessed.

Results: Overall 3097 abstracts were screened for inclusion; 39 studies were suitable for final analysis. A mean fulfillment ratio of 51% (±28%) was found. A high correlation between impact factor and adherence to CONSORT criteria was shown (r = 0.7664; p < 0.0001). Citation frequency per year was related to ratio of CONSORT item fulfillment (r = 0.6747; p < 0.0001) and to the impact factor of the publishing journal (r = 0.7310; p < 0.0001). Of note, the items 10 (randomization: implementation) and 21 (generalizability) showed particularly high rates of non-fulfillment (87 and 85%). The majority of studies (95%) complied with item 2b (specific objectives or hypotheses), but strikingly objectives were mostly described vaguely. Other essential criteria such as sample size determination, definition of outcome parameters, and participant flow were only fulfilled weakly (51, 54, and 39%).

Conclusions: Over 20 years after its inception there is still weak adherence to the CONSORT statement. As a consequence, conclusions are hampered by inadequate planning and/or reporting. Particularly with respect to pathologies as ICH lacking clear, evidence-based guidelines adherence to the CONSORT statement might improve research quality in order to define valuable treatment strategies.

Neurosurgical Randomized Controlled Trials-Distance Travelled

BACKGROUND

The evidence base for many neurosurgical procedures has been limited. We performed a comprehensive and systematic analysis of study design, quality of reporting, and trial results of neurosurgical randomized controlled trials (RCTs).

OBJECTIVE

To systematically assess the design and quality characteristics of neurosurgical RCTs.

METHODS

From January 1961 to June 2016, RCTs with >5 patients assessing any 1 neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, Scopus, and Cochrane Library.

RESULTS

The median sample size in the 401 eligible RCTs was 73 patients with a mean patient age of 49.6. Only 111 trials (27.1%) described allocation concealment, 140 (34.6%) provided power calculations, and 117 (28.9%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 226 reports (56.4%), no difference between the procedures was found in 166 trials (41.4%), and significant harm was reported in 9 trials (2.2%). Trials with a larger sample size were more likely to report randomization mode, specify allocation concealment, and power calculations (all P < .001). Government funding was associated with better specification of power calculations (P = .008) and of allocation concealment (P = .026), while industry funding was associated with reporting significant efficacy (P = .02). Reporting of funding, specification of randomization mode and primary outcomes, and mention of power calculations improved significantly (all, P < .05) over time.

CONCLUSION

Several aspects of the design and reporting of RCTs on neurosurgical procedures have improved over time. Better powered and accurately reported trials are needed in neurosurgery to deliver evidence-based care and achieve optimal outcomes.

Electronic search strategies fail to identify randomized controlled trials (RCTs) in neurosurgery

Randomized controlled trials (RCTs) are the gold standard studies to evaluate the efficacy of therapeutic interventions. Although they are frequently identified through open searches in electronic databases, no studies have evaluated how easy it is to identify RCTs in neurosurgery using electronic search strategies. The present study evaluated the sensitivity and specificity of different search strategies applied to commonly used databases to identify RCTs in neurosurgery. The total number of RCTs in neurosurgery published between 1960 and 2013 was determined through a detailed search involving open keyword searches in PubMed, Cochrane Library and Center for Reviews and Dissemination (CRD) databases, a PubMed search based on clinical entity-related keywords and hand-searches on the reference list of identified articles. The sensitivity and specificity were calculated for the open keyword searches on PubMed, the Cochrane Library and the CRD database and for the Cochrane's HSSS, based on the total number of the identified RCTs. Compared to the total of 1102 RCTs identified, PubMed open search yielded 4660 articles, among which 365 were RCTs (sensitivity: 33.1%; specificity: 7.8%). Cochrane open search yielded 621 among which 36 were RCTs (sensitivity: 3.2%; specificity: 5.8%) and CRD open search returned 78 articles, among which 4 were RCTs (sensitivity: 0.4% sensitivity; specificity: 5.1%). The Cochrane HSSS retrieved 10702 results, among which 340 were RCTs (sensitivity: 30.9%; specificity: 3.2%). Most RCTs in neurosurgery cannot be identified by commonly used search strategies, which emphasizes the need to improve their indexing.

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

Effects of Astaxanthin Supplementation on Oxidative Stress

A systematic review and meta-analysis was conducted in six databases from 1948 to 2015 to assess the antioxidant activity of astaxanthin in humans. Nine randomized controlled trials were included in the systematic review. Results of meta-analysis revealed a borderline significant antioxidant effect of astaxanthin between the intervention and control groups, with a malondialdehyde-lowering effect for lipid peroxidation (p = 0.050). However, the data included here are insufficient. When compared with the baseline in intervention groups, the meta-analysis suggested that astaxanthin supplements significantly decreased plasma malondialdehyde {Standard mean difference (SMD) -1.32 μmol/L [95% CI -1.92, -0.72]; p < 0.0001} and isoprostane (SMD -3.10 ng/mL [95% CI -4.69, -1.51]; p < 0.0001). However, they increased superoxide dismutase (SMD 1.57 U/mL [95% CI 0.57, 2.56]; p = 0.002) and total antioxidant capacity (SMD 0.77 mmol 95% CI [0.12, 1.43]; p = 0.018). For dosage subgroup analysis, high dose (≥20 mg/day) of astaxanthin showed significant antioxidant effect (on total antioxidant capacity, isoprostane, and superoxide dismutase, p < 0.05). However, low dose (<20 mg/day) showed no significant effect (p > 0.05). Further duration subgroup analysis indicated that astaxanthin showed antioxidant effect after a 3-week intervention (p < 0.001), whereas this effect was not observed after a 12-week or 3-month intervention (on isoprostane and superoxide dismutase, p > 0.05). This review suggested that the antioxidant effect of astaxanthin on humans is unclear.

Randomized controlled trials comparing surgery to non-operative management in neurosurgery: a systematic review

BACKGROUND

A randomized controlled trial (RCT) remains the pinnacle of clinical research design. However, RCTs in neurosurgery, especially those comparing surgery to non-operative treatment, are rare and their relevance and applicability have been questioned. This study set out to assess trial design and quality and identify their influence on outcomes in recent neurosurgical trials that compare surgery to non-operative treatment.

METHODS

From 2000 to 2017, PubMed and Embase databases and four trial registries were searched. RCTs were evaluated for study design, funding, adjustments to reported outcome measures, accrual of patients, and academic impact.

RESULTS

Eighty-two neurosurgical RCTs were identified, 40 in spine disorders, 19 neurovascular and neurotrauma, 11 functional neurosurgery, ten peripheral nerve, and two pituitary surgery. Eighty-four RCTs were registered, of which some are ongoing. Trial registration rate differed per subspecialty. Funding was mostly from non-industry institutions (58.5%), but 25.6% of RCTs did not report funding sources. 36.4% of RCTs did not report a difference between surgical and non-operative treatment, 3.7% favored non-operative management. Primary and secondary outcome measures were changed in 13.2% and 34.2% of RCTs respectively and varied by subspecialty. 41.9% of RCTs subtracted ≥ 10% of the anticipated accrual and 12.9% of RCTs added ≥ 10%. 7.3% of registered RCTs were terminated, mostly due to too slow recruitment. Subspecialty, registration, funding, masking, population size, and changing outcome measures were not significantly associated with a reported benefit of surgery. High Jadad scores (≥ 4) were negatively associated with a demonstration of surgical benefit (P < 0.05).

CONCLUSIONS

Neurosurgical RCTs comparing surgical to non-operative treatment often find a benefit for surgical treatment. Changes to outcome measurements and anticipated accrual are common and funding sources are not always reported.

Efficacy and safety of combined androgen blockade with antiandrogen for advanced prostate cancer

Background

Combined androgen blockade (cab) is a promising treatment modality for prostate cancer (pca). In the present meta-analysis, we compared the efficacy and safety of first-line cab using an antiandrogen (aa) with castration monotherapy in patients with advanced pca.

Methods

PubMed, embase, Cochrane, and Google Scholar were searched for randomized controlled trials (rcts) published through 12 December 2016. Hazard ratios (hrs) with 95% confidence intervals (cis) were determined for primary outcomes: overall survival (os) and progression-free survival (pfs). Subgroup analyses were performed for Western compared with Eastern patients and use of a nonsteroidal aa (nsaa) compared with a steroidal aa (saa).

Results

Compared with castration monotherapy, cab using an aa was associated with significantly improved os (n = 14; hr: 0.90; 95% ci: 0.84 to 0.97; p = 0.003) and pfs (n = 13; hr: 0.89; 95% ci: 0.80 to 1.00; p = 0.04). No significant difference in os (p = 0.71) and pfs (p = 0.49) was observed between the Western and Eastern patients. Compared with castration monotherapy, cab using a nsaa was associated with significantly improved os (hr: 0.88; 95% ci: 0.82 to 0.95; p = 0.0009) and pfs (hr: 0.85; 95% ci: 0.73 to 0.98; p = 0.007)-a result that was not achieved with cab using a saa. The safety profiles of cab and monotherapy were similar in terms of adverse events, including hot flushes, impotence, and grade 3 or 4 events, with the exception of risk of diarrhea and liver dysfunction or elevation in liver enzymes, which were statistically greater with cab using an aa.

Conclusions

Compared with castration monotherapy, first-line cab therapy with an aa, especially a nsaa, resulted in significantly improved os and pfs, and had an acceptable safety profile in patients with advanced pca.

Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update

PURPOSE

Reporting guidelines (eg, Consolidated Standards of Reporting Trials [CONSORT] statement) are intended to improve reporting standards and enhance the transparency and reproducibility of research findings. Despite accessibility of such guidelines, researchers are not required to adhere to them. Our goal was to determine the current status of reporting quality in the medical literature and examine whether adherence of reporting guidelines has improved since the inception of reporting guidelines.

MATERIALS AND METHODS

Eight reporting guidelines, such as CONSORT, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), Quality of Reporting of Meta-analysis (QUOROM), STAndards for Reporting of Diagnostic accuracy (STARD), Animal Research: Reporting In Vivo Experiments (ARRIVE), Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) were examined. Our inclusion criteria included reviews published between January 1996 to September 2016 which investigated the adherence to reporting guidelines in the literature that addressed clinical trials, systematic reviews, observational studies, meta-analysis, diagnostic accuracy, economic evaluations, and preclinical animal studies that were in English. All reviews were found on Web of Science, Excerpta Medical Database (EMBASE), MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL).

RESULTS

Among the general searching of 26,819 studies by using the designed searching method, 124 studies were included post screening. We found that 87.9% of the included studies reported suboptimal adherence to reporting guidelines. Factors associated with poor adherence included non-pharmacological interventions, year of publication, and trials concluding with significant results. Improved adherence was associated with better study designs such as allocation concealment, random sequence, large sample sizes, adequately powered studies, multiple authorships, and being published in journals endorsing guidelines.

CONCLUSION

We conclude that the level of adherence to reporting guidelines remains suboptimal. Endorsement of reporting guidelines by journals is important and recommended.

Launching Effectiveness Research to Guide Practice in Neurosurgery: A National Institute Neurological Disorders and Stroke Workshop Report

This workshop addressed challenges of clinical research in neurosurgery. Randomized controlled clinical trials (RCTs) have high internal validity, but often insufficiently generalize to real-world practice. Observational studies are inclusive but often lack sufficient rigor. The workshop considered possible solutions, such as (1) statistical methods for demonstrating causality using observational data; (2) characteristics required of a registry supporting effectiveness research; (3) trial designs combining advantages of observational studies and RCTs; and (4) equipoise, an identified challenge for RCTs. In the future, advances in information technology potentially could lead to creation of a massive database where clinical data from all neurosurgeons are integrated and analyzed, ending the separation of clinical research and practice and leading to a new "science of practice."

Randomized controlled trials in neurosurgery: an observational analysis of trial discontinuation and publication outcome

OBJECTIVE

This study aimed to determine the trial discontinuation and publication rate of randomized controlled trials (RCTs) in neurosurgery.

METHODS

Trials registered from 2000 to 2012 were identified on the website clinicaltrials.gov using a range of key words related to neurosurgery. Any trials that were actively recruiting or had unknown status were excluded. Included trials were assessed for whether they were discontinued early on the clinicaltrials.gov database; this included trials identified as withdrawn, suspended, or terminated in the database. For included trials, a range of parameters was identified including the subspecialty, primary country, study start date, type of intervention, number of centers, and funding status. Subsequently, a systematic search for published peer-reviewed articles was undertaken. For trials that were discontinued early or were found to be unpublished, principal investigators were sent a querying email.

RESULTS

Sixty-four neurosurgical trials fulfilled our inclusion criteria. Of these 64, 26.6% were discontinued early, with slow or insufficient recruitment cited as the major reason (57%). Of the 47 completed trials, 14 (30%) remained unpublished. Discontinued trials showed a statistically significant higher chance of remaining unpublished (88%) compared with completed trials (p = 0.0002). Industry-funded trials had a higher discontinuation rate (31%) compared with non–industry-funded trials (23%), but this result did not reach significance (p = 0.57). Reporting of primary outcome measures was complete in 20 (61%) of 33 trials. For secondary outcome measures, complete reporting occurred in only 11 (33.3%) of 33.

CONCLUSIONS

More than a fifth (26.6%) of neurosurgical RCTs are discontinued early and almost a third of those that are completed remain unpublished. This result highlights significant waste of financial resources and clinical data.

Interpretation of CIs in clinical trials with non-significant results: systematic review and recommendations

OBJECTIVES

Interpretation of CIs in randomised clinical trials (RCTs) with treatment effects that are not statistically significant can distinguish between results that are 'negative' (the data are not consistent with a clinically meaningful treatment effect) or 'inconclusive' (the data remain consistent with the possibility of a clinically meaningful treatment effect). This interpretation is important to ensure that potentially beneficial treatments are not prematurely abandoned in future research or clinical practice based on invalid conclusions.

DESIGN

Systematic review of RCT reports published in 2014 in Annals of Internal Medicine, New England Journal of Medicine, JAMA, JAMA Internal Medicine and The Lancet (n=247).

RESULTS

85 of 99 articles with statistically non-significant results reported CIs for the treatment effect. Only 17 of those 99 articles interpreted the CI. Of the 22 articles in which CIs indicated an inconclusive result, only four acknowledged that the study could not rule out a clinically meaningful treatment effect.

CONCLUSIONS

Interpretation of CIs is important but occurs infrequently in study reports of trials with treatment effects that are not statistically significant. Increased author interpretation of CIs could improve application of RCT results. Reporting recommendations are provided.

Outcome Measures Used in Arthroplasty Trials: Systematic Review of the 2008 and 2013 Literature

OBJECTIVE

Previously published literature assessing the reporting of outcome measures used in joint replacement randomized controlled trials (RCT) has revealed disappointing results. It remains unknown whether international initiatives have led to any improvement in the quality of reporting and/or a reduction in the heterogeneity of outcome measures used. Our objective was to systematically assess and compare primary outcome measures and the risk of bias in joint replacement RCT published in 2008 and 2013.

METHODS

We searched MEDLINE, EMBASE, and CENTRAL for RCT investigating adult patients undergoing joint replacement surgery. Two authors independently identified eligible trials, extracted data, and assessed risk of bias using the Cochrane tool.

RESULTS

Seventy RCT (30 in 2008, 40 in 2013) met the eligibility criteria. There was no significant difference in the number of trials judged to be at low overall risk of bias (n = 6, 20%) in 2008 compared with 2013 [6 (15%); chi-square = 0.302, p = 0.75]. Significantly more trials published in 2008 did not specify a primary outcome measure (n = 25, 83%) compared with 18 trials (45%) in 2013 (chi-square = 10.6316, p = 0.001). When specified, there was significant heterogeneity in the measures used to assess primary outcomes.

CONCLUSION

While less than a quarter of trials published in both 2008 and 2013 were judged to be at low overall risk of bias, significantly more trials published in 2013 specified a primary outcome. Although this might represent a temporal trend toward improvement, the overall frequency of primary outcome reporting and the wide heterogeneity in primary outcomes reported remain suboptimal.

Framing of research question using the PICOT format in randomised controlled trials of venous ulcer disease: a protocol for a systematic survey of the literature

INTRODUCTION

Although venous ulcers have a great social and economic impact, there is a lack of evidence from randomised controlled trials (RCTs) to support appropriate management for this disease. Framing the research question using the Population; Intervention; Comparator; Outcome; Time frame (PICOT) format in RCTs can improve the quality of the research design.

OBJECTIVES

To evaluate how the PICOT format is used to frame a research question in reports of RCTs of venous ulcer disease and to determine the factors associated with better adherence to the PICOT format in framing the research question.

METHODS AND ANALYSES

We will conduct a systematic survey of RCTs on venous ulcers published in the National Institute of Health, PubMed database between January 2009 and May 2016. We will include all RCTs addressing therapeutic intervention for venous ulcer disease involving human subjects, and published in the English language. The selection process will be carried out in duplicate by two independent investigators. First, titles and abstracts will be screened, then full-text articles. We will examine whether the five elements of the PICOT format are used in formulating the research question and give a score between 0 and 5. The primary outcome will be the proportion of studies that have adequately reported all five PICOT elements.

DISSEMINATION

This will be the first survey to assess how the PICOT format is used to frame research questions on the management of venous ulcers in reports of RCTs. On completion, this review will be submitted to a peer-reviewed biomedical journal for publication and the findings will also be presented at scientific conferences.

A Practical Methodological Approach Towards Identifying Core Competencies in Medical Education Based on Literature Trends: A Feasibility Study Based on Vestibular Schwannoma Science

BACKGROUND

Competency-based medical education (CBME) is gaining momentum in postgraduate residency and fellowship training. While randomized trials, consensus statements, and practice guidelines can help delineate some of the core competencies for CBME, they are not applicable to all clinical scenarios.

OBJECTIVE

To propose and assess the feasibility of a practical methodology for addressing this issue using radiosurgery for vestibular schwannoma (VS) science as an example.

METHODS

The Web of Science electronic database was searched using relevant terms. A 3-step review of titles and abstracts was used. Studies were classified independently and in duplicate as either efficacy or effectiveness analyses. Cohen's kappa score was used to assess inter-rater agreement.

RESULTS

Overall, 1818 surgical and 943 radiosurgical publications were identified. The number of effectiveness studies surpassed that of efficacy studies in the late 1980s for surgical studies, and in the early-to-mid 1990s among radiosurgical studies. The publication rate was higher for radiosurgery in the mid 1990s, but it paralleled that of surgical studies beyond the early 2000s. Variations in this overall trend corresponded to the emergence of studies that assessed the role of endoscopy and the utility of dose reduction in radiosurgery.

CONCLUSION

We have confirmed the feasibility and accuracy of this objective methodological approach. By understanding how the peer-reviewed literature reflects actual practice interests, educators can tailor curricula to ensure that trainees remain current. While further validation studies are needed, this methodology can serve as a supplemental strategy for identifying additional core competencies in CBME.

Randomized controlled trials and neurosurgery: the ideal fit or should alternative methodologies be considered?

OBJECTIVE

Randomized-controlled trials (RCTs) are advocated to provide high-level medical evidence. However, in neurosurgery, there are barriers to conducting RCTs. The authors of this study sought to analyze the quality of neurosurgical RCTs since 2000 to determine the adequacy of their design and reporting.

METHODS

A search of the MEDLINE and EMBASE databases (2000-2014) was conducted. The medical subject heading (MeSH) terms used in the search included: "neurosurgery" OR "neurosurgical procedure," "brain neoplasms," "infarction" and "decompression," "carotid stenosis," "cerebral hemorrhage," and "spinal fusion." These studies were limited to RCTs, in humans, and in the English language. The Consolidated Standards for Reporting of Trials (CONSORT) and Jadad scales were used to assess the quality of RCT design and reporting. The standardized median times cited (median citations divided by years since publication) were used to assess impact. A pragmatic-explanatory continuum indicator summary-based scale was used to assess the design of the studies as primarily pragmatic or explanatory.

RESULTS

Sixty-one articles were identified, and the following subspecialties were the most common: vascular (23, 37%), followed by functional neurosurgery and neurooncology (both 13, 21%). The following nations were the primary leaders in RCTs: US (25 studies, 41%), Germany (8 studies, 13%), and the United Kingdom (7 studies, 11%). Median sample size was 100 (interquartile range [IQR] 41.5-279). The majority of the studies (40, 66%) had pragmatic objectives. The median number of times cited overall was 69 (IQR 20.5-193). The combined median CONSORT score was 36 (IQR 27.5-39). Blinding was most deficiently reported. Other areas with a relatively low quality of reporting were sample size calculation (34.2% of surgical, 38.5% of drug, and 20% of device studies), allocation concealment (28.9% of surgical, 23.1% of drug, and 50% of device studies), and protocol implementation (18.4% of surgical, 23% of drug, and 20% of device studies). The quality of reporting did not correlate with the study impact. All studies had a median Jadad score ≤ 3. Thirty-three pragmatic studies (83%) and 5 explanatory studies (25%) met the design objectives. All pragmatic studies based on drug and device trials met their objectives, while 74% of pragmatic surgical trials met their objectives.

CONCLUSIONS

The prevalence of neurosurgical RCTs is low. The quality of RCT design and reporting in neurosurgery is also low. Many study designs are not compatible with stated objectives. Pragmatic studies were more likely to meet design objectives. Given the role of RCTs as one of the highest levels of evidence, it is critical to improve on their methodology and reporting.

CONSORT in China: past development and future direction

The Consolidated Standards of Reporting Trials (CONSORT) Statement was published in 1996, and first introduced to China in 2001. Although CONSORT has been widely accepted in high-quality international journals, we still need to have more investigation on how many Chinese journals have adopted the CONSORT Statement, and whether the quality of reporting has improved. A systematic search of the “Instructions to authors” in all Chinese medical journals in China Academic Journals (CAJ) Full-text Database was conducted up to February 2012 and only 7 journals officially listed the requirements of the CONSORT Statement. The research articles about randomized controlled trials (RCTs) published in 2002, 2004, 2006, 2008, and 2010 from journals which had specifically adopted the CONSORT Statement, and from 30 top journals based on the Chinese Science Citation Index (CSCI) 2011 as the control group, were identified. The quality of both cohorts of articles was assessed using the revised CONSORT Checklist and Jadad scale. A total of 1221 Chinese medical journals was identified. Only seven journals stated clearly in the “Instructions to authors” that authors should adopt the CONSORT requirement in the clinical trial paper. None of these journals is among the control group in the CSCI 2011. In the selected years, a total of 171 articles from 7 journals which had adopted CONSORT and 232 articles in the control were identified as including RCT trials. The average scores according to the revised CONSORT Checklist were 29.47 for the CONSORT-adopting journals and 25.57 for the control group; while the average scores based on the Jadad scale were 2.53 for CONSORT-adopting journals and 1.97 for the control group. Few journals among Chinese medical journals have adopted the CONSORT Statement. The overall quality of RCT reports in the 7 journals which have adopted CONSORT was better than those in the top 30 journals which have not adopteded CONSORT. The quality of RCT reports in Chinese journals needs further improvement, and the CONSORT Statement could be a very helpful guideline.

Recent Randomized Controlled Trials in Otolaryngology

Objective

To assess recent trends in the prevalence and quality of reporting of randomized controlled trials (RCTs) in 4 otolaryngology journals.

Study Design

Methodology and reporting analysis.

Setting

Randomized controlled trials in 4 otolaryngology journals.

Subjects and Methods

All RCTs published from 2011 to 2013 in 4 major otolaryngology journals were examined for characteristics of study design, quality of design and reporting, and funding.

Results

Of 5279 articles published in 4 leading otolaryngology journals from 2011 to 2013, 189 (3.3%) were RCTs. The majority of RCTs were clinical studies (86%), with the largest proportion consisting of sinonasal topics (31%). Most interventions were medical (46%), followed by surgical (38%) and mixed (16%). In terms of quality, randomization method was reported in 54% of RCTs, blinding in 33%, and adverse events in 65%. Intention-to-treat analysis was used in 32%; P values were reported in 87% and confidence intervals in 10%. Research funding was most often absent or not reported (55%), followed by not-for-profit (25%).

Conclusions

Based on review of 4 otolaryngology journals, RCTs are still a small proportion of all published studies in the field of otolaryngology. There seem to be trends toward improvement in quality of design and reporting of RCTs, although many quality features remain suboptimal. Practitioners both designing and interpreting RCTs should critically evaluate RCTs for quality.

The reporting quality of parallel randomised controlled trials in ophthalmic surgery in 2011: a systematic review

PURPOSE

Randomised controlled trials (RCTs) represent a gold standard for evaluating therapeutic interventions. However, poor reporting clarity can prevent readers from assessing potential bias that can arise from a lack of methodological rigour. The Consolidated Standards of Reporting Trials statement for non-pharmacological interventions 2008 (CONSORT NPT) was developed to aid reporting. RCTs in ophthalmic surgery pose particular challenges in study design and implementation. We aim to provide the first assessment of the compliance of RCTs in ophthalmic surgery to the CONSORT NPT statement.

METHOD

In August 2012, the Medline database was searched for RCTs in ophthalmic surgery reported between 1 January 2011 and 31 December 2011. Results were searched by two authors and relevant papers selected. Papers were scored against the 23-item CONSORT NPT checklist and compared against surrogate markers of paper quality. The CONSORT score was also compared between different RCT designs.

RESULTS

In all, 186 papers were retrieved. Sixty-five RCTs, involving 5803 patients, met the inclusion criteria. The mean CONSORT score was 8.9 out of 23 (39%, range 3.0-14.7, SD 2.49). The least reported items related to the title and abstract (1.6%), reporting intervention adherence (3.1%), and interpretation of results (4.7%). No significant correlation was found between CONSORT score and journal impact factor (R=0.14, P=0.29), number of authors (R=0.01, P=0.93), or whether the RCT used paired-eye, one-eye, or two-eye designs in their randomisation (P=0.97).

CONCLUSIONS

The reporting of RCTs in ophthalmic surgery is suboptimal. Further work is needed by trial groups, funding agencies, authors, and journals to improve reporting clarity.

Gamma Knife Radiosurgery for Cerebellopontine Angle Meningiomas

BACKGROUND:

Resection of cerebellopontine angle (CPA) meningiomas may result in significant neurological morbidity. Radiosurgery offers a minimally invasive alternative to surgery.

OBJECTIVE:

To evaluate, in a multicenter cohort study, the outcomes of patients harboring CPA meningiomas who underwent Gamma Knife radiosurgery (GKRS).

METHODS:

From 7 institutions participating in the North American Gamma Knife Consortium, 177 patients with benign CPA meningiomas treated with GKRS and at least 6 months radiologic follow-up were included for analysis. The mean age was 59 years and 84% were female. Dizziness or imbalance (48%) and cranial nerve (CN) VIII dysfunction (45%) were the most common presenting symptoms. The median tumor volume and prescription dose were 3.6 cc and 13 Gy, respectively. The mean radiologic and clinical follow-up durations were 47 and 46 months, respectively. Multivariate regression analyses were performed to identify the predictors of tumor progression and neurological deterioration.

RESULTS:

The actuarial rates of progression-free survival at 5 and 10 years were 93% and 77%, respectively. Male sex (P = .014), prior fractionated radiation therapy (P = .010), and ataxia at presentation (P = .002) were independent predictors of tumor progression. Symptomatic adverse radiation effects and permanent neurological deterioration were observed in 1.1% and 9% of patients, respectively. Facial spasms at presentation (P = .007) and lower maximal dose (P = .011) were independently associated with neurological deterioration.

CONCLUSION:

GKRS is an effective therapy for CPA meningiomas. Depending on the patient and tumor characteristics, radiosurgery can be an adjuvant treatment to initial surgical resection or a standalone procedure that obviates the need for resection in most patients.

Proposal for evaluating the quality of reports of surgical interventions in the treatment of trigeminal neuralgia: the Surgical Trigeminal Neuralgia Score

OBJECT

The aim of this study was to design a checklist with a scoring system for reporting on studies of surgical interventions for trigeminal neuralgia (TN) and to validate it by a review of the recent literature.

METHODS

A checklist with a scoring system, the Surgical Trigeminal Neuralgia Score (STNS), was devised partially based on the validated STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) criteria and customized for TN after a literature review and then applied to a series of articles. These articles were identified using a prespecified MEDLINE and Embase search covering the period from 2008 to 2010. Of the 584 articles found, 59 were studies of interventional procedures for TN that fulfilled the inclusion criteria and 56 could be obtained in full. The STNS was then applied independently by 3 of the authors.

RESULTS

The maximum STNS came to 30, and was reliable and reproducible when used by the 3 authors who performed the scoring. The range of scores was 6-23.5, with a mean of 14 for all the journals. The impact factor scores of the journals in which the papers were published ranged from 0 to 4.8. Twenty-four of the studies were published in the Journal of Neurosurgery or in Neurosurgery. Studies published in neurosurgical journals ranked higher on the STNS scale than those published in nonneurosurgical journals. There was no statistically significant correlation between STNS and impact factors. Stereotactic radiosurgery (n = 25) and microvascular decompression (n = 15) were the most commonly reported procedures. The diagnostic criteria were stated in 35% of the studies, and 4 studies reported subtypes of TN. An increasing number of studies (46%) used the recommended Kaplan-Meier methodology for pain survival outcomes. The follow-up period was unclear in 8 studies, and 26 reported follow-ups of more than 5 years. Complications were reported fairly consistently but the temporal course was not always indicated. Direct interview, telephone conversation, and questionnaires were used to measure outcomes. Independent assessment of outcome was only clearly stated in 7 studies. Only 2 studies used the 36-Item Short Form Health Survey to measure quality of life and 4 studies reported on the severity of preoperative pain. The Barrow Neurological Institute pain questionnaire was the most commonly used outcome measure (n = 13), followed by the visual analog scale.

CONCLUSIONS

Similar to the STROBE criteria that provide a checklist of items that should be included in reports of observational studies in general, the authors' suggested checklist for the STNS could help editors and reviewers ensure that quality reports are published, and could prove useful for colleagues when reporting their results specifically on the surgical management of TN. It would help the patient and clinicians make a decision about selecting the appropriate neurosurgical procedure.

Barriers to investigator-initiated deep brain stimulation and device research

The success of device-based research in the clinical neurosciences has overshadowed a critical and emerging problem in the biomedical research environment in the United States. Neuroprosthetic devices, such as deep brain stimulation (DBS), have been shown in humans to be promising technologies for scientific exploration of neural pathways and as powerful treatments. Large device companies have, over the past several decades, funded and developed major research programs. However, both the structure of clinical trial funding and the current regulation of device research threaten investigator-initiated efforts in neurologic disorders. The current atmosphere dissuades clinical investigators from pursuing formal and prospective research with novel devices or novel indications. We review our experience in conducting a federally funded, investigator-initiated, device-based clinical trial that utilized DBS for thalamic pain syndrome. We also explore barriers that clinical investigators face in conducting device-based clinical trials, particularly in early-stage studies or small disease populations. We discuss 5 specific areas for potential reform and integration: (1) alternative pathways for device approval; (2) eliminating right of reference requirements; (3) combining federal grant awards with regulatory approval; (4) consolidation of oversight for human subjects research; and (5) private insurance coverage for clinical trials. Careful reformulation of regulatory policy and funding mechanisms is critical for expanding investigator-initiated device research, which has great potential to benefit science, industry, and, most importantly, patients.

Methodology and reporting of meta-analyses in the neurosurgical literature

OBJECT

Neurosurgeons are inundated with vast amounts of new clinical research on a daily basis, making it difficult and time-consuming to keep up with the latest literature. Meta-analysis is an extension of a systematic review that employs statistical techniques to pool the data from the literature in order to calculate a cumulative effect size. This is done to answer a clearly defined a priori question. Despite their increasing popularity in the neurosurgery literature, meta-analyses have not been scrutinized in terms of reporting and methodology.

METHODS

The authors performed a literature search using PubMed/MEDLINE to locate all meta-analyses that have been published in the JNS Publishing Group journals (Journal of Neurosurgery, Journal of Neurosurgery: Pediatrics, Journal of Neurosurgery: Spine, and Neurosurgical Focus) or Neurosurgery. Accepted checklists for reporting (PRISMA) and methodology (AMSTAR) were applied to each meta-analysis, and the number of items within each checklist that were satisfactorily fulfilled was recorded. The authors sought to answer 4 specific questions: Are meta-analyses improving 1) with time; 2) when the study met their definition of a meta-analysis; 3) when clinicians collaborated with a potential expert in meta-analysis; and 4) when the meta-analysis was the only focus of the paper?

RESULTS

Seventy-two meta-analyses were published in the JNS Publishing Group journals and Neurosurgery between 1990 and 2012. The number of published meta-analyses has increased dramatically in the last several years. The most common topics were vascular, and most were based on observational studies. Only 11 papers were prepared using an established checklist. The average AMSTAR and PRISMA scores (proportion of items satisfactorily fulfilled divided by the total number of eligible items in the respective instrument) were 31% and 55%, respectively. Major deficiencies were identified, including the lack of a comprehensive search strategy, study selection and data extraction, assessment of heterogeneity, publication bias, and study quality. Almost one-third of the papers did not meet our basic definition of a meta-analysis. The quality of reporting and methodology was better 1) when the study met our definition of a meta-analysis; 2) when one or more of the authors had experience or expertise in conducting a meta-analysis; 3) when the meta-analysis was not conducted alongside an evaluation of the authors' own data; and 4) in more recent studies.

CONCLUSIONS

Reporting and methodology of meta-analyses in the neurosurgery literature is excessively variable and overall poor. As these papers are being published with increasing frequency, neurosurgical journals need to adopt a clear definition of a meta-analysis and insist that they be created using checklists for both reporting and methodology. Standardization will ensure high-quality publications.

The evidence-based medicine paradigm: where are we 20 years later? Part 2

In Part 2, we discuss the challenges of keeping up with the 'literature,' evidence-based medicine (EBM) in emerging economies and the Neurosciences, and two recent approaches to classifying evidence. We conclude by summarizing information from Parts 1 and 2 which suggest the need to critically re-appraise core elements of the EBM paradigm: (1) the hierarchical ranking of evidence, (2) randomized controlled trials or systematic reviews as the gold standard for all clinical questions or situations, (3) the statistical tests that have become integral to the 'measurements' for analyzing evidence, and (4) re-incorporating a role for evidence from basic sciences and pathophysiology. An understanding of how cognitive processes influence clinical decisions is also necessary to improve evidence-based practice. Emerging economies may have to modify the design and conduct of clinical research to their settings. Like all paradigms, EBM must keep improving with input from the grassroots to remain beneficial.

Assessment of the reporting quality of randomized controlled trials on the treatment of diabetes mellitus with traditional chinese medicine: a systematic review

BACKGROUND

After the publication of the CONSORT 2010 statement, few studies have been conducted to assess the reporting quality of randomized clinical trials (RCTs) on treatment of diabetes mellitus with Traditional Chinese Medicine (TCM) published in Chinese journals.

OBJECTIVE

To investigate the current situation of the reporting quality of RCTs in leading medical journals in China with the CONSORT 2010 statement as criteria.

METHODS

The China National Knowledge Infrastructure (CNKI) electronic database was searched for RCTs on the treatment of diabetes mellitus with TCM published in the Journal of Traditional Chinese Medicine, Chinese Journal of Integrated Traditional & Western Medicine, and the China Journal of Chinese Materia Medica from January to December 2011. We excluded trials reported as "animal studies", "in vitro studies", "case studies", or "systematic reviews". The CONSORT checklist was applied by two independent raters to evaluate the reporting quality of all eligible trials after discussing and comprehending the items thoroughly. Each item in the checklist was graded as either "yes" or "no" depending on whether it had been reported by the authors.

RESULTS

We identified 27 RCTs. According to the 37 items in the CONSORT checklist, the average reporting percentage was 45.0%, in which the average reporting percentage for the "title and abstract", the "introduction", the "methods", the "results", the "discussion" and the "other information" was 33.3%, 88.9%, 36.4%, 54.4%, 71.6% and 14.8%, respectively. In the Journal of Traditional Chinese Medicine, Chinese Journal of Integrated Traditional & Western Medicine, and the China Journal of Chinese Materia Medica the average reporting percentage was 42.2%, 56.8%, and 46.0%, respectively.

CONCLUSIONS

The reporting quality of RCTs in these three journals was insufficient to allow readers to assess the validity of the trials. We recommend that editors require authors to use the CONSORT statement when reporting their trial results as a condition of publication.

A systematic scoping review of adherence to reporting guidelines in health care literature

Background

Reporting guidelines have been available for the past 17 years since the inception of the Consolidated Standards of Reporting Trials statement in 1996. These guidelines were developed to improve the quality of reporting of studies in medical literature. Despite the widespread availability of these guidelines, the quality of reporting of medical literature remained suboptimal. In this study, we assess the current adherence practice to reporting guidelines; determine key factors associated with better adherence to these guidelines; and provide recommendations to enhance adherence to reporting guidelines for future studies.

Methods

We undertook a systematic scoping review of systematic reviews of adherence to reporting guidelines across different clinical areas and study designs. We searched four electronic databases (Cumulative Index to Nursing and Allied Health Literature, Web of Science, Embase, and Medline) from January 1996 to September 2012. Studies were included if they addressed adherence to one of the following guidelines: Consolidated Standards of Reporting Trials (CONSORT), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), Quality of Reporting of Meta-analysis (QUOROM), Transparent Reporting of Evaluations with Nonrandomized Designs (TREND), Meta-analysis Of Observational Studies in Epidemiology (MOOSE) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). A protocol for this study was devised. A literature search, data extraction, and quality assessment were performed independently by two authors in duplicate. This study reporting follows the PRISMA guidelines.

Results

Our search retrieved 5159 titles, of which 50 were eligible. Overall, 86.0% of studies reported suboptimal levels of adherence to reporting guidelines. Factors associated with better adherence included journal impact factor and endorsement of guidelines, publication date, funding source, multisite studies, pharmacological interventions and larger studies.

Conclusion

Reporting guidelines in the clinical literature are important to improve the standards of reporting of clinical studies; however, adherence to these guidelines remains suboptimal. Action is therefore needed to enhance the adherence to these standards. Strategies to enhance adherence include journal editorial policies endorsing these guidelines.

Methodological and ethical quality of randomized controlled clinical trials in gastrointestinal surgery

BACKGROUND

The randomized controlled trial (RCT) is the gold standard tool used to evaluate therapeutic interventions. Methodological and ethical aspects should be adequately reported to enable readers to make informed and justified judgments regarding the validity of a trial and the treatment effectiveness.

OBJECTIVE

The aim of this study was to evaluate the methodological and ethical qualities of randomized clinical trials in gastrointestinal surgery and to assess the relationship between these two qualities.

STUDY DESIGN

All of the articles chosen for review reported on phase III randomized controlled gastrointestinal surgical trials were published in 12 international journals during 2006 and 2007. The eligible studies were identified, selected, and then evaluated based on a broad set of predetermined criteria. The methodological quality was evaluated using the Jadad scale, and the ethical quality was evaluated using the Berdeu score.

RESULTS

The mean Jadad score was 9.7 ± 1.78. The methodological quality was insufficient in 64 RCTs (37.4 %; Jadad score <9). The mean Berdeu score was 0.36 ± 0.08. The journal impact factor, number of randomized patients, and number of centers correlated with the outcome of the Jadad score, and the journal impact factor, industry funding, and year in which the trial began correlated with the outcome of the Berdeu score. Informed consent from patients was not obtained in 7 % (n = 12) of the RCTs, and research ethics committee approval was not mentioned in 14.6 % (n = 25) of the RCTs.

CONCLUSIONS

The reporting of gastrointestinal surgery RCTs is less than optimal. In our study, the trials of higher methodological quality were more likely to provide information about their ethical aspects. These results suggest the need for more attention to be paid to the conduct of clinical research and the reporting of ethical aspects. The appropriation of the ethical rules by surgeons involved in human clinical trials could improve the methodology and reporting of RCTs in gastrointestinal surgery.

Standards for reporting randomized controlled trials in medical informatics: a systematic review of CONSORT adherence in RCTs on clinical decision support

INTRODUCTION

The Consolidated Standards for Reporting Trials (CONSORT) were published to standardize reporting and improve the quality of clinical trials. The objective of this study is to assess CONSORT adherence in randomized clinical trials (RCT) of disease specific clinical decision support (CDS).

METHODS

A systematic search was conducted of the Medline, EMBASE, and Cochrane databases. RCTs on CDS were assessed against CONSORT guidelines and the Jadad score.

RESULT

32 of 3784 papers identified in the primary search were included in the final review. 181 702 patients and 7315 physicians participated in the selected trials. Most trials were performed in primary care (22), including 897 general practitioner offices. RCTs assessing CDS for asthma (4), diabetes (4), and hyperlipidemia (3) were the most common. Thirteen CDS systems (40%) were implemented in electronic medical records, and 14 (43%) provided automatic alerts. CONSORT and Jadad scores were generally low; the mean CONSORT score was 30.75 (95% CI 27.0 to 34.5), median score 32, range 21-38. Fourteen trials (43%) did not clearly define the study objective, and 11 studies (34%) did not include a sample size calculation. Outcome measures were adequately identified and defined in 23 (71%) trials; adverse events or side effects were not reported in 20 trials (62%). Thirteen trials (40%) were of superior quality according to the Jadad score (≥3 points). Six trials (18%) reported on long-term implementation of CDS.

CONCLUSION

The overall quality of reporting RCTs was low. There is a need to develop standards for reporting RCTs in medical informatics.