Enhancing the quality of evidence, comparability, and reproducibility in brain arteriovenous malformations treated with open surgery research: a systematic review and proposal of a reporting guideline for surgical and clinical outcomes

Brain Arteriovenous Malformations (bAVMs) are rare but high-risk developmental anomalies of the vascular system. Microsurgery through craniotomy is believed to be the mainstay standard treatment for many grades of bAVMs. However, a significant challenge emerges in the existing body of clinical studies on open surgery for bAVMs: the lack of reproducibility and comparability. This study aims to assess the quality of studies reporting clinical and surgical outcomes for bAVMs treated by open surgery and develop a reporting guideline checklist focusing on essential elements to ensure comparability and reproducibility. This is a systematic literature review that followed the PRISMA guidelines with the search in Medline, Embase, and Web of Science databases, for studies published between January 1, 2018, and December 1, 2023. Included studies were scrutinized focusing on seven domains: (1) Assessment of How Studies Reported on the Baseline Characteristics of the Patient Sample; (2) Assessment and reporting on bAVMs grading, anatomical characteristics, and radiological aspects; (3) Angioarchitecture Assessment and Reporting; (4) Reporting on Pivotal Concepts Definitions; (5) Reporting on Neurosurgeon(s) and Staff Characteristics; (6) Reporting on Surgical Details; (7) Assessing and Reporting Clinical and Surgical Outcomes and AEs. A total of 47 studies comprising 5,884 patients were included. The scrutiny of the studies identified that the current literature in bAVM open surgery is deficient in many aspects, ranging from fundamental pieces of information of methodology to baseline characteristics of included patients and data reporting. Included studies demonstrated a lack of reproducibility that hinders building cumulative evidence. A bAVM Open Surgery Reporting Guideline with 65 items distributed across eight domains was developed and is proposed in this study aiming to address these shortcomings. This systematic review identified that the available literature regarding microsurgery for bAVM treatment, particularly in studies reporting clinical and surgical outcomes, lacks rigorous scientific methodology and quality in reporting. The proposed bAVM Open Surgery Reporting Guideline covers all essential aspects and is a potential solution to address these shortcomings and increase transparency, comparability, and reproducibility in this scenario. This proposal aims to advance the level of evidence and enhance knowledge regarding the Open Surgery treatment for bAVMs.

Wheel Replacing Pyramid: Better Paradigm Representing Totality of Evidence-Based Medicine

Background

Evidence-based medicine (EBM), as originally conceived, used all types of peer-reviewed evidence to guide medical practice and decision-making. During the SARS-CoV-2 Coronavirus disease (COVID-19) pandemic, the standard usage of EBM, modeled by the Evidence-Based Medicine Pyramid, undermined EBM by incorrectly using pyramid levels to assign relative quality. The resulting pyramid-based thinking is biased against reports both in levels beneath randomized control trials (RCTs) and those omitted from the pyramid entirely. Thus, much of the evidence was ignored. Our desire for a more encompassing and effective medical decision-making process to apply to repurposed drugs led us to develop an alternative to the EBM Pyramid for EBM. Herein, we propose the totality of evidence (T-EBM) wheel.

Objectives

To create an easily understood graphic that models EBM by incorporating all peer-reviewed evidence that applies to both new and repurposed medicines, and to demonstrate its potential utility using ivermectin as a case study.

Methods

The graphics were produced using Microsoft Office Visio Professional 2003 except for part of the T-EBM wheel sunburst chart, which was produced using Microsoft 365 Excel. For the case study, PubMed® was used by searching for peer-reviewed reports containing "ivermectin" and either "covid" or "sars" in the title. Reports were filtered for those using ivermectin-based protocols in the treatment of COVID-19. The resulting 265 reports were evaluated for their study design types and treatment outcomes. The three-ringed graphical T-EBM wheel was composed of two inner rings showing all types of reports and an outer ring showing outcomes for each type.

Findings-Conclusions

The T-EBM wheel avoids the biases of the EBM Pyramid and includes all types of reports in the pyramid along with reports such as population and mechanistic studies. In both early and late stages of medical emergencies, pyramid-based thinking may overlook indications of efficacy in regions of the T-EBM wheel beyond RCTs. This is especially true when searching for ways to prevent and treat a novel disease with repurposed therapeutics before RCTs, safety assessments, and mechanisms of action of novel therapeutics are established. As such, T-EBM Wheels should replace the EBM Pyramids in medical decision-making and education. T-EBM Wheels can be expanded upon by implementing multiple outer rings, one for each different kind of outcome (efficacy, safety, etc.). A T-EBM Wheel can be created for any proprietary or generic medicine. The ivermectin (IVM) T-EBM Wheel displays the efficacy of IVM-based treatments of COVID-19 in a color-coded graphic, visualizing each type of evidence and the proportions of each of their outcomes (positive, inconclusive, negative).

Impact of Exercise Interventions on Sustained Brain Health Outcomes in Frail Older Individuals: A Comprehensive Review of Systematic Reviews

Several systematic review studies highlight exercise's positive impact on brain health outcomes for frail individuals. This study adopts a Comprehensive Review of reviews (CRs) approach to amalgamate data from existing reviews, focusing on exercise's influence on brain health outcomes in older frail and pre-frail adults. The methodology involves a thorough search of Portuguese, Spanish, and English-indexed databases (i.e., Ebsco Health, Scielo, ERIC, LILACS, Medline, Web of Science, SportDiscus) from 1990 to 2022, with the AMSTAR-2 tool assessing evidence robustness. The search terms "physical exercise", "elderly frail", and "systematic review" were employed. Results: Out of 12 systematically reviewed studies, four presented high-quality (with metanalyses), while eight exhibit critically low quality. Positive trends emerge in specific cognitive and neuromotor aspects, yet challenges persist in psychosocial domains, complex cognitive tasks, and ADL outcomes. This study yields reasonable and promising evidence regarding exercise's influence on quality of life and depression in frail older individuals. However, the impact on biochemical markers remains inconclusive, emphasizing the need for standardized methodologies. Conclusions: The findings highlight the importance of acknowledging methodological nuances for clinicians and policymakers when translating these results into impactful interventions for aging populations. This emphasizes the necessity for a comprehensive and customized approach to exercise interventions aimed at fostering the sustainability of overall well-being in older individuals, aligning with United Nations Sustainable Development Goal 3.

Quality of athlete screening for high-risk cardiovascular conditions-A systematic review

BACKGROUND

Sudden cardiac death (SCD) is the leading medical cause of death in athletes. To prevent SCD, screening for high-risk cardiovascular conditions (HRCC) is recommended. Screening strategies are based on a limited number of studies and expert consensus. However, evidence and efficacy of athlete HRCC screening is unclear.

OBJECTIVE

To determine methodological quality and quality of evidence of athlete screening, and screening efficacy to detect HRCC in a systematic review.

METHODS

We performed a systematic search of Medline, Embase, Scopus and Cochrane Library up to June 2021. We included articles containing original data of athlete cardiovascular screening, providing details of screening strategies, test results and HRCC detection. We assessed methodological quality of the included articles by QUADAS-2, quality of evidence of athlete HRCC screening by GRADE, and athlete HRCC screening efficacy by SWiM.

RESULTS

Of 2720 citations, we included 33 articles (1991-2018), comprising 82 417 athletes (26.7% elite, 73.4% competitive, 21.7% women, 75.2% aged ≤35). Methodological quality was 'very low' (33 articles), caused by absence of data blinding and inappropriate statistical analysis. Quality of evidence was 'very low' (33 articles), due to observational designs and population heterogeneity. Screening efficacy could not be reliably established. The prevalence of HRCC was 0.43% with false positive rate (FPR) 13.0%.

CONCLUSIONS

Methodological quality and quality of evidence on athlete screening are suboptimal. Efficacy could not be reliably established. The prevalence of screen detected HRCC was very low and FPR high. Given the limitations of the evidence, individual recommendations need to be prudent.

Review time of oncology drugs and its underlying factors: an exploration in China

Introduction: How the launch delay of drugs and other factors of interest can influence the length of the review period by drug agencies is still unknown, and understanding this can help better strike the trade-off related to review speed. Methods: We included all new oncology drug applications submitted to China's National Medical Product Administration (NMPA) between 1 January 2018 and 31 December 2021, and ultimately succeeded in achieving marketing approval. For each drug, the length of the NMPA review process and other major characteristics were collected, including the registration class, approval class, priority review designation, and launch delay relative to the United States, as well as the number of patients enrolled, comparator, and primary endpoint of the pivotal trials supporting the approval. Linear regression model was employed to analyze the effects of factors of interest on the NMPA review time. Results: From 2018 to 2021, NMPA received 137 oncology applications that were ultimately approved. Half of the approvals [76 (55.5%)] were first licensed in the US, leaving a median launch delay of 2.71 years (IQR, 1.03-5.59) in China. In the pivotal studies, the median enrollment was 361 participants (IQR, 131-682), and the use of control groups [90 (65.7%)] and surrogate endpoints [101 (73.7%)] was prevalent. The median review time was 304 days (IQR, 253-376). Multivariate analysis for log-transformed review time showed that larger enrollment (> 92) was associated with a drop of 20.55% in review time (coefficient = -0.230; 95% CI, -0.404 to -0.055; p = 0.010); and a short delay (0 < delay ≤ 1.95 years) was associated with a drop of 17.63% in review time (coefficient = -0.194; 95% CI, -0.325 to -0.062; p = 0.004). Discussion: The short launch delay relative to the US was one important driver to the review speed of NMPA, which might suggest its latent regulatory reliance on the other global regulator during the post-marketing period when new information on the drug's clinical benefit was still lacking.

Hierarchy of Evidence: An Appraisal Tool for Weighting the Evidence in Healthcare Design Research Based on Internal Validity

PURPOSE

This inquiry aims to develop an appraisal tool to offer greater granularity in weighing evidence in the field of healthcare design research.

BACKGROUND

In evidence-based design (EBD), the goal is to implement interventions that result in a meaningful and optimal effect based on current best evidence. Although multiple appraisal tools (many adopted from medical disciplines) have been instrumental in evaluating studies in the field of healthcare design research to identify the best evidence, they do not necessarily consider the unique contexts of healthcare design research, and methodologies appropriate to the field.

METHODS

Five basic types of studies are ranked based on the level of confidence that they offer regarding the estimate of an effect: (1) meta-analysis studies, (2) causal studies, (3) correlational studies, (4) descriptive studies, and (5) anecdotal evidence. Causal studies are further divided into four levels based on the interaction of two factors: (1) type of intervention and (2) groups' equivalency and extraneous variable control.

RESULTS

An eight-level hierarchy of evidence for healthcare design research is proposed that is expected to improve upon previous hierarchies in three major ways: (a) including research methods that are more relevant to healthcare design research, (b) enhancing evaluation accuracy and reliability by providing a clearer definition of studies based on their key components rather than using study labels alone, and (c) distinguishing different levels of evidence, particularly in causal studies.

CONCLUSIONS

The proposed appraisal tool is developed specifically for EBD by reflecting on the unique context of healthcare design research and practice.

The Composite Quality Score (CQS) as an Appraisal Tool for Prospective, Controlled Clinical Therapy Trials: Rationale and Current Evidence

BACKGROUND

Current evidence appraisal concepts, such as the Assessment, Development and Evaluation (GRADE) approach and Cochrane's Risk of Bias (RoB) tool, rely on assumptions related to the classic problem of inductive reasoning and may suffer from insufficient inter-rater reliability.

DISCUSSION

The Composite Quality Score (CQS) has emerged as a possible trial appraisal tool that does not rely on inductive assumptions and has been shown to be of potentially very high inter-rater reliability.

CONCLUSION

Although the current CQS concept is still under development, its current evidence is encouraging and justifies further study. This article presents the rationale and currently available research concerning the CQS and shows where further research is required.

The Effectiveness of Interventions to Increase Participation and Physical Activities in Parks: A Systematic Review of the Literature

Although a variety of interventions in many countries have been developed to increase park use and to improve public participation in physical activities in parks, knowledge of their overall effectiveness is lacking. A systematic literature review was undertaken to further understand the effectiveness of various interventions that aim to increase the use of parks and physical activity participation in parks. This systematic review utilized the standard Preferred Reporting Items for Systematic Reviews and Meta-Analysis procedure, and of the 3801 studies identified, 60 studies were reviewed in-depth, with 44 studies assessed for quality of evidence and risk of bias. Most of the 44 reviewed studies found that various interventions had positive impacts on park use and physical activity participation in parks. Interventions were classified into both demand and supply, with more studies focusing on the supply side. The strongest evidence on the effectiveness of various interventions was found for park prescriptions; safe access to parks; playgrounds, and park renovation and renewal/design. However, the assessment of quality of evidence and risk of bias showed that most studies suffer from potential biases and evidence weaknesses, suggesting a need to further establish external validity.

Cost-effectiveness of clinical interventions for delirium: A systematic literature review of economic evaluations

OBJECTIVE

Little is known about the economic value of clinical interventions for delirium. This review aims to synthesise and appraise available economic evidence, including resource use, costs, and cost-effectiveness of interventions for reducing, preventing, and treating delirium.

METHODS

Systematic review of published and grey literature on full and partial economic evaluations. Study quality was assessed using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS).

RESULTS

Fourteen economic evaluations (43% full, 57% partial) across nine multicomponent and nonpharmacological intervention types met inclusion criteria. The intervention costs ranged between US$386 and $553 per person in inpatient settings. Multicomponent delirium prevention intervention and the Hospital Elder Life Program (HELP) reported statistically significant cost savings or cost offsets somewhere else in the health system. Cost savings related to inpatient, outpatient, and out-of-pocket costs ranged between $194 and $6022 per person. The average CHEERS score was 74% (±SD 10%).

CONCLUSION

Evidence on a joint distribution of costs and outcomes of delirium interventions was limited, varied and of generally low quality. Directed expansion of health economics towards the evaluation of delirium care is necessary to ensure effective implementation that meets patients' needs and is cost-effective in achieving similar or better outcomes for the same or lower cost.

GRADE summary of findings tables Enhanced Understanding of Values and Preferences Evidence

OBJECTIVE

We conceptualize patient values and preferences as the relative importance of health outcomes (RIO) which are often obtained through utility elicitation research. A transparent and structured approach to present synthesized RIO evidence and the certainty of this evidence is needed. This study aims to adapt the summary of findings (SoF) table to describe the RIO.

STUDY DESIGN AND SETTING

We performed three interactive workshops with a protype version of the SoF table for RIO evidence adapted from the SoF table for intervention effects. We then tested the new format through semi-structured interviews with professionals who interpret RIO evidence (e.g., systematic review authors and guideline developers).

RESULTS

We adapted the SoF table for the presentation of RIO evidence. This SoF table may be easy to use but bears one risk: some participants misunderstood the utility information and the variability around the RIO. We added a visual analogue scale to clarify the concept of utilities.

CONCLUSION

Through a multi-stage process including brainstorming sessions and interviews, we adapted the SoF table to present RIO evidence. This table may enhance understanding of evidence synthesis of values and preferences, facilitating the incorporation of this type of evidence in decision making.

Skip pattern approach toward the early access of innovative anticancer drugs

BACKGROUND

With the rapid development of innovative anticancer treatments, the optimization of tools able to accelerate the access of new drugs to the market by the regulatory authority is a major issue. The aim of the project was to propose a reliable methodological pathway for the assessment of clinical value of new therapeutic innovative options, to objectively identify drugs which deserve early access (EA) priority for solid and possibly in other cancer scenarios, such as the hematological ones.

MATERIALS AND METHODS

After a comprehensive review of the European Public Assessment Report of 21 drugs, to which innovation had previously been attributed by the Italian Medicines Agency (Agenzia Italiana del Farmaco, AIFA), an expert panel formulated an algorithm for the balanced use of three parameters: Unmet Medical Need (UMN) according to AIFA criteria, Added Benefit (AB) according to the European Society for Medical Oncology's Magnitude of Clinical Benefit Scale (ESMO-MCBS) criteria and Quality of Evidence (QE) assessed by the Grades of Recommendation Assessment, Development and Evaluation (GRADE) method. By sequentially combining the above indicators, a final priority status (i.e. EA or not) was obtained using the skip pattern approach (SPA).

RESULTS

By applying the SPA to the non-curative setting in solid cancers, the EA status was obtained by 5 out of 14 investigated drugs (36%); by enhancing the role of some categories of the UMN, additional 4 drugs, for a total of 9 (64%), reached the EA status: 2 and 3 drugs were excluded for not achieving an adequate score according to AB and QE criteria, respectively. For hematology cancer, only the UMN criteria were found to be adequate.

CONCLUSIONS

The use of this model may represent a reliable tool for assessment available to the various stakeholders involved in the EA process and may help regulatory agencies in a more comprehensive and objective definition of new treatments' value in these contexts. Its generalizability in other national contexts needs further evaluation.

Professional judgement in clinical practice (part 2): knowledge into practice

RATIONALE, AIMS, AND OBJECTIVES

Though strong evidence-based medicine is assertive in its claims, an insufficient theoretical basis and patchwork of arguments provide a good case that rather than introducing a new paradigm, EBM is resisting a shift to actually revolutionary complexity theory and other emergent approaches. This refusal to pass beyond discredited positivism is manifest in strong EBM's unsuccessful attempts to continually modify its already inadequate previous modifications, as did the defenders of the Ptolemaic astronomical model who increased the number of circular epicycles until the entire epicycle-deferent system proved untenable.

METHODS

Narrative Review.

RESULTS

The analysis in Part 1 of this three part series showed epistemological confusion as strong EBM plays the discredited positivistic tradition out to the end, thus repeating in a medical sphere and vocabulary the major assumptions and inadequacies that have appeared in the trajectory of modern science. Paper 2 in this series examines application, attending to strong EBM's claim of direct transferability of EBM research findings to clinical settings and its assertion of epistemological normativity. EBM's contention that it provides the "only valid" approach to knowledge and action is questioned by analyzing the troubled story of proposed hierarchies of the quality of research findings (especially of RCTs, with other factors marginalized), which falsely identifies evaluating findings with operationally utilizing them in clinical recommendations and decision-making. Further, its claim of carrying over its normative guidelines to cover the ethical responsibilities of researchers and clinicians is questioned.

The Role of Acupuncture in Treating Perimenopausal Insomnia: An Overview and Quality Assessment of Systematic Reviews and Meta-Analyses

OBJECTIVE

To summarize and critically assess the reliability of the methodological quality and outcome measures from systematic reviews (SRs)/meta-analyses (MAs) and provide an overall verdict about the therapeutic value of acupuncture for perimenopausal insomnia (PMI).

METHODS

We conducted a comprehensive literature search for SRs/MAs of seven major databases (English and Chinese). For each included review, the methodological quality was appraised according to the Assessing the Methodological Quality of Systematic Reviews 2 (AMSTAR-2), the evidence quality was classified on the basis of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE), and reporting quality was evaluated complying with Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2009 (PRISMA-2009). Veritas plots were used to quantify the quality of included SRs/MAs.

RESULTS

Nine SRs/MAs were deemed eligible for the present overview. Considering the assessment of results from the AMSTAR-2 checklist, the methodological quality of one SR/MA was considered low, and the remaining eight were critically low. Major methodological deficiencies were concentrated on item 2 (the lack of protocol and/or registration information), item 7 (the lack of a list of excluded studies), and item 10 (the lack of reports on funding sources for individual studies included in the SRs/MAs). For the GRADE system, of the 25 outcomes, only three (12%) were rated as moderate-quality, while the remaining 22 were rated between low- and very low-quality. The PRISMA-2009 statement indicated three major reporting quality limitations in most SRs/MAs, namely: 1) only search terms without specific retrieval strategy; 2) incomplete descriptions for study characteristics, particularly the specific dosage and frequency of interventions in treatment/control groups; and 3) inadequate investigation and explanation of the source of high heterogeneity among original randomized control trials included. According to Veritas plots, quality rank scores of included SRs/MAs ranged from 3.3 to 8.3, with an average score of 6.4 ± 1.7.

CONCLUSION

Acupuncture appears to be beneficial for PMI management, but the quality of evidence is weakened by the unsatisfactory quality of both SRs/MAs and original trials included.

Section Two Summary: Taking Stock of Health Literacy Practices

This summary of the brief reports notes common themes, activities, and directions in health literacy practices across diverse settings, organizations, and populations. The summary also discusses how a 'best practices' approach could help build an evidence base, solidify evidence of impact, and advance the use of health literacy techniques.

Review and Analysis of Research Trends in Surgical Treatment of Pediatric Chronic Sinusitis

Background

Chronic sinusitis is a common otolaryngologic complaint with a significant impact on patients’ quality of life. There is current debate and differences in quality of evidence regarding the best surgical management approach to pediatric chronic sinusitis.

Objective

To investigate changes in publishing trends over time in surgical management of pediatric chronic sinusitis.

Methods

A systematic literature review was conducted in January 2019 using Embase, PubMed, and Web of Science, of all articles that included surgical treatments for pediatric chronic sinusitis. Publications were grouped into 3 time periods: pre-1998, 1999–2008, and 2009–2018. In addition, a subgroup of publications pertaining to endoscopic sinus surgery, balloon sinuplasty, and adenoidectomy were grouped into 2-year periods since 1988 to compare the number of publications on each topic and to examine overall trends.

Results

A total of 327 abstracts were included in the data collection and analysis. There was an overall significant increase in total number of publications ( P < .0001), total number of authors ( P = .001), and comparison studies ( P = .003) from pre-1989 to 2018. Relative number of prospective studies, systemic review, and case studies/expert opinions have not significantly increased over time when comparing decades ( P > .05). Among the surgery types, functional endoscopic sinus surgery has the most amount of publications regardless of year despite a statistically significant increase in publications pertaining to balloon sinuplasty in the pediatric literature from 1999–2008 to 2009–2018 ( P = .001).

Conclusions

Studies on pediatric chronic sinusitis have increased significantly during each of the last 2 decades with an increase in the concomitant number of authors and comparison studies. Although most publications are still Level C or D evidence, there has been a nonsignificant increase in Level A evidence in the past decade. Endoscopic sinus surgery still remains the most studied procedure despite the recent increase in publications on balloon sinuplasty.

Effect on perinatal outcome of prophylactic antibiotics in preterm prelabor rupture of membranes: network meta-analysis of randomized controlled trials

OBJECTIVES

Prophylactic antibiotics are recommended routinely for preterm prelabor rupture of membranes (PPROM), but there is an abundance of potential treatments and a paucity of comparative information. The aims of this network meta-analysis were to compare the efficiency of different antibiotic regimens on perinatal outcomes and to assess the quality of the current evidence.

METHODS

This was a network meta-analysis of randomized controlled trials comparing prophylactic antibiotics, or regimens of antibiotics, with each other or with placebo/no treatment, in women with PPROM. MEDLINE, Scopus, Cochrane Central Register of Controlled Trials, US Registry of Clinical Trials ( www.ClinicalTrials.gov) and gray literature sources were searched. The primary outcomes were neonatal mortality and chorioamnionitis; secondary outcomes included other measures of perinatal morbidity. Relative effect sizes were estimated using risk ratios (RR) and the relative ranking of the interventions was obtained using cumulative ranking curves. The quality of evidence for the primary outcomes was assessed according to GRADE guidelines, adapted for network meta-analysis.

RESULTS

The analysis included 20 studies (7169 participants randomized to 15 therapeutic regimens). For the outcome of chorioamnionitis, clindamycin + gentamycin (network RR, 0.19 (95% CI, 0.05-0.83)), penicillin (RR, 0.31 (95% CI, 0.16-0.6)), ampicillin/sulbactam + amoxicillin/clavulanic acid (RR, 0.32 (95% CI, 0.12-0.92)), ampicillin (RR, 0.52 (95% CI, 0.34-0.81)) and erythromycin + ampicillin + amoxicillin (RR, 0.71 (95% CI, 0.55-0.92)) were superior to placebo/no treatment. Erythromycin was the only effective drug for neonatal sepsis (RR, 0.74 (95% CI, 0.56-0.97)). Clindamycin + gentamycin (RR, 0.32 (95% CI, 0.11-0.89)) and erythromycin + ampicillin + amoxicillin (RR, 0.83 (95% CI, 0.69-0.99)) were the only effective regimens for respiratory distress syndrome, whereas ampicillin (RR, 0.42 (95% CI, 0.20-0.92)) and penicillin (RR, 0.49 (95% CI, 0.25-0.96)) were effective in reducing the rates of Grade-3/4 intraventricular hemorrhage. None of the antibiotics appeared significantly more effective than placebo/no treatment in reducing the rates of neonatal death, perinatal death and necrotizing enterocolitis. No network RR could be estimated for neonatal intensive care unit admission. The overall quality of the evidence, according to GRADE guidelines, was moderate to very low, depending on the outcome and comparison.

CONCLUSIONS

Several antibiotics appear to be more effective than placebo/no treatment in reducing the rate of chorioamnionitis after PPROM. However, none of them is clearly and consistently superior compared to other antibiotics, and most are not superior to placebo/no treatment for other outcomes. The overall quality of the evidence is low and needs to be updated, as microbial resistance may have emerged for some antibiotics, while others are underrepresented in the existing evidence. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

Effects of antidepressants on QT interval in people with mental disorders

INTRODUCTION

Drug-induced QT prolongation is associated with higher cardiovascular mortality.

MATERIAL AND METHODS

We conducted a protocol-based comprehensive review of antidepressant-induced QT prolongation in people with mental disorders.

RESULTS

Based on findings from 47 published randomized controlled trials (RCTs), 3 unpublished RCTs, 14 observational studies, 662 case reports of torsades de pointes, and 168 cases of QT prolongation, we conclude that all antidepressants should be used only with licensed doses, and that all patients receiving antidepressants require monitoring of QT prolongation and clinical symptoms of cardiac arrhythmias. Large observational studies suggest increased mortality associated with all antidepressants (RR = 1.62, 95% CI: 1.60-1.63, number of adults: 1,716,552), high doses of tricyclic antidepressants (OR = 2.11, 85% CI 1.10-4.22), selective serotonin reuptake inhibitors (OR = 2.78, 95% CI: 1.24-6.24), venlafaxine (OR = 3.73, 95% CI: 1.33-10.45, number of adults: 4,040), and nortriptyline (OR = 4.60, 95% CI: 1.20-18.40, number of adults: 5,298).

CONCLUSIONS

Evidence regarding the risk of QT prolongation in children is sparse.

Critical Evaluation of the Methodologic Quality of the Top 50 Cited Articles Relating to Knee Dislocation and Multiligamentous Knee Injury

Background:

Many studies have evaluated the management of knee dislocations (KDs) and multiligamentous knee injuries (MLKIs). However, no study to date has analyzed the quality of the most cited articles in this literature.

Hypothesis:

There is a positive correlation between the number of article citations in the KD and MLKI literature and their methodologic quality.

Study Design:

Systematic review.

Methods:

The Web of Science online database was searched to identify the top 50 cited articles in KD and MLKI care. Demographic data were recorded for each study. The Modified Coleman Methodology Score (MCMS) and the Methodological Index for Non-randomized Studies (MINORS) were used to analyze the methodological quality of each article. Spearman correlation coefficients (r_s) were then calculated.

Results:

The articles identified were published between 1958 and 2015 in a wide variety of peer-reviewed journals (n = 16). The majority of study level of evidence (LOE) was of low quality (level 5, 16%; level 4, 54%; level 3, 16%; level 2, 14%). There were no studies of level 1 evidence. The mean MCMS and MINORS scores were 29.0 (SD, 19.1; range, 3-72) and 6.1 (SD, 3.7; range, 0-14), respectively. No significant correlation was identified between the number of citations and the publication year, LOE, MCMS, or MINORS (r_s = 0.123 [P = .396]; r_s = 0.125 [P = .389]; r_s = 0.182 [P = .204]; and r_s = 0.175 [P = .224], respectively). Positive correlations were observed between improved MCMS and MINORS scores and more recent year of publication (r_s = 0.43 [P = .002]; r_s = 0.32 [P = .022]) as well as improved study LOE (r_s = 0.65 [P < .001]; r_s = 0.67 [P < .001]).

Conclusion:

The top 50 cited articles on KD and MLKI care consisted of low LOE and methodological quality, with no existing level 1 articles. There was no significant correlation between the number of citations and publication year, LOE, or study methodological quality. Positive correlations were observed between later publication date and improved methodological quality.

At-a-glance - Bringing equity into the fold: a review of interventions to improve mental health

In Canada, it is challenging to find examples of positive population mental health interventions that meet scientific standards of evidence. It is even more difficult to identify effective interventions that address health equity. The discrepancy between standards of evidence in the health sciences, and the evidence that can be gleaned from social experiments, is not new. Efforts to reconcile these differences show a general tendency toward controlled interventions in public health. However, it is possible to extract findings from quasi-experimental interventions that meet scientific standards while also showing promise of positive impacts on mental health equity. This article describes work undertaken in 2015 to begin to address this evidence gap.

Assessment of Consistency between Claims and References Referred to in Pharmaceutical Advertising Brochures in the Kingdom of Saudi Arabia

Drug advertisement brochures (DABs) contain claims that are often supplemented by references in medical literature. Several studies have evaluated the DABs as they are commonly distributed by drug companies to practicing physicians. The objective of this study is to assess the consistency between the claims and references referred to in the DABs in Saudi Arabia. DABs were collected from medical practitioners in Riyadh, Saudi Arabia. Authors developed a protocol to be followed for quality assessment of the DABs. The vast majority of cited scientific papers were indexed in PubMed. Consequently, each reference was categorized as: justifiable, false, exaggerated or ambiguous. A total of 89 DABs were collected; 48 (53.9%) brochures were excluded from further analysis and the remaining 41 brochures (46.1%) contained 240 references with an approximate average of 5.9 references per DAB. A total of 201 cited papers were traced (83.8%). The majority of references (93.0%) supported the claims for which they were cited. However, 1.5%, 4.0% and 1.5% of claims were deemed inaccurate/false, exaggerated, and ambiguous, respectively. This study supports that the majority of the claims made in the DABs of pharmaceutical companies in Saudi Arabia were unreferenced. However, most of the evidence presented to substantiate claims made was considered true.

GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks

When direct and indirect estimates of treatment effects are coherent, network meta-analysis (NMA) estimates should have increased precision (narrower confidence or credible intervals compared with relying on direct estimates alone), a benefit of NMA. We have, however, observed cases of sparse networks in which combining direct and indirect estimates results in marked widening of the confidence intervals. In many cases, the assumption of common between-study heterogeneity across the network seems to be responsible for this counterintuitive result. Although the assumption of common between-study heterogeneity across paired comparisons may, in many cases, not be appropriate, it is required to ensure the feasibility of estimating NMA treatment effects. This is especially the case in sparse networks, in which data are insufficient to reliably estimate different variances across the network. The result, however, may be spuriously wide confidence intervals for some of the comparisons in the network (and, in the Grading of Recommendations Assessment, Development, and Evaluation approach, inappropriately low ratings of the certainty of the evidence through rating down for serious imprecision). Systematic reviewers should be aware of the problem and plan sensitivity analyses that produce intuitively sensible confidence intervals. These sensitivity analyses may include using informative priors for the between-study heterogeneity parameter in the Bayesian framework and the use of fixed effects models.

Using systematic reviews in guideline development: the GRADE approach

Systematic reviews are essential to produce trustworthy guidelines. To assess the certainty of a body of evidence included in a systematic review the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has developed an approach that is currently used by over 100 organisations, including the World Health Organization and the Cochrane Collaboration. GRADE provides operational definitions and instructions to rate the certainty of the evidence for each outcome in a review as high, moderate, low, or very low for the effects of interventions, prognostic estimates, values and preferences, test accuracy and resource utilization. The assessment includes assessing risk of bias, imprecision, inconsistency, indirectness, and publication bias, the magnitude of effects, dose-response relations and the impact of residual confounding and bias. Summary statistical information and assessments of certainty are presented in GRADE evidence summary tables, which can be produced using GRADE's official GRADEpro software tool (www.gradepro.org/). The evidence summary tables feed into the GRADE Evidence to Decision frameworks which guideline panels can use to produce recommendations.

What can we learn from top-cited articles in inflammatory bowel disease? A bibliometric analysis and assessment of the level of evidence

BACKGROUND AND OBJECTIVES

Despite increasing number of publications in inflammatory bowel disease (IBD), no bibliometric analysis has been conducted to evaluate the significance of highly cited articles. Our objectives were to identify the top-cited articles in IBD, assessing their characteristics and determining the quality of evidence provided by these articles.

DESIGN AND OUTCOME MEASURES

IBD and related terms were used in searching the Web of Science to identify English language articles. The 50 top-cited articles were analysed by year, journal impact factor (JIF), authorship, females in authorship, institute, country and grants received. The level of evidence was determined using the Oxford Centre for Evidence-Based Medicine guidelines.

RESULTS

The number of citations varied from 871 to 3555 with a total of 74 638, and a median 1339.50 (IQR=587). No correlations were found between the number of citations and number of years since publication (r=0.042, p=0.771), JIF (r=0.186, p=0.196), number of authors (r=0.061, p=0.674), females in authorship (r=0.064, p=0.661), number of institutes (r=0.076, p=0.602), number of countries (r=0.101, p=0.483) or number of grants (r=-0.015, p=0.915). The first authors were from the USA (n=24), the UK (n=6), Germany (n=5), France (n=5), Belgium (n=3) and Canada (n=3). The levels of evidence were 12 articles at level 1b, 9 articles at level 3a and 15 articles at level 3b and fewer were at other levels.

CONCLUSIONS

Research papers represented 66% of articles. The majority of items have reasonably high levels of evidence, which may have contributed to the higher number of citations. The study also shows a gender gap in authorship in this area.

Correlation Between Quality of Evidence and Number of Citations in Top 50 Cited Articles in Rotator Cuff Repair Surgery

BACKGROUND

The number of article citations has been used as a measure for the impact of an article in the medical literature, with little emphasis on quality.

PURPOSE

To (1) identify the top 50 most cited articles related to rotator cuff repair surgery, (2) determine whether there was a correlation between the top cited articles and level of evidence, and (3) determine whether there was a correlation between the top cited articles and study quality.

STUDY DESIGN

Cross-sectional study.

METHODS

The Web of Science and Scopus online databases were searched to identify the top 50 cited articles in rotator cuff repair surgery. Methodological quality was analyzed for each article using the Modified Coleman Methodology Score (MCMS), Jadad scale, and Methodological Index for Non-Randomized Studies (MINORS). Correlation coefficients were calculated to determine the degree of correlation between the top cited articles and level of evidence and study quality using each quality score.

RESULTS

The mean number of citations for each article in each of the 2 databases was 319 ± 187 (range, 177.5-1033.5). Twenty-nine articles (58%) were from the United States. The most common level of evidence was level 4 (54%), with 3 (6%) articles being level 1. There was no significant correlation between the mean number of citations and level of evidence (rs = -0.28), the MCMS (rs = -0.29), and the MINORS score (rs = -0.25). There was a weak negative correlation between the mean number of citations and the Jadad score (rs = -0.36).

CONCLUSION

The top 50 cited articles in rotator cuff repair surgery comprise a variety of years, journals, countries of origin, and study types. Despite being the 50 most cited articles, the most common type of article was the level 4 case series with a poor mean quality assessment score. There was no significant correlation between the mean number of citations and level of evidence or methodological quality using a variety of scores.

Correlation Between Quality of Evidence and Number of Citations in Top 50 Cited Articles on Elbow Medial Ulnar Collateral Ligament Surgery

Background:

Several studies have analyzed the most cited articles in shoulder, elbow, pediatrics, and foot and ankle surgery. However, no study has analyzed the quality of the most cited articles in elbow medial ulnar collateral ligament (UCL) surgery.

Purpose:

To (1) identify the top 50 most cited articles related to UCL surgery, (2) determine whether there was a correlation between the top cited articles and level of evidence, and (3) determine whether there was a correlation between study methodological quality and the top cited articles.

Study Design:

Systematic review.

Methods:

Web of Science and Scopus online databases were searched to identify the top 50 cited articles in UCL surgery. Level of evidence, number of times cited, year of publication, name of journal, country of origin, and study type were recorded for each study. Study methodological quality was analyzed for each article with the Modified Coleman Methodology Score (MCMS) and the Methodological Index for Non-randomized Studies (MINORS). Correlation coefficients were calculated.

Results:

The 50 most cited articles were published between 1981 and 2015. The number of citations per article ranged from 20 to 301 (mean ± SD, 71 ± 62 citations). Most articles (92%) were from the United States and were level 3 (16%), level 4 (58%), or unclassified (16%) evidence. There were no articles of level 1 evidence quality. The mean MCMS and MINORS scores were 28.1 ± 13.4 (range, 3-52) and 9.2 ± 3.6 (range, 2-19), respectively. There was no significant correlation between the mean number of citations and level of evidence or quality (r_s = –0.01, P = .917), MCMS (r_s = 0.09, P = .571), or MINORS (r_s = –0.26, P = .089).

Conclusion:

The top 50 cited articles in UCL surgery constitute a low level of evidence and low methodological quality, including no level 1 articles. There was no significant correlation between the mean number of citations and level of evidence or study methodological quality. However, weak correlations were observed for later publication date and improved level of evidence and methodological quality.

WHO Environmental Noise Guidelines for the European Region: A Systematic Review on Environmental Noise and Adverse Birth Outcomes

Introduction: Three recent systematic reviews suggested a relationship between noise exposure and adverse birth outcomes. The aim of this review was to evaluate the evidence for the World Health Organization (WHO) noise guidelines and conduct an updated systematic review of environmental noise, specifically aircraft and road traffic noise and birth outcomes, such as preterm birth, low birth weight, being small for gestational age and congenital malformations. Materials and methods: We reviewed again all the papers on environmental noise and birth outcomes included in the previous three systematic reviews and conducted a systematic search on noise and birth outcomes to update previous reviews. Web of Science, PubMed and Embase electronic databases were searched for papers published between June 2014 (end date of previous systematic review) and December 2016 using a list of specific search terms. Studies were also screened in the reference list of relevant reviews/articles. Further inclusion and exclusion criteria for the studies provided by the WHO expert group were applied. Risk of bias was assessed according to criteria from the Newcastle-Ottawa quality assessment scale for case-control and cohort studies. Finally, we applied the GRADE principles to our systematic review in a reproducible and appropriate way for judgment about quality of evidence. Results: In total, 14 studies are included in this review, six studies on aircraft noise and birth outcomes, five studies (two with more or less the same population) on road traffic noise and birth outcomes and three related studies on total ambient noise that is likely to be mostly traffic noise that met the criteria. The number of studies on environmental noise and birth outcomes is small and the quality of evidence generally ranges from very low to low, particularly in case of the older studies. The quality is better for the more recent traffic noise and birth outcomes studies. As there were too few studies, we did not conduct meta-analyses. Discussion: This systematic review is supported by previous systematic reviews and meta-analyses that suggested that there may be some suggestive evidence for an association between environmental noise exposure and birth outcomes, although they pointed more generally to a stronger role of occupational noise exposure, which tends to be higher and last longer. Very strict criteria for inclusion and exclusion of studies, performance of quality assessment for risk of bias, and finally applying GRADE principles for judgment of quality of evidence are the strengths of this review. Conclusions: We found evidence of very low quality for associations between aircraft noise and preterm birth, low birth weight and congenital anomalies, and low quality evidence for an association between road traffic noise and low birth weight, preterm birth and small for gestational age. Further high quality studies are required to establish such associations. Future studies are recommended to apply robust exposure assessment methods (e.g., modeled or measured noise levels at bedroom façade), disentangle associations for different sources of noise as well as daytime and nighttime noise, evaluate the impacts of noise evens (that stand out of the noise background), and control the analyses for confounding factors, such as socioeconomic status, lifestyle factors and other environmental factors, especially air pollution.

Synthesizing and Appraising the Quality of the Evidence on Factors Associated with Medication Adherence in Diabetes: A Systematic Review of Systematic Reviews

BACKGROUND

Nonadherence to medications is a common phenomenon in patients with diabetes. Several studies and systematic reviews have investigated the barriers to medication adherence in diabetes. However, no study has evaluated the quality of the existing literature and synthesized the plethora of evidence with a goal to design holistic conceptual frameworks and interventions.

OBJECTIVES

The aims of this review were to systematically evaluate existing systematic reviews focusing on factors associated with medication adherence in diabetes in an effort to synthesize the evidence, determine their methodological quality, and identify the gaps in the current literature.

METHODS

Fourteen databases and gray literature sources were systematically searched through June 2016. Systematic reviews reporting factors associated with medication adherence (barriers and facilitators) in patients with diabetes were selected on the basis of predetermined criteria. Studies were appraised for quality using AMSTAR (A Measurement Tool to Assess Systematic Reviews).

RESULTS

Seventeen systematic reviews including 542 primary studies, most of which were cross-sectional quantitative studies, were included. All the reviews were rated as moderate to low quality and exhibited common methodological pitfalls. Factors influencing medication adherence identified were categorized as patient-, medication-, disease-, health care provider-, health care system-, and social-related factors.

CONCLUSIONS

Factors influencing medication adherence are multifactorial with remarkably consistent findings across the existing reviews; yet, most reviews were judged to be of low to moderate quality. Further comprehensive and well-conducted original studies and systematic reviews on this topic shall be conducted taking into consideration the drawbacks of existing ones.

A systematic review of genetic syndromes with obesity

Syndromic monogenic obesity typically follows Mendelian patterns of inheritance and involves the co-presentation of other characteristics, such as mental retardation, dysmorphic features and organ-specific abnormalities. Previous reviews on obesity have reported 20 to 30 syndromes but no systematic review has yet been conducted on syndromic obesity. We searched seven databases using terms such as 'obesity', 'syndrome' and 'gene' to conduct a systematic review of literature on syndromic obesity. Our literature search identified 13,719 references. After abstract and full-text review, 119 relevant papers were eligible, and 42 papers were identified through additional searches. Our analysis of these 161 papers found that 79 obesity syndromes have been reported in literature. Of the 79 syndromes, 19 have been fully genetically elucidated, 11 have been partially elucidated, 27 have been mapped to a chromosomal region and for the remaining 22, neither the gene(s) nor the chromosomal location(s) have yet been identified. Interestingly, 54.4% of the syndromes have not been assigned a name, whereas 13.9% have more than one name. We report on organizational inconsistencies (e.g. naming discrepancies and syndrome classification) and provide suggestions for improvements. Overall, this review illustrates the need for increased clinical and genetic research on syndromes with obesity.

Treatment of adult patients with schizophrenia and complex mental health needs - A national clinical guideline

BACKGROUND AND AIM

The Danish Health and Medicines Authority assembled a group of experts to develop a national clinical guideline for patients with schizophrenia and complex mental health needs. Within this context, ten explicit review questions were formulated, covering several identified key issues.

METHODS

Systematic literature searches were performed stepwise for each review question to identify relevant guidelines, systematic reviews/meta-analyses, and randomized controlled trials. The quality of the body of evidence for each review question was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Clinical recommendations were developed on the basis of the evidence, assessment of the risk-benefit ratio, and perceived patient preferences.

RESULTS

Based on the identified evidence, a guideline development group (GDG) recommended that the following interventions should be offered routinely: antipsychotic maintenance therapy, family intervention and assertive community treatment. The following interventions should be considered: long-acting injectable antipsychotics, neurocognitive training, social cognitive training, cognitive behavioural therapy for persistent positive and/or negative symptoms, and the combination of cognitive behavioural therapy and motivational interviewing for cannabis and/or central stimulant abuse. SSRI or SNRI add-on treatment for persistent negative symptoms should be used only cautiously. Where no evidence was available, the GDG agreed on a good practice recommendation.

CONCLUSIONS

The implementation of this guideline in daily clinical practice can facilitate good treatment outcomes within the population of patients with schizophrenia and complex mental health needs. The guideline does not cover all available interventions and should be used in conjunction with other relevant guidelines.

Systematic reviews with language restrictions and no author contact have lower overall credibility: a methodology study

BACKGROUND

High-quality systematic reviews (SRs) require rigorous approaches to identify, appraise, select, and synthesize research evidence relevant to a specific question. In this study, we evaluated the association between two steps in the conduct of an SR - restricting the search to English, and author contact for missing data - and the overall credibility of a SR.

METHODS

All SRs cited by the Endocrine Society's Clinical Practice Guidelines published from October 2006 through January 2012 were included. The main outcome was the overall A Measurement Tool to Assess Systematic Reviews (AMSTAR) score, as a surrogate of SR credibility. Nonparametric Kruskal-Wallis tests and multivariable linear regression models were used to investigate the association between language restriction, author contact for missing data, and the overall AMSTAR score.

RESULTS

In all, 69 SRs were included in the analysis. Only 31 SRs (45%) reported searching non-English literature, with an average AMSTAR score of 7.90 (standard deviation [SD] =1.64). SRs that reported language restriction received significantly lower AMSTAR scores (mean =5.25, SD =2.32) (P<0.001). Only 30 SRs (43%) reported contacting authors for missing data, and these received, on average, 2.59 more AMSTAR points (SD =1.95) than those who did not (P<0.001). In multivariable analyses, AMSTAR score was significantly correlated with language restriction (beta =-1.31, 95% confidence interval [CI]: -2.62, -0.01, P=0.05) and author contact for missing data (beta =2.16, 95% CI: 0.91, 3.41, P=0.001). However, after adjusting for compliance with reporting guidelines, language restriction was no longer significantly associated with the AMSTAR score.

CONCLUSION

Fewer than half of the SRs conducted to support the clinical practice guidelines we examined reported contacting study authors or searched non-English literature. SRs that did not conduct these two steps had lower quality scores, suggesting the importance of these two steps for overall SR credibility.

What is the level of evidence substantiating the Medicare local coverage determinations?

Total joint arthroplasty (TJA) accounts for large expenditures of Medicare resources. Recovery audit contractors (RACs) utilize Local Coverage Determinations (LCDs) documents to retrospectively assess the medical necessity of performed procedures. The Florida LCD requires 3 months of documented unsuccessful nonoperative care prior to TJA. The purpose of our study was to evaluate the applicability and quality of the evidence cited in the Florida LCD for patients undergoing TJA. Applicability was assessed by evaluating the efficacy of nonoperative treatment in candidates for TJA. There were 23 citations, of which 11 mentioned nonoperative treatment and only 5 provided references. No citations provided Level I or II evidence substantiating the effectiveness of 3 months of nonoperative treatment in patients who would otherwise be candidates for TJA.

Methodological considerations of the GRADE method

The GRADE method (Grading of Recommendations, Assessment, Development, and Evaluation) provides a tool for rating the quality of evidence for systematic reviews and clinical guidelines. This article aims to analyse conceptually how well grounded the GRADE method is, and to suggest improvements. The eight criteria for rating the quality of evidence as proposed by GRADE are here analysed in terms of each criterion's potential to provide valid information for grading evidence. Secondly, the GRADE method of allocating weights and summarizing the values of the criteria is considered. It is concluded that three GRADE criteria have an appropriate conceptual basis to be used as indicators of confidence in research evidence in systematic reviews: internal validity of a study, consistency of the findings, and publication bias. In network meta-analyses, the indirectness of evidence may also be considered. It is here proposed that the grade for the internal validity of a study could in some instances justifiably decrease the overall grade by three grades (e.g. from high to very low) instead of the up to two grade decrease, as suggested by the GRADE method.

Corroborating evidence-based medicine

Proponents of evidence-based medicine (EBM) have argued convincingly for applying this scientific method to medicine. However, the current methodological framework of the EBM movement has recently been called into question, especially in epidemiology and the philosophy of science. The debate has focused on whether the methodology of randomized controlled trials provides the best evidence available. This paper attempts to shift the focus of the debate by arguing that clinical reasoning involves a patchwork of evidential approaches and that the emphasis on evidence hierarchies of methodology fails to lend credence to the common practice of corroboration in medicine. I argue that the strength of evidence lies in the evidence itself, and not the methodology used to obtain that evidence. Ultimately, when it comes to evaluating the effectiveness of medical interventions, it is the evidence obtained from the methodology rather than the methodology that should establish the strength of the evidence.

[GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations]

This article describes the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to classifying the direction and strength of recommendations. The strength of a recommendation, separated into strong and weak, is defined as the extent to which one can be confident that the desirable effects of an intervention outweigh its undesirable effects. Alternative terms for a weak recommendation include conditional, discretionary, or qualified. The strength of a recommendation has specific implications for patients, the public, clinicians, and policy makers. Occasionally, guideline developers may choose to make "only-in-research" recommendations. Although panels may choose not to make recommendations, this choice leaves those looking for answers from guidelines without the guidance they are seeking. GRADE therefore encourages panels to, wherever possible, offer recommendations.

[GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes]

GRADE requires guideline developers to make an overall rating of confidence in estimates of effect (quality of evidence-high, moderate, low, or very low) for each important or critical outcome. GRADE suggests, for each outcome, the initial separate consideration of five domains of reasons for rating down the confidence in effect estimates, thereby allowing systematic review authors and guideline developers to arrive at an outcome-specific rating of confidence. Although this rating system represents discrete steps on an ordinal scale, it is helpful to view confidence in estimates as a continuum, and the final rating of confidence may differ from that suggested by separate consideration of each domain. An overall rating of confidence in estimates of effect is only relevant in settings when recommendations are being made. In general, it is based on the critical outcome that provides the lowest confidence.

Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework

BACKGROUND

Prognosis research aims to identify factors associated with the course of health conditions. It is often challenging to judge the overall quality of research evidence in systematic reviews about prognosis due to the nature of the primary studies. Standards aimed at improving the quality of primary studies on the prognosis of health conditions have been created, but these standards are often not adequately followed causing confusion about how to judge the evidence.

METHODS

This article presents a proposed adaptation of Grading of Recommendations Assessment, Development and Evaluation (GRADE), which was developed to rate the quality of evidence in intervention research, to judge the quality of prognostic evidence.

RESULTS

We propose modifications to the GRADE framework for use in prognosis research along with illustrative examples from an ongoing systematic review in the pediatric pain literature. We propose six factors that can decrease the quality of evidence (phase of investigation, study limitations, inconsistency, indirectness, imprecision, publication bias) and two factors that can increase it (moderate or large effect size, exposure-response gradient).

CONCLUSIONS

We describe criteria for evaluating the potential impact of each of these factors on the quality of evidence when conducting a review including a narrative synthesis or a meta-analysis. These recommendations require further investigation and testing.

Long-term opioid treatment of chronic nonmalignant pain: unproven efficacy and neglected safety?

BACKGROUND

For the past 30 years, opioids have been used to treat chronic nonmalignant pain. This study tests the following hypotheses: (1) there is no strong evidence-based foundation for the conclusion that long-term opioid treatment of chronic nonmalignant pain is effective; and (2) the main problem associated with the safety of such treatment - assessment of the risk of addiction - has been neglected.

METHODS

Scientometric analysis of the articles representing clinical research in this area was performed to assess (1) the quality of presented evidence (type of study); and (2) the duration of the treatment phase. The sufficiency of representation of addiction was assessed by counting the number of articles that represent (1) editorials; (2) articles in the top specialty journals; and (3) articles with titles clearly indicating that the addiction-related safety is involved (topic-in-title articles).

RESULTS

Not a single randomized controlled trial with opioid treatment lasting >3 months was found. All studies with a duration of opioid treatment ≥6 months (n = 16) were conducted without a proper control group. Such studies cannot provide the consistent good-quality evidence necessary for a strong clinical recommendation. There were profound differences in the number of addiction articles related specifically to chronic nonmalignant pain patients and to opioid addiction in general. An inadequate number of chronic pain-related publications were observed with all three types of counted articles: editorials, articles in the top specialty journals, and topic-in-title articles.

CONCLUSION

There is no strong evidence-based foundation for the conclusion that long-term opioid treatment of chronic nonmalignant pain is effective. The above identified signs indicating neglect of addiction associated with the opioid treatment of chronic nonmalignant pain were present.

How to critically appraise a clinical practice guideline

PURPOSE

Clinical practice guidelines play a critical role in guiding the evidence-based clinical practice of urology. We describe a systematic approach to critical appraisal of urology guidelines.

MATERIALS AND METHODS

Based on a focused clinical question derived from a clinical scenario, we identified a relevant clinical practice guideline that we critically appraised using the Users' Guide to the Medical Literature framework as to whether the results are valid, what are the results, and can they be applied to the care of an individual patient.

RESULTS

A clinical practice guideline by the National Institute for Clinical Excellence on the use of sunitinib as the first line treatment for patients with metastatic renal cell carcinoma was identified. The guideline development process was found to be appropriately rigorous and included an explicit rating of the quality of evidence. The recommendations were clearly stated and appeared applicable to the specific patient in the clinical scenario.

CONCLUSIONS

Clinical practice guidelines should be developed using rigorous evidence-based methodology. Urologists should have the skills and knowledge to critically appraise a guideline before applying it to the care of their patients.

Could media reports include a standardized scale for quality of evidence?

While some media reports offer accurate interpretations of clinical research, other reports are misleading. The uneven accuracy of medical reporting may act in concert with its sheer volume to confuse the lay public about which health messages are most important and evidence-based. I outline one possible step towards a solution: medical journals can embed quality of evidence ratings in article summaries and create incentives for inclusion of these ratings in lay media reports.