Massage for rehabilitation after total knee arthroplasty: a systematic review and meta-analysis of randomized controlled trials

OBJECTIVE

This study aimed to evaluate the effectiveness of massage for postoperative rehabilitation after total knee arthroplasty (TKA).

DATA SOURCES

The PubMed, Web of Science, EMBASE, Cochrane Library, and China National Knowledge Infrastructure (CNKI) databases were systematically searched from inception to May 2024.

STUDY SELECTION

Any randomized controlled trials on the use of massage for postoperative TKA rehabilitation were included.

DATA EXTRACTION

A meta-analysis of outcomes, including postoperative pain, knee range of motion (ROM), postoperative D-dimer levels, and length of hospital stay, was performed. The Cochrane Risk of Bias Assessment Tool was used to assess the risk of bias, and the data for each included study were extracted independently by two researchers.

DATA SYNTHESIS

Eleven randomized controlled clinical trials with 940 subjects were included. The results showed that compared with the control group, the massage group experienced more significant pain relief on the 7th, 14th and 21st days after the operation. Moreover, the improvement in knee ROM was more pronounced on postoperative days 7 and 14. In addition, the massage group reported fewer adverse events. However, there was no statistically significant difference in the reduction in postoperative D-dimer levels between the patients and controls. Subgroup analysis revealed that massage shortened the length of hospital stay for postoperative patients in China but not significantly for patients in other regions. Nevertheless, the heterogeneity of the studies was large.

CONCLUSIONS

Increased massage treatment was more effective at alleviating pain and improving knee ROM in early post-TKA patients. However, massage did not perform better in reducing D-dimer levels in patients after TKA. Based on the current evidence, massage can be used as an adjunctive treatment for rehabilitation after TKA.

Rethinking the pros and cons of randomized controlled trials and observational studies in the era of big data and advanced methods: a panel discussion

Randomized controlled trials (RCTs) have traditionally been considered the gold standard for medical evidence. However, in light of emerging methodologies in data science, many experts question the role of RCTs. Within this context, experts in the USA and Canada came together to debate whether the primacy of RCTs as the gold standard for medical evidence, still holds in light of recent methodological advances in data science and in the era of big data. The purpose of this manuscript, aims to raise awareness of the pros and cons of RCTs and observational studies in order to help guide clinicians, researchers, students, and decision-makers in making informed decisions on the quality of medical evidence to support their work. In particular, new and underappreciated advantages and disadvantages of both designs are contrasted. Innovations taking place in both of these research methodologies, which can blur the lines between the two, are also discussed. Finally, practical guidance for clinicians and future directions in assessing the quality of evidence is offered.

Effects of timing of umbilical cord clamping for mother and newborn: a narrative review

OBJECTIVE

This narrative review was performed to evaluate the correct timing of umbilical cord clamping for term infants. It was intended to determine any advantages or disadvantages from early or delayed cord clamping for newborns, infants or mothers.

METHODS

A systematic search on two databases was conducted using the PICO pattern to define a wide search. Out of 43 trials, 12 were included in this review. Three of the included studies are meta-analyses, nine are randomized controlled trials.

RESULTS

Early or delayed cord clamping was defined differently in all the included trials. However, there are many advantages from delayed cord clamping of at least > 60 s for newborns and infants up to 12 months of age. The trials showed no disadvantages for newborns or mothers from delayed cord clamping, except for a lightly increased risk of jaundice or the need for phototherapy.

CONCLUSION

Delayed umbilical cord clamping for term infants should be performed. Further research is needed to improve knowledge on physiological timing of umbilical cord clamping in term infants, which also leads to the same advantages as delayed cord clamping.

Efficacy of optometric phototherapy: a systematic review

PURPOSE

To analyse the scientific evidence about the efficacy of Syntonic phototherapy for producing changes in visual function.

MATERIAL AND METHODS

A systematic review was performed to obtain studies on the effects of Syntonic phototherapy on vision. A search in health science databases (Medline, Scopus, Web of Science and PsycINFO) for studies published between 1980 and 2022 was conducted in accordance with the principles of Cochrane approach. The search identified 197 articles. Only clinical studies which used the Syntonic phototherapy as a vision therapy for any visual condition were included. Clinical cases and case series were excluded. Following the inclusion criteria, 8 clinical studies met inclusion, 5 of them being pseudo-experimental studies with an equivalent control group and 3 pre-post pseudo-experimental studies. GRADE tool was used to assess the certainty of the evidence of the studies. The GRADE evidence profile for the studies through the Soft table was made to analyse data.

RESULTS

The studies analysed seven outcomes: visual symptoms, functional visual fields, visual acuity, contrast sensitivity, deviation (phoria/tropia), stereopsis and reading abilities. Finding table about results (Soft Table) showed that for all outcomes reviewed, all studies yielded very low certainty of evidence. Results revealed a lack of scientific evidence of the efficacy of Syntonic optometric phototherapy to produce changes in the visual function.

CONCLUSION

This systematic review found no consistent evidence for the efficacy of Syntonic phototherapy to cause changes in visual function. There is no scientific evidence to support its clinical use for treating any type of visual anomalies.

A Systematic Review on Navigation Programs for Persons Living With Dementia and Their Caregivers

BACKGROUND AND OBJECTIVES

System navigation programs are becoming more available to meet the needs of patients with complex care needs. The aim of this review was to systematically assess the outcomes of navigation programs for persons with dementia and their family caregivers.

RESEARCH DESIGN AND METHODS

A systematic review methodology was employed. Ten databases were searched for all relevant articles published until October 30, 2021. English-language full-text articles were included if they focused on implemented navigation program(s) that primarily supported persons with dementia who were aged 50 or older. Methodological quality was assessed by 2 independent raters using the Physiotherapy Evidence Database Scale, the STrengthening the Reporting of OBservational studies in Epidemiology checklist, and the Mixed Methods Appraisal Tool.

RESULTS

Fourteen articles were included in the review. There was Level 1 evidence for the benefits of system navigation programs on delaying institutionalization, wherein benefits appeared to be specific to interventions that had an in-person component. There was Level 1 (n = 4) and Level 3 (n = 1) evidence on service use from time of diagnosis to continued management of dementia. Finally, Level 1 to Level 5 evidence indicated a number of benefits on caregiver outcomes.

DISCUSSION AND IMPLICATIONS

There is strong evidence on the benefits of system navigation for people with dementia on delaying institutionalization and caregiver outcomes, but outcomes across other domains (i.e., functional independence) are less clear, which may be due to the varied approaches within system navigation models of care.

Grading of recommendations assessment, development, and evaluation concept article 5: addressing intransitivity in a network meta-analysis

OBJECTIVES

This article describes considerations for addressing intransitivity when assessing the certainty of the evidence from network meta-analysis (NMA) using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. Intransitivity is induced by effect modification, that is, when the magnitude of the effect between an intervention and outcome differs depending on the level of another factor.

STUDY DESIGN AND SETTING

To develop this GRADE concept paper, the lead authors conducted iterative discussions, computer simulations, and presentations to the GRADE project group and at GRADE working group meetings. The GRADE Working Group formally approved the article in July 2022.

RESULTS

NMA authors can have a higher or a lower threshold to rate down the certainty of the evidence due to intransitivity, which depends on the extent of their concerns regarding the trustworthiness of indirect comparisons, and their view of the relative problems with rating down excessively or insufficiently. NMA authors should consider three main factors when addressing intransitivity: the credibility of effect modification, the strength of the effect modification, and the distribution of effect modifiers across the direct comparisons. To avoid double counting limitations of the evidence, authors should consider the relationship between intransitivity and other GRADE domains.

CONCLUSION

NMA authors face theoretic and pragmatic challenges and in most situations need to assess intransitivity without the availability of empirical data. Thus, explicitness regarding perspective is crucial.

Exploring the feasibility of using the ICER Evidence Rating Matrix for Comparative Clinical Effectiveness in assessing treatment benefit and certainty in the clinical evidence on orphan therapies for paediatric indications

BACKGROUND

The evaluation of clinical evidence takes account of health benefit (efficacy and safety) and the degree of certainty in the estimate of benefit. In orphan indications practical and ethical challenges in conducting clinical trials, particularly in paediatric patients, often limit the available evidence, rendering structured evaluation challenging. While acknowledging the paucity of evidence, regulators and reimbursement authorities compare the efficacy and safety of alternative treatments for a given indication, often in the context of the benefits of other treatments for similar or different conditions. This study explores the feasibility of using the Institute for Clinical and Economic Review (ICER) Evidence Rating Matrix for Comparative Clinical Effectiveness in structured assessment of both the magnitude of clinical benefit (net health benefit, NHB) and the certainty of the effect estimate in a sample of orphan therapies for paediatric indications.

RESULTS

Eleven systemic therapies with European Medicines Agency (EMA) orphan medicinal product designation, licensed for 16 paediatric indications between January 2017 and March 2020 were identified using OrphaNet and EMA databases and were selected for evaluation with the ICER Evidence Rating Matrix: burosumab; cannabidiol; cerliponase alfa; chenodeoxycholic acid (CDCA); dinutuximab beta; glibenclamide; metreleptin; nusinersen; tisagenlecleucel; velmanase alfa; and vestronidase alfa. EMA European Public Assessment Reports, PubMed, EMBASE, the Cochrane Library, Clinical Key, and conference presentations from January 2016 to April 2021 were searched for evidence on efficacy and safety. Two of the identified therapies were graded as "substantial" NHB: dinutuximab beta (neuroblastoma maintenance) and nusinersen (Type I SMA), and one as "comparable" NHB (CDCA). The NHB grade of the remaining therapies fell between "comparable" and "substantial". No therapies were graded as having negative NHB. The certainty of the estimate ranged from "high" (dinutuximab beta in neuroblastoma maintenance) to "low" (CDCA, metreleptin and vestronidase alfa). The certainty of the other therapies was graded between "low" and "high". The ICER Evidence Rating Matrix overall rating "A" (the highest) was given to two therapies, "B+" to 6 therapies, "C+" to five therapies, and "I" (the lowest) to three therapies. The scores varied between rating authors with mean agreement over all indications of 71.9% for NHB, 56.3% for certainty and 68.8% for the overall rating.

CONCLUSIONS

Using the ICER Matrix to grade orphan therapies according to their treatment benefit and certainty is feasible. However, the assessment involves subjective judgements based on heterogenous evidence. Tools such as the ICER Matrix might aid decision makers to evaluate treatment benefit and its certainty when comparing therapies across indications.

One-Third of Systematic Reviews in Rehabilitation Applied the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) System to Evaluate Certainty of Evidence: A Meta-Research Study

OBJECTIVE

To determine how many systematic reviews (SRs) of the literature in rehabilitation assess the certainty of evidence (CoE) and how many apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system to do this.

DATA SOURCES

For this meta-research study, we searched PubMed and Cochrane Database of Systematic Reviews databases for SRs on rehabilitation published in 2020.

STUDY SELECTION AND DATA EXTRACTION

Two reviewers independently selected the SRs and extracted the data. Reporting characteristics and appropriate use of the GRADE system were assessed.

DATA SYNTHESIS

The search retrieved 827 records: 29% (239/827) SRs evaluated CoE, 68% (163/239) of which applied the GRADE system. GRADE was used by SRs of randomized controlled trials (RCTs, 88%; 144/163), non-randomized intervention studies (NRIS, 2%; 3/163), and both RCT and NRIS (10%; 16/163). In the latter case, a separate GRADE assessment according to the study design was not provided in 75% (12/16). The reasons for GRADE judgment were reported in 82% (134/163) of SRs.

CONCLUSIONS

One-third of SRs in rehabilitation assessed CoE with the GRADE system. GRADE assessment was presented transparently by most SRs. Journal editors and funders should encourage the uptake of the GRADE system when considering SRs in rehabilitation for publication. The authors should pre-define GRADE assessment in a registered and/or published protocol.

Editor's Choice - European Society for Vascular Surgery Clinical Practice Guideline Development Scheme: An Overview of Evidence Quality Assessment Methods, Evidence to Decision Frameworks, and Reporting Standards in Guideline Development

OBJECTIVE

A structured and transparent approach is instrumental in translating research evidence to health recommendations and evidence informed clinical decisions. The aim was to conduct an overview and analysis of principles and methodologies for health guideline development.

METHODS

A literature review on methodologies, strategies, and fundamental steps in the process of guideline development was performed. The clinical practice guideline development process and methodology adopted by the European Society for Vascular Surgery are also presented.

RESULTS

Sophisticated methodologies for health guideline development are being applied increasingly by national and international organisations. Their overarching principle is a systematic, structured, transparent, and iterative process that is aimed at making well informed healthcare choices. Critical steps in guideline development include the assessment of the certainty of the body of evidence; evidence to decision frameworks; and guideline reporting. The goal of strength of evidence assessments is to provide well reasoned judgements about the guideline developers' confidence in study findings, and several evidence hierarchy schemes and evidence rating systems have been described for this purpose. Evidence to decision frameworks help guideline developers and users conceptualise and interpret the construct of the quality of the body of evidence. The most widely used evidence to decision frameworks are those developed by the GRADE Working Group and the WHO-INTEGRATE, and are structured into three distinct components: background; assessment; and conclusions. Health guideline reporting tools are employed to ensure methodological rigour and transparency in guideline development. Such reporting instruments include the AGREE II and RIGHT, with the former being used for guideline development and appraisal, as well as reporting.

CONCLUSION

This guide will help guideline developers/expert panels enhance their methodology, and patients/clinicians/policymakers interpret guideline recommendations and put them in context. This document may be a useful methodological summary for health guideline development by other societies and organisations.

Network meta-analyses: Methodological prerequisites and clinical usefulness

It is an undeniable fact that systematic reviews play a crucial role in informing clinical practice; however, conventional head-to-head meta-analyses do have limitations. In particular, studies can only be compared in a pair-wise fashion, and conclusions can only be drawn in the light of direct evidence. In contrast, network meta-analyses can not only compare multiple interventions but also utilize indirect evidence which increases their precision. On top of that, they can also rank competing interventions. In this mini-review, we have aimed to elaborate on the principles and techniques governing network meta-analyses to achieve a methodologically sound synthesis, thus enabling safe conclusions to be drawn in clinical practice. We have emphasized the prerequisites of a well-conducted Network Meta-Analysis (NMA), the value of selecting appropriate outcomes according to guidelines for transparent reporting, and the clarity achieved via sophisticated graphical tools. What is more, we have addressed the importance of incorporating the level of evidence into the results and interpreting the findings according to validated appraisal systems (i.e., the Grade of Recommendations, Assessment, Development, and Evaluation system - GRADE). Lastly, we have addressed the possibility of planning future research via NMAs. Thus, we can conclude that NMAs could be of great value to clinical practice.

Critical appraisal of international adult bronchiectasis guidelines using the AGREE II tool

BACKGROUND

Guidelines aim to standardize and optimize diagnosis and management. We evaluated the quality of evidence supporting recommendations from different international adult guidelines on bronchiectasis, and classified with the GRADE system.

METHODS

Quality of eligible clinical practice guidelines was assessed for six domains using the AGREE II tool, with ≥ 80% rating as excellent.

RESULTS

Seven guidelines (283 recommendations) were analyzed, and four of them were considered "recommended for use" (three reported after 2017 as excellent). Overall, 144 (50.9%) recommendations were based on low-quality evidence, representing 81.5% in diagnosis and 36.2% in therapy. In contrast, 5/92 (5.4%) and 40/191 (20.9%) recommendations regarding diagnostic and treatment (respectively) were based on high-quality evidence. Quality agreement ratings were significantly (p< 0.05) higher for guidelines delivered after 2015, progressing from 27.7% to 58.3%, qualifying as excellent. Highest scores were documented in the domains of "scope and purpose" followed by "clarifying of presentation" and "editorial independence".

CONCLUSION

Updated guidelines reported after 2017 improved quality, although well-designed randomized clinical trials remain an unmet need. AGREE II quality assessment identified four guidelines qualified as recommended for use. Improvements are required in stakeholder involvement and applicability.

Assessing the quality of evidence in studies estimating prevalence of exposure to occupational risk factors: The QoE-SPEO approach applied in the systematic reviews from the WHO/ILO Joint Estimates of the Work-related burden of disease and Injury

BACKGROUND

The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Quality of Evidence in Studies estimating Prevalence of Exposure to Occupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further.

METHODS

Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (P_i) between individual raters and rater teams for the downgrade domains. P_i ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews.

RESULTS

We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range P_i: 0.36-1.00) and rater teams (0.20-1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings.

CONCLUSIONS

We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area.

GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines

BACKGROUND AND OBJECTIVE

This is the 24th in the ongoing series of articles describing the GRADE approach for assessing the certainty of a body of evidence in systematic reviews and health technology assessments and how to move from evidence to recommendations in guidelines.

METHODS

Guideline developers and authors of systematic reviews and other evidence syntheses use randomized controlled studies (RCTs) and non-randomized studies of interventions (NRSI) as sources of evidence for questions about health interventions. RCTs with low risk of bias are the most trustworthy source of evidence for estimating relative effects of interventions because of protection against confounding and other biases. However, in several instances, NRSI can still provide valuable information as complementary, sequential, or replacement evidence for RCTs.

RESULTS

In this article we offer guidance on the decision regarding when to search for and include either or both types of studies in systematic reviews to inform health recommendations.

CONCLUSION

This work aims to help methodologists in review teams, technology assessors, guideline panelists, and anyone conducting evidence syntheses using GRADE.

The methodological quality and clinical applicability of meta-analyses on probiotics in 2020: A cross-sectional study

Systematic reviews with meta-analyses (SR/MA) are frequently conducted to investigate clinical efficacy of probiotics. However, only rigorously prepared analyses can serve as the highest level of evidence for a specified research question. We have aimed to determine (1) what is the methodological quality of recent SR/MA conducted to assess the efficacy of probiotics; (2) whether the results of SR/MA have a clinical application; and (3) what are factors associated with better quality and applicability of the SR/MA. We systematically searched 4 databases for SR/MA on the probiotics efficacy published in 2020 (PROSPERO CRD42020222716). The AMSTAR 2 tool and pre-defined authors' criteria were used to evaluate methodological quality and clinical applicability, respectively. A total of 114 SR/MA were appraised. In the case of 88 papers (77%), the overall confidence in the results was rated as "critically low". The most prevalent flaws were lack of list of excluded studies with justification (79.8%), lack of study protocol (60.5%), and problems with appropriate results combination(54.4%). A declaration of conduction a probiotic efficacy SR/MA could have been misleading in case of 18 studies that included also synbiotics, paraprobiotics, and prebiotics trials in analyses. Only 14 SR/MA provided results that can be apply in clinical practice. Higher journal impact factor and European affiliation of the 1st and corresponding authors were most consistently associated with higher odds of AMSTAR 2 items fulfillments. Based on our findings, SR/MA of probiotics trials cannot be treated as the highest level of evidence without a careful evaluation of their methodological validity.

[GRADE Guidelines: 19. Assessing the certainty of evidence in the importance of outcomes or values and preferences: Risk of bias and indirectness]

OBJECTIVES

The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group defines patient values and preferences as the relative importance patients place on the main health outcomes. We provide GRADE guidance for assessing the risk of bias and indirectness domains for certainty of evidence about the relative importance of outcomes.

STUDY DESIGN AND SETTING

We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted GRADE members and other stakeholders for feedback.

RESULTS

This is the first of two articles. A body of evidence addressing the importance of outcomes starts at "high certainty"; concerns with risk of bias, indirectness, inconsistency, imprecision, and publication bias lead to downgrading to moderate, low, or very low certainty. We propose the following subdomains of risk of bias: selection of the study population, missing data, the type of measurement instrument, and confounding; we have developed items for each subdomain. The population, intervention, comparison, and outcome elements associated with the evidence determine the degree of indirectness.

CONCLUSION

This article provides guidance and examples for rating the risk of bias and indirectness for a body of evidence summarizing the importance of outcomes.

Are clinical practice guidelines for low back pain interventions of high quality and updated? A systematic review using the AGREE II instrument

BACKGROUND

Clinical practice guidelines (CPGs) provide recommendations for practice, but the proliferation of CPGs issued by multiple organisations in recent years has raised concern about their quality. The aim of this study was to systematically appraise CPGs quality for low back pain (LBP) interventions and to explore inter-rater reliability (IRR) between quality appraisers. The time between systematic review search and publication of CPGs was recorded.

METHODS

Electronic databases (PubMed, Embase, PEDro, TRIP), guideline organisation databases, websites, and grey literature were searched from January 2016 to January 2020 to identify GPCs on rehabilitative, pharmacological or surgical intervention for LBP management. Four independent reviewers used the Appraisal of Guidelines for Research and Evaluation II (AGREE II) tool to evaluate CPGs quality and record the year the CPGs were published and the year the search strategies were conducted.

RESULTS

A total of 21 CPGs met the inclusion criteria and were appraised. Seven (33%) were broad in scope and involved surgery, rehabilitation or pharmacological intervention. The score for each AGREE II item was: Editorial Independence (median 67%, interquartile range [IQR] 31-84%), Scope and Purpose (median 64%, IQR 22-83%), Rigour of Development (median 50%, IQR 21-72%), Clarity and Presentation (median 50%, IQR 28-79%), Stakeholder Involvement (median 36%, IQR 10-74%), and Applicability (median 11%, IQR 0-46%). The IRR between the assessors was nearly perfect (interclass correlation 0.90; 95% confidence interval 0.88-0.91). The median time span was 2 years (range, 1-4), however, 38% of the CPGs did not report the coverage dates for systematic searches.

CONCLUSIONS

We found methodological limitations that affect CPGs quality. In our opinion, a universal database is needed in which guidelines can be registered and recommendations dynamically developed through a living systematic reviews approach to ensure that guidelines are based on updated evidence.

LEVEL OF EVIDENCE

1 TRIAL REGISTRATION: REGISTRATION PROSPERO DETAILS: CRD42019127619 .

Evaluation of the quality of evidence supporting guideline recommendations for the nutritional management of critically ill adults

AIMS

The primary aim of this study was to evaluate the quality of evidence supporting the 2019 European Society for Clinical Nutrition and Metabolism (ESPEN) and 2016 American Society for Parenteral and Enteral Nutrition (ASPEN) recommendations for medical nutrition therapy in critically ill patients. Secondary objectives are to assess the differences between 2019 ESPEN and 2016 ASPEN recommendations and to inform relevant stakeholders of areas requiring improvement in the research.

METHODS

The 2019 ESPEN and 2016 ASPEN guidelines were identified and downloaded from the official websites. The level of evidence and strength of recommendations from the guidelines were standardised to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) system. Level of evidence was classified as high-quality (randomised controlled trials (RCTs) without important limitations), moderate-quality (downgraded RCTs or upgraded observational studies) or low-quality (observational studies without specific strengths or important limitations, case series, case reports). In addition, good practice points (GPP; recommendations based on the clinical experience of the guideline development group) were considered. Strength of recommendation was reported as strong or weak.

RESULTS

From 152 total recommendations, only five (3.3%) were supported by high-quality evidence, with 14 being strong recommendations. A total of 79 (52.0%) recommendations were GPPs. Overall, the proportion of recommendations supported by high-quality (7% [ESPEN] vs. 1.1% [ASPEN], p < 0.05) and moderate-quality evidence (33.3% [ESPEN] vs. 8.4% [ASPEN], p < 0.01) was significantly higher in ESPEN guidelines. On the other hand, ASPEN guidelines reported a greater proportion of recommendations supported by GPPs (58.9% [ASPEN] vs. 40.4% [ESPEN], p = 0.03). In enteral and parenteral nutrition, the proportion of recommendations supported by moderate-quality evidence (50% [ESPEN] vs. 15.8% [ASPEN], p < 0.01) was significantly higher in ESPEN guidelines.

CONCLUSION

Published guideline recommendations for the nutritional management of critically ill adults remain largely supported by expert opinion and only a minority by high-quality evidence. An urgent unmet clinical need for high-quality clinical trials is warranted.

[Quality deficits of drug trials for older patients: An analysis of a series of systematic reviews]

INTRODUCTION

The aim of this work is to present an exemplary methodological review of the quality of included studies on drug therapy in older patients, using a published series of six systematic reviews (SRs). These six SRs included 48 systematic reviews, 65 intervention studies and 33 observational studies. The series of SRs has been carried out in the PRIMA-eDS-project (www.prima-eds.eu) to develop recommendations for the treatment of elderly patients with polypharmacy.

METHODS

The research question was to which extent recommendations on drug therapy in older patients are based on sound evidence. To this purpose, we performed a quality assessment of all studies included using AMSTAR for systematic reviews, CASP for observational studies, and the Cochrane "Risk of Bias" tool for intervention studies.

RESULTS

The evidence base for commonly prescribed drugs in the elderly is weak. The studies identified by the systematic reviews revealed a significant lack of studies addressing the target population as well as a lack of high-quality evidence. Among the 33 observational studies, it was unclear in nearly half of the publications whether the follow-up was sufficiently long and complete. For one-third, the conclusions did not match the observed evidence. The greatest risk of bias in the intervention trials was due to selection and incorrect blinding. Quality deficits of the systematic reviews consisted in the provision of a complete study list and the lack of consideration of potential publication bias.

DISCUSSION

Overall, many methodological deficits were revealed, making it difficult or almost impossible to derive reliable recommendations.

CONCLUSION

Our work illustrates the immense need for research in the treatment of older patients as well as the importance of ensuring the highest quality standards when conducting intervention and observational studies or carrying out systematic reviews.

Quality of evidence matters: is it well reported and interpreted in infertility journals?

PURPOSE

To evaluate if the authors of published systematic reviews (SRs) reported the level of quality of evidence (QoE) in the top 5 impact factor infertility journals and to analyze if they used an appropriate wording to describe it.

METHODS

This is a cross-sectional study. We searched in PubMed for SRs published in 2017 in the five infertility journals with the highest impact factor. We analyzed the proportion of SRs published in the top 5 impact factor infertility journals that reported the SRs' QoE, and the proportion of those SRs in which authors used consistent wording to describe QoE and magnitude of effect.

RESULTS

The QoE was reported in only 21.4% of the 42 included SRs and in less than 10% of the abstracts. Although we did not find important differences in the report of QoE of those that showed statistically significant differences or not, p value was associated with the wording chosen by the authors. We found inconsistent reporting of the size the effect estimate in 54.8% (23/42) and in the level of QoE in 92.9% (39/42). Whereas the effect size was more consistently expressed in studies with statistically significant findings, QoE was better expressed in those cases in which the p value was over 0.05.

CONCLUSION

We found that in 2017, less than 25% of the authors reported the overall QoE when publishing SRs. Authors focused more on statistical significance as a binary concept than on methodological limitations like study design, imprecision, indirectness, inconsistency, and publication bias. Authors should make efforts to report the QoE and interpret results accordingly.

Evidence supporting recommendations from international guidelines on treatment, diagnosis, and prevention of HAP and VAP in adults

Clinical practice guidelines (CPGs) are intended to support clinical decisions and should be based on high-quality evidence. The objective of the study was to evaluate the quality of evidence supporting the recommendations issued in CPGs for therapy, diagnosis, and prevention of hospital-acquired and ventilator-associated pneumonia (HAP/VAP). CPGs released by international scientific societies after year 2000, using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) methodology, were analyzed. Number and strength of recommendations and quality of evidence (high, moderate, low, and very low) were extracted and indexed in the aforementioned sections. High-quality evidence was based on randomized control trials (RCT) without important limitations and exceptionally on rigorous observational studies. Eighty recommendations were assessed, with 7 (8.7%), 24 (30.0%), 29 (36.3%), and 20 (25.0%) being supported by high, moderate, low, and very low-quality evidence, respectively. Highest evidence degree was reported for 26 prevention recommendations, with 7 (26.9%) supported by high-quality evidence and no recommendation based on very low-quality evidence. In contrast, among 9 recommendations for diagnosis and 45 for therapy, none was supported by high-quality evidence, in spite of being recommended as strong in 33.3% and 46.7%, respectively. Among HAP/VAP diagnosis recommendations, the majority of evidence was rated as low or very low-quality (55.6% and 22.2%, respectively) whereas among HAP/VAP therapy recommendations, 4/5 were rated as low and very low-quality (40% each). In conclusion, among HAP/VAP international guidelines, most recommendations, particularly in therapy, remain supported by observational studies, case reports, and expert opinion. Well-designed RCTs are urgently needed.

[Urolithiasis research-big data and artificial intelligence : How we can use the new structures of the medical informatics initiative of the Federal Ministry of Education and Research]

BACKGROUND

The digital transformation of society has a tremendous impact on both medicine and healthcare. The generation and processing of continuously growing amounts of digital data can be used to facilitate new approaches in research, particularly for healthcare research of common diseases such as urolithiasis.

OBJECTIVE

Presentation of the design of the German medical informatics initiative (MI-I) and the resulting possibilities for healthcare research in the field of urolithiasis.

RESULTS

For a meaningful utilization patient data must be readily available for research purposes and suitable methods for the analysis, interpretation and utilization must be developed. The aim of the German MI‑I is to make patient data collected during hospitalization available for research. The formation of so-called data integration centers will create a digital network that will facilitate the utilization and exchange of data between institutions. Advanced artificial intelligence algorithms will be used for analysis of the immense amounts of data. In May 2019 the German Federal Ministry of Education and Research boosted funding for the set-up of a digital nationwide registry on urolithiasis where the data foundation is to be generated based on the digital infrastructure of the MI‑I. The registry is intended to answer questions from the three relevant areas: "impact of the disease on individuals and society" (e.g. limitations in the quality of life and ability to work, treatment costs), "identification of further risk factors" (e.g. personalized medicine, especially taking account of lifestyle and nutrition, development of a risk score) and "evaluation of various treatment approaches" (which is the best treatment for an individual patient?).

Risk of bias assessment of sequence generation: a study of 100 systematic reviews of trials

BACKGROUND

Systematic reviews of randomised trials guide policy and healthcare decisions. Yet, we observed that some reviews judge randomised trials as high or unclear risk of bias (ROB) for sequence generation, potentially introducing bias. However, to date, the extent of this issue has not been well examined. We evaluated the consistency in the ROB assessment for sequence generation of randomised trials in Cochrane and non-Cochrane reviews, and explored the reviewers' judgement of the quality of evidence for the related outcomes.

METHODS

Cochrane intervention reviews (01/01/2017-31/03/2017) were retrieved from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in ten general medical journals with highest impact factors (01/01/2016-31/03/2017). We examined the proportion of reviews that rated the sequence generation domain as high, low or unclear risk of selection bias. For reviews that had rated any randomised trials as high or unclear risk of bias, we examined the proportion that had assessed the quality of evidence.

RESULTS

Overall, 100 systematic reviews were included in our analysis. We evaluated 64 Cochrane reviews which comprised of 984 randomised trials; 0.8% (n = 8) and 52.2% (n = 514) were rated as high and unclear ROB for sequence generation respectively. We further evaluated 36 non-Cochrane reviews which comprised of 1376 trials; 5.8% (n = 80) and 39.6% (n = 545) were rated as high and unclear ROB respectively. Ninety percent (n = 10) of non-Cochrane reviews which rated randomised trials as high ROB for sequence generation did not report an underlying reason. All Cochrane reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only just over half had assessed the quality of evidence.

CONCLUSION

Systematic reviews of interventions frequently rate randomised trials as high or unclear ROB for sequence generation. In general, Cochrane reviews were more transparent than non-Cochrane reviews in ROB and quality of evidence assessment. The scientific community should more strongly promote consistent ROB assessment for sequence generation to minimise selection bias and support transparent quality of evidence assessment. Consistency ensures that appropriate conclusions are drawn from the data.

[Risk of bias assessment: (9) Application of the risk of bias assessment results]

In this last paper of the series about risk of bias assessment, we introduce the application of risk of bias assessment results. Risk of bias assessment is one of the key steps in the assessment of quality of evidence. The risk of bias assessment results could be the "diagnosis" of individual studies, which helps decision making related to the inclusion and exclusion of individual studies, as well as the data analysis in the systematic review process. This paper focuses on how to incorporate risk of bias assessment results in the GRADE assessment for quality of evidence, including the principles and the tips for the application.

Certainty ranges facilitated explicit and transparent judgments regarding evidence credibility

OBJECTIVES

The Grading of Recommendations Assessment, Development and Evaluation approach to rating certainty of evidence includes five domains of reasons for rating down certainty. Only one of these, precision, is easily amenable-through the confidence interval-to quantitation. The other four (risk of bias, inconsistency, indirectness, and publication bias) are not. Nevertheless, conceptually, one could consider a quantified "certainty range" within which the true effect lies. The certainty range would be at least as wide as the confidence interval and would expand with each additional reason for uncertainty.

STUDY DESIGN AND SETTING

We have applied this concept to rating the certainty of evidence in the baseline risk of venous thromboembolism (VTE) and bleeding in patients undergoing urological surgery. We considered rating up moderate or low quality evidence when the net benefit of VTE prophylaxis was unequivocally positive, that is, when the smallest plausible value of VTE reduction was greater than the largest plausible value of increased bleeding. To establish whether the net benefit was unequivocally positive, we expanded the range of plausible values by 20% for each of the four nonquantitative domains in which there were serious limitations.

RESULTS

We present how we applied these methods to examples of open radical cystectomy and laparoscopic partial nephrectomy. In high-VTE risk laparoscopic partial nephrectomy patients and high- and medium-VTE risk open radical cystectomy patients, results proved robust to expanded certainty intervals, justifying rating up quality of evidence. In low-risk patients, the results were not robust, and rating up was therefore not appropriate.

CONCLUSION

This work represents the first empirical application in a decision-making context of the previously suggested concept of certainty ranges and should stimulate further exploration of the associated theoretical and practical issues.

Considerations for Assessment and Applicability of Studies of Intervention

Evidence-based practice changed how health care professionals are taught medical practice. Despite all best intentions, many health care professionals have limited knowledge to assess the validity and usefulness of the medical literature. The goal of this article is to review the following basic considerations while appraising studies of intervention: (1) the hierarchy of evidence and the strengths and weaknesses of clinical studies, (2) the relevant elements of study design that impact validity of study results, and (3) the spectrum of efficacy and effectiveness of clinical studies and how these influence the applicability of study results to individual patients.

Blood pressure targets for hypertension in patients with type 2 diabetes

BACKGROUND

Clinical guidelines vary in determining optimal blood pressure targets in adults with diabetes mellitus.

METHODS

We systematically searched PubMed, EMBASE, Cochrane Library, and clinicaltrials.gov in March 2018; conducted random effects frequentist meta-analyses of direct aggregate data; and appraised the quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology.

RESULTS

From eligible 14 meta-analyses and 95 publications of randomized controlled trials (RCT), only 6 RCTs directly compared lower versus higher blood pressure targets; remaining RCTs aimed at comparative effectiveness of hypotensive drugs. In adults with diabetes mellitus and elevated systolic blood pressure (SBP), direct evidence (2 RCTs) suggests that intensive target SBP <120-140 mmHg decreases the risk of diabetes-related mortality [relative risk (RR) =0.68; 95% confidence interval (CI), 0.50-0.92], fatal (RR =0.41; 95% CI, 0.20-0.84) or nonfatal stroke (RR =0.60; 95% CI, 0.43-0.83), prevalence of left ventricular hypertrophy and electrocardiogram (ECG) abnormalities, macroalbuminuria, and non-spine bone fractures, with no differences in all-cause or cardiovascular mortality or falls. In adults with diabetes mellitus and elevated diastolic blood pressure (DBP) ≥90 mmHg, direct evidence (2 RCTs) suggests that intensive DBP target ≤80 versus 80-90 mmHg decreases the risk of major cardiovascular events. Published meta-analyses of aggregate data suggested a significant association between lower baseline and attained blood pressure and increased cardiovascular mortality.

CONCLUSIONS

We concluded that in adults with diabetes mellitus and arterial hypertension, in order to reduce the risk of stroke, clinicians should target blood pressure at 120-130/80 mmHg, with close monitoring for all drug-related harms.

Sensitivity of treatment recommendations to bias in network meta-analysis

Network meta-analysis (NMA) pools evidence on multiple treatments to estimate relative treatment effects. Included studies are typically assessed for risk of bias; however, this provides no indication of the impact of potential bias on a decision based on the NMA. We propose methods to derive bias adjustment thresholds which measure the smallest changes to the data that result in a change of treatment decision. The methods use efficient matrix operations and can be applied to explore the consequences of bias in individual studies or aggregate treatment contrasts, in both fixed and random-effects NMA models. Complex models with multiple types of data input are handled by using an approximation to the hypothetical aggregate likelihood. The methods are illustrated with a simple NMA of thrombolytic treatments and a more complex example comparing social anxiety interventions. An accompanying R package is provided.

Effects of atypical antipsychotic drugs on QT interval in patients with mental disorders

Background

Drug-induced QT prolongation is associated with higher risk of cardiac arrhythmias and cardiovascular mortality. We investigated the effects of atypical antipsychotic drugs on QT interval in children and adults with mental disorders.

Methods

We conducted random-effects direct frequentist meta-analyses of aggregate data from randomized controlled trials (RCT) and appraised the quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology. Our search in PubMed, EMBASE, the Cochrane Library, clinicaltrials.gov, and PharmaPendium up to October 2017 identified studies that examined aripiprazole, quetiapine, risperidone, olanzapine, ziprasidone and brexpiprazole.

Results

Low quality evidence suggests that aripiprazole (four meta-analyses and twelve RCTs), brexpiprazole (one systematic review and four RCTs) or olanzapine (five meta-analyses and twenty RCTs) do not increase QT interval. Low quality evidence suggests that ziprasidone (five meta-analyses and 11 RCTs) increases QT interval and the rates of QT prolongation while risperidone (four meta-analyses, 70 RCTs) and quetiapine (two meta-analyses and seven RCTs) are associated with QT prolongation and greater odds of torsades de pointes ventricular tachycardia especially in cases of drug overdose.

Conclusions

The main conclusion of our study is that in people with mental disorders and under treatment with atypical antipsychotic drugs, in order to avoid QT prolongation and reduce the risk of ventricular tachycardia clinicians may recommend aripiprazole, brexpiprazole or olanzapine in licensed doses. Long-term comparative safety needs to be established.

GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence

OBJECTIVE

To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process.

STUDY DESIGN AND SETTING

The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group.

RESULTS

We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I.

CONCLUSIONS

The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.

Comparative effectiveness and safety of empagliflozin on cardiovascular mortality and morbidity in adults with type 2 diabetes

Background

Based on a single placebo-controlled randomized clinical trial, empagliflozin is licensed to reduce cardiovascular death in diabetes and comorbid cardiovascular disease.

Methods

We examined the comparative effectiveness of empagliflozin on mortality and cardiovascular morbidity in type 2 diabetes. We conducted random-effects direct frequentist meta-analyses of aggregate data and appraised the quality of evidence using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology. Our search in PubMed, EMBASE, the Cochrane Library, clinicaltrials.gov, and PharmaPendium up to May 2017 identified 11 meta-analyses, multiple publications, and unpublished data from 29 randomized controlled trials (RCTs).

Results

Empagliflozin reduces all-cause mortality [relative risk (RR) of death, 0.69; 95% confidence interval (CI): 0.58-0.82; number needed to treat (NNT) to postpone mortality in one patient, 39; 95% CI: 26-79; 1 RCT of 7,020 patients) in patients with but not without (RR, 0.90; 95% CI: 0.36-2.23; 14 RCTs of 7,707 patients) established cardiovascular disease when compared with placebo. Empagliflozin reduces cardiovascular mortality (RR, 0.62; 95% CI: 0.50-0.78; NNT, 45; 95% CI: 30-90; 1 RCT of 7,020 patients) in patients with but not without (RR, 0.98; 95% CI: 0.29-3.33; 10 RCTs of 5,429 patients) established cardiovascular disease when compared with placebo. There are no differences in cardiovascular morbidity and mortality and all-cause mortality between empagliflozin and metformin (4 RCTs of 1,344 patients), glimepiride (1 RCT of 1,549 patients), linagliptin (2 RCTs of 1,348 patients), or sitagliptin (3 RCTs of 1,483 patients). Two network meta-analyses concluded that sodium-glucose cotransporter 2 (SGLT2) inhibitors, mostly due to empagliflozin, decrease all-cause and cardiovascular mortality but increase the risk of nonfatal stroke, genital infection, and volume depletion.

Conclusions

We conclude that empagliflozin reduces all-cause and cardiovascular mortality in patients with established cardiovascular disease and type 2 diabetes. Sparse direct evidence suggests no difference in mortality between empagliflozin and metformin, glimepiride, linagliptin, or sitagliptin. Long-term comparative safety needs to be established.

Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis

This article describes conceptual advances of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group guidance to evaluate the certainty of evidence (confidence in evidence, quality of evidence) from network meta-analysis (NMA). Application of the original GRADE guidance, published in 2014, in a number of NMAs has resulted in advances that strengthen its conceptual basis and make the process more efficient. This guidance will be useful for systematic review authors who aim to assess the certainty of all pairwise comparisons from an NMA and who are familiar with the basic concepts of NMA and the traditional GRADE approach for pairwise meta-analysis. Two principles of the original GRADE NMA guidance are that we need to rate the certainty of the evidence for each pairwise comparison within a network separately and that in doing so we need to consider both the direct and indirect evidence. We present, discuss, and illustrate four conceptual advances: (1) consideration of imprecision is not necessary when rating the direct and indirect estimates to inform the rating of NMA estimates, (2) there is no need to rate the indirect evidence when the certainty of the direct evidence is high and the contribution of the direct evidence to the network estimate is at least as great as that of the indirect evidence, (3) we should not trust a statistical test of global incoherence of the network to assess incoherence at the pairwise comparison level, and (4) in the presence of incoherence between direct and indirect evidence, the certainty of the evidence of each estimate can help decide which estimate to believe.

The effect of blinding on estimates of mortality in randomised clinical trials of intensive care interventions: protocol for a systematic review and meta-analysis

INTRODUCTION

Evidence exists that unblinded randomised clinical trials (RCTs) overestimate intervention effects compared with blinded RCTs. It has been suggested that this is less pronounced for objective (ie, not subject to interpretation) outcome measures, including mortality. This may not apply in the intensive care unit (ICU), as most deaths are preceded by decisions to withhold or withdraw treatments. Lack of blinding of physicians in RCTs of ICU interventions may potentially influence the decision towards a higher threshold for discontinuing treatment in patients who receive the investigational treatment and/or a lower threshold for discontinuing treatment in patients who receive the comparator (control). This may have important implications for patients, caregivers, researchers and society. Accordingly, we aim to assess whether lack of blinding affects mortality effect estimates in RCTs of ICU interventions.

METHODS AND ANALYSIS

We will conduct a systematic review with meta-analyses and assess the effect of blinding versus no blinding on mortality effect estimates in RCTs of interventions used in adult ICU patients.We will systematically search the Cochrane Library for systematic reviews reporting mortality effect estimates of any intervention used in adult ICU patients which includes at least one RCT with 'low risk of bias' in the bias domains 'blinding of participants and personnel' and/or 'blinding of outcome assessment' and one RCT with 'unclear' or 'high risk of bias' in the same bias domain(s). For each intervention, we will compare summary mortality effect estimates in blinded versus unblinded trials.

ETHICS AND DISSEMINATION

This research does not require ethical approval as we will use summary data from trials already approved by relevant ethical institutions. We will report the results in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement and submit the final paper to an international peer-reviewed journal.

TRIAL REGISTRATION NUMBER

PROSPERO, registration number: CRD42017056212.

Benefits and harms of atypical antipsychotics for agitation in adults with dementia

We evaluated the most current evidence regarding the benefits and harms of atypical antipsychotics in adults with dementia. In June 2016, following a protocol developed a priori, we systematically searched several databases for published and unpublished data from randomized controlled trials (RCT), observational studies, and meta-analyses; conducted direct meta-analyses using a random effects model; and graded the quality of evidence according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group. One high-quality meta-analysis and published and unpublished data from 8 RCTs and 12 large observational studies met inclusion criteria. When compared with placebo, aripiprazole, risperidone, and olanzapine but not quetiapine result in modest (standardized mean difference <0.5 standard deviations) improvement in neuropsychiatric symptoms. Aripiprazole, risperidone, quetiapine, and olanzapine are associated with increased odds of acute myocardial infraction, and risperidone and olanzapine are associated with increased odds of hip fracture. Observational studies suggest no differences in all-cause mortality between atypical antipsychotics. Observational studies suggest that atypical antipsychotics are associated with lower risk of all-cause mortality and extrapyramidal symptoms but higher risk of stroke when compared with conventional antipsychotics. To manage agitation in adults with progressive dementia, clinicians may recommend atypical antipsychotics with continuous monitoring of behavioral symptoms, informing patients and their families or caregivers of the significant risk of adverse effects and baseline risk of acute myocardial infraction and bone fractures.

Chinese herbal medicine for the treatment of primary hypertension: a methodology overview of systematic reviews

BACKGROUND

Chinese herbal medicine has been used to treat hypertension in China and East Asia since centuries. In this study, we conduct an overview of systematic reviews of Chinese herbal medicine in the treatment of primary hypertension to 1) summarize the conclusions of these reviews, 2) evaluate the methodological quality of these reviews, and 3) rate the confidence in the effect on each outcome.

METHODS

We comprehensively searched six databases to retrieve systematic reviews of Chinese herbal medicine for primary hypertension from inception to December 31, 2015. We used AMSTAR to evaluate the methodological quality of included reviews, and we classified the quality of evidence for each outcome in included reviews using the GRADE approach.

RESULTS

A total of 12 systematic reviews with 31 outcomes were included, among which 11 systematic reviews focus on the therapeutic effect of Chinese herbal medicine combined with conventional medicine or simple Chinese herbal medicine versus simple conventional medicine. Among the 11 items of AMSTAR, the lowest quality was "providing a priori design" item, none review conformed to this item, the next was "stating the conflict of interest" item, only three reviews conformed to this item. Five reviews scored less than seven in AMSTAR, which means that the overall methodological quality was fairly poor. For GRADE, of the 31 outcomes, the quality of evidence was high in none (0 %), moderate in three (10 %), low in 19 (61 %), and very low in nine (29 %). Of the five downgrading factors, risk of bias (100 %) was the most common downgrading factor in the included reviews, followed by imprecision (42 %), inconsistency (39 %), publication bias (39 %), and indirectness (0 %).

CONCLUSIONS

The methodological quality of systematic reviews about Chinese herbal medicine for primary hypertension is fairly poor, and the quality of evidence level is low. Physicians should be cautious when applying the interventions in these reviews for primary hypertension patients in clinical practice.

Considering evidence: The approach taken by the Hazardous Substances Advisory Committee in the UK

The Hazardous Substances Advisory Committee (HSAC) provides expert advice to UK officials, Ministers and other relevant bodies on the protection of the environment, and human health via the environment, from potentially hazardous substances and articles. Hazardous substances are often the subject of controversy, on which individuals, and different groups in society, hold divergent views. This paper details the approach taken by HSAC when considering the evidence to provide advice on hazardous substances. Firstly HSAC reviews the range of evidence and determines its quality considering: transparency of aims, the methodology and results, completeness, independent review and accessibility. HSAC does not follow one explicit methodology as the wide range of hazardous substances we consider means they need to be addressed on a case by case basis. Most notably HSAC considers the evidence in the wider context, being aware of factors that influence individuals in their decision making when receiving a HSAC opinion e.g. trust in the source of the evidence, defensibility, conformity to a 'world view' and framing. HSACs also reflect on its own perspectives with the aim of addressing bias by the diversity of its membership. The Committee's intention, in adopting this rounded approach, is to reach opinions that are robust, relevant and defensible.

Guidelines on vitamin D replacement in bariatric surgery: Identification and systematic appraisal

INTRODUCTION

Bariatric surgery is the most effective therapeutic option to reduce weight in morbidly obese individuals, but it results in a number of mineral and vitamin deficiencies. Clinical Practice Guidelines (CPGs) attempt to balance those benefits and harms to provide guidance to physicians and patients.

OBJECTIVES

We compare and evaluate the quality of the evidence and of the development process of current CPGs that provide recommendations on vitamin D replacement in patients undergoing bariatric surgery, using a validated tool.

METHODS

We searched 4 databases, with no time restriction, to identify relevant and current CPGs. Two reviewers assessed eligibility and abstracted data, in duplicate. They evaluated the quality of CPGs development process using the Appraisal of Guidelines, Research, and Evaluation II (AGREE II) tool that consists of 6 domains. A content expert verified those assessments.

RESULTS

We identified 3 eligible CPGs: (1) the Endocrine Society (ES) guidelines (2010); (2) the American Association of Clinical Endocrinologists (AACE), The Obesity Society (TOS), and the American Society for Metabolic & Bariatric Surgery (ASMBS) guidelines (update 2013); and (3) the Interdisciplinary European (IE) guidelines on Metabolic and Bariatric Surgery (latest update 2014). The ES and the AACE/TOS/ASMBS guidelines recommended high doses of vitamin D, varying from 3000IU daily to 50,000IU 1-3 times weekly. Vitamin D doses were not mentioned in the IE guidelines. The recommendations were based on a low quality of evidence, if any, or limited to a single high quality trial, for some outcomes. In terms of quality, only the IE guidelines described their search methodology but none of the CPGs provided details on evidence selection and appraisal. None of the three CPGs rigorously assessed the preferences of the target population, resource implications, and the applicability of these guidelines. According to the AGREE II tool, we rated the ES guidelines as average in quality, and the other two as low in quality.

CONCLUSION

Current CPGs recommendations on vitamin D supplementation in bariatric surgery differ between societies. They do not fulfill criteria for optimal guideline development, in part possibly due to limited resources, and are based on expert opinion. Thus, the pressing need for high quality randomized trials to inform CPGs, to be developed based on recommended standards.

Outcomes in systematic reviews of complex interventions never reached "high" GRADE ratings when compared with those of simple interventions

OBJECTIVES

To investigate the application of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach and the quality of evidence ratings in systematic reviews of complex interventions.

STUDY DESIGN AND SETTING

This study examined all 40 systematic reviews published in three Cochrane Review Groups from 2013 to May 2014: Cochrane Developmental, Psychosocial and Learning Problems Group (CDPLPG); Cochrane Public Health Group (CPHG); and Cochrane Depression, Anxiety, and Neurosis Group (CCDAN). The reviews were coded and classified into "complex" (n = 24) and "simple" (n = 16) intervention review groups based on the predefined complexity dimensions from the extant literature mapped into the PICOTS framework. All the data were analyzed in these two groups to help identify specific patterns of the GRADE ratings in the reviews of complex interventions.

RESULTS

Outcomes of complex intervention reviews had higher proportions of "very low" quality of evidence ratings compared with those of simple intervention reviews (37.5% vs. 9.1% for the primary benefit outcomes) and were more frequently downgraded for inconsistency, performance bias, and study design. None of the outcomes of complex intervention reviews (0%) were given "high" GRADE ratings.

CONCLUSION

Results suggest that the GRADE assessment may not adequately describe the evidence base of complex interventions.

An algorithm was developed to assign GRADE levels of evidence to comparisons within systematic reviews

OBJECTIVES

One recommended use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is supporting quality assessment of evidence of comparisons included within a Cochrane overview of reviews. Within our overview, reviewers found that current GRADE guidance was insufficient to make reliable and consistent judgments. To support our ratings, we developed an algorithm to grade quality of evidence using concrete rules.

METHODS

Using a pragmatic, exploratory approach, we explored the challenges of applying GRADE levels of evidence and developed an algorithm to applying GRADE levels of evidence in a consistent and transparent approach. Our methods involved application of algorithms and formulas to samples of reviews, expert panel discussion, and iterative refinement and revision.

RESULTS

The developed algorithm incorporated four key criteria: number of participants, risk of bias of trials, heterogeneity, and methodological quality of the review. A formula for applying GRADE level of evidence from the number of downgrades assigned by the algorithm was agreed.

CONCLUSION

Our algorithm which assigns GRADE levels of evidence using a set of concrete rules was successfully applied within our Cochrane overview. We propose that this methodological approach has implications for assessment of quality of evidence within future evidence syntheses.

The quality of evidence in Chinese meta-analyses needs to be improved

OBJECTIVES

We conducted a systematic review using GRADE (Grading of Recommendations Assessment, Development and Evaluation) system to assess the quality of evidence of Chinese meta-analyses (MAs).

DESIGN/SETTING

A systematic review of MAs listed in Chinese Biomedicine Literature Database from January 2010 to December 2012.

METHODS

Mesh term "meta-analysis" was used to search the Chinese Biomedicine Literature Database from January 2010 to December 2012. Characteristics and main outcomes of each included MA were extracted, and the GRADE system was used to assess the quality of evidence for each outcome. A 10% random sample of Cochrane MAs between 2010 and 2012 was also assessed as control group.

RESULTS

A total of 564 Chinese MAs (including 1,237 main outcomes) and 95 Cochrane MAs (including 251 main outcomes) were identified. Almost half (600, 48.5%) of the outcomes in Chinese MAs were rated as low, and the proportion of outcomes with high or moderate quality of evidence was lower in Chinese MAs than Cochrane MAs (Chinese, 406 [32.8%] vs. Cochrane MAs, 155 [61.8%], P < 0.001). Of the outcomes in Chinese MAs, 1,012 (81.8%) were downgraded for risk of bias. Other common factors for downgrading were imprecision (448, 36.2%), publication bias (418, 33.8%), inconsistency (351, 28.4%), and indirectness (1, 0.1%).

CONCLUSIONS

Chinese MAs were of low quality of evidence. Risk of bias, inconsistency, and publication bias were the three most common downgrade factors in Chinese MAs. Efforts must be made to improve quality of evidence of Chinese MAs, and a call for more rigorous training of investigators in China is warranted.

Acting on black box warnings requires a GRADE evidence table and an implementation guide: the case of teriparatide

OBJECTIVES

To assess how well do the black box warnings present and communicate evidence in a way that is consistent with evidence-based patient-centered practice, through evaluating the boxed warning on teriparatide-induced osteosarcoma.

STUDY DESIGN AND SETTING

We critically appraised teriparatide black box warning for osteosarcoma by reviewing human and animal studies that were used as basis for the warning. We also evaluated the quality of the evidence using the Grading of Recommendations Assessment, Development and Evaluation framework.

RESULTS

We found that this warning was based on very low-quality evidence that was derived primarily from animal studies. The quality of evidence was rated down because of high risk of bias in addition to inconsistency, indirectness, and imprecision of the estimates. The warning does not provide sufficient guidance for practice like an implementation tool or an evidence profile to clarify the limitations of the evidence.

CONCLUSION

Black box warning for teriparatide-associated osteosarcoma does not explicitly present the quality of evidence, and therefore, it could be of limited use in evidence-based practice. We propose that black box warnings should include an evidence profile and an implementation guide to be more useful in evidence-based patient-centered practice.

Quality of evidence is a key determinant for making a strong GRADE guidelines recommendation

OBJECTIVES

The objective of the study was to assess the association between Grading of Recommendations Assessment, Development and Evaluation (GRADE) factors and the strength of recommendations.

STUDY DESIGN AND SETTING

The study was conducted as part of the development of clinical practice guideline (CPG) by American Association of Blood Banking related to role of prophylactic vs. therapeutic transfusion for the management of thrombocytopenia. The association between GRADE factors and strength of recommendations was assessed using logistic regression and multilevel mixed effect logistic regression model.

RESULTS

Seventeen members of the CPG panel participated in the recommendation process. The quality of evidence was the only statistically significant (odds ratio = 4.5; P < 0.001) GRADE factor associated with the strength of recommendations. The predictive model showed that there is about 90% probability that panelists would issue the same (strong) recommendation when confidence in the effects of intervention is high vs. 10% when the quality of evidence is very low.

CONCLUSION

The results showed that quality of evidence is a key determinant for making a strong vs. a weak recommendation.

Representing and extracting lung cancer study metadata: study objective and study design

This paper describes the information retrieval step in Casama (Contextualized Semantic Maps), a project that summarizes and contextualizes current research papers on driver mutations in non-small cell lung cancer. Casama׳s representation of lung cancer studies aims to capture elements that will assist an end-user in retrieving studies and, importantly, judging their strength. This paper focuses on two types of study metadata: study objective and study design. 430 abstracts on EGFR and ALK mutations in lung cancer were annotated manually. Casama׳s support vector machine (SVM) automatically classified the abstracts by study objective with as much as 129% higher F-scores compared to PubMed׳s built-in filters. A second SVM classified the abstracts by epidemiological study design, suggesting strength of evidence at a more granular level than in previous work. The classification results and the top features determined by the classifiers suggest that this scheme would be generalizable to other mutations in lung cancer, as well as studies on driver mutations in other cancer domains.

Grades for quality of evidence were associated with distinct likelihoods that treatment effects will remain stable

OBJECTIVES

We sought to determine whether producers or users of systematic reviews using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach or a close variation give the same meanings to terms intended to convey uncertainty about treatment effects when interpreting grades for the quality or strength of evidence.

STUDY DESIGN AND SETTING

Following exploratory interviews with stakeholders and user testing, we conducted an international Web-based survey among producers and users of systematic reviews. For each quality grade (high, moderate, low, very low/insufficient), we asked participants to assign a minimum likelihood that treatment effects will not change substantially as new studies emerge. Using multivariate analysis of covariance, we tested whether the estimated likelihoods differed between producers and users.

RESULTS

In all, 244 participants completed the survey. The associated minimum likelihoods that treatment effects will not change substantially for high, moderate, and low grades of quality of evidence (QOE) were 86% [95% confidence interval (CI): 85%, 87%], 61% (95% CI: 59%, 63%), and 34% (95% CI: 32%, 36%), respectively (very low was preset at 0%). Likelihoods for each grade were similar between producers and users of systematic reviews (P > 0.05 for all comparisons).

CONCLUSION

GRADE is, in general, a suitable method to convey uncertainties for systematic review producers to users. The wide ranges of likelihoods associated with GRADE terms suggest that current definitions of levels of QOE that rely exclusively on qualitative certainty expressions should be augmented by numerical predictions once such data are available.

A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients

Objectives

Primary care patients typically have less severe illness than those in hospital and may be overtreated if clinical guideline evidence is inappropriately generalized. We aimed to assess whether guideline recommendations for primary care were based on relevant research.

Study Design and Setting

Literature review of all publications cited in support of National Institute for Health and Care Excellence (NICE) recommendations for primary care. The relevance to primary care of all 45 NICE clinical guidelines published in 2010 and 2011, and their recommendations, was assessed by an expert panel.

Results

Twenty-two of 45 NICE clinical guidelines published in 2010 and 2011 were relevant to primary care. These 22 guidelines contained 1,185 recommendations, of which 495 were relevant to primary care, and cited evidence from 1,573 research publications. Of these cited publications, 590 (38%, range by guideline 6–74%) were based on patients typical of primary care.

Conclusion

Nearly two-third (62%) of publications cited to support primary care recommendations were of uncertain relevance to patients in primary care. Guideline development groups should more clearly identify which recommendations are intended for primary care and uncertainties about the relevance of the supporting evidence to primary care patients, to avoid potential overtreatment.

Using GRADE for evaluating the quality of evidence in hyperbaric oxygen therapy clarifies evidence limitations

OBJECTIVES

The current evidence rating for hyperbaric oxygen therapy indications uses the American Heart Association system, which mainly depends on the study design.

STUDY DESIGN AND SETTING

We systematically reviewed the literature and applied the Grading of Evidence, Assessment, Development and Evaluation (GRADE) approach to the main patient-important outcomes in each indication.

RESULTS

We included 17 systematic reviews that synthesized 44 randomized trials and 131 observational studies enrolling 8,145 participants. The quality of evidence for seven indications with category A was high (1), moderate (2), low (2), and very low (2); for 10 indications with category B, it was moderate (1), low (5), and very low (4); and for 1 indication with category C, it was high. The quality of evidence was rated down for the risk of bias and imprecision for most indications and rated up because of large effect size for some indications. Most discrepant ratings were in the indications of decompression illness (C, high), carbon monoxide poisoning (A, very low), and later presentations of idiopathic sudden hearing loss (A, very low).

CONCLUSION

The GRADE approach uncovered factors affecting the quality of evidence that were otherwise implicit. Knowing these factors can influence clinicians' confidence in applying hyperbaric oxygen therapy and orient the research agenda.