Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study

Quality evaluation of French guidelines in primary care infectious disease: An AGREE II assessment

RATIONALE

Antibiotic prescription, its nature and its duration are a very common decision-making situation in primary care practice. Clinical practice guidelines (CPGs) are regularly emitted by various organisations on this topic.

AIMS AND OBJECTIVES

Our goal is to run a quality appraisal of the current French guidelines, for the most common primary care infectious pathologies.

METHOD

We collected all primary care CPGs that are currently prevailing in France through a systematic review of the french website Antibioclic®. For each of these guidelines, a quality assessment was run by 3 independent reviewers, by means of the Appraisal of Guidelines for REsearch & Evaluation II instrument. The main outcome was a 'reliability score', defined as the sum of the scores in domains 'rigour of development' and 'editorial independence'. To be considered 'reliable', the CPG had to reach a 60% threshold in these two domains. Secondary outcomes were as follows: global quality score of CPGs, number and ratio of CPGs for which a systematic review has been conducted during its conception.

RESULTS

Over the 43 CPGs that have been assessed, none reached the 60%-threshold as to the reliability score. Only one CPG (2.33%) gets an over-60% quality assessment in the domain of rigour of development (D3), whereas three CPGs (6.98%) reach this threshold in the domain of editorial independence (D6). One CPG (2.33%) met the quality threshold of 60% as to overall assessment. Rigour of development and editorial independence are the domains that obtained the lowest average score, respectively, 11% and 21%. Overall assessment received an average score of 29%. A systematic review of the literature was mentioned for 10 CPGs (23.26%).

CONCLUSION

There is a lack of quality in the development process of the current French guidelines in primary care infectiology. This process should be reconsidered, with higher insistence as to its quality.

The impact of Cochrane Reviews that apply network meta-analysis in clinical guidelines: A systematic review

BACKGROUND

Systematic reviews, such as those prepared by Cochrane, are the strongest evidence on which to base recommendations in clinical guidelines. Network meta-analysis (NMA) can be used to combine the results of studies to compare multiple treatments, which is advantageous over pair-wise meta-analysis (PW-MA) that compares two treatments. We aimed to summarise which, when, where, who, and why Cochrane Reviews that applied NMA were cited in guidelines; and to compare the citation of NMA reviews in guidelines with PW-MA reviews.

METHODS AND FINDINGS

We carried out a systematic review of Cochrane reviews that applied NMA and we summarised their citation in guidelines. The Cochrane Database of Systematic Reviews was searched (15th January 2024). Additionally, a cohort of Cochrane reviews that applied PW-MA was matched to the NMA reviews. Two authors assessed eligibility and extracted data. We summarised review and guideline characteristics, and the use of the review in guidelines.

RESULTS

Of the 60 included NMA reviews, 26 reviews (43%) were cited in 89 guidelines (1-13 per review). 15 NMA reviews (58%) were first cited within two years of publication, with the remaining 11 reviews (42%) cited 2-6 years later. 52 guideline developers authored the guidelines. The number of citations was higher for NMA than PW-MA reviews (rate ratio 1.53 (1.08 to 2.19), p = 0.02). The number of times reviews were commissioned or cited alongside a recommendation was also higher for NMA than PW-MA reviews (rate ratio 4.40 (1.80 to 13.14), p = 0.003). NMA reviews were more likely to be cited in the text surrounding a recommendation or used for NICE guideline development (1.94 (1.08 to 3.63), p = 0.03).

CONCLUSIONS

Cochrane NMA reviews appear to have more impact than PW-MA reviews, but many are not cited in guidelines. Further work is needed to explore the barriers to use of NMAs and promote their use.

):

Systematic reviews and meta-analyses have become crucial for evidence-based decision-making in recent decades. However, it is common for the results of multiple reviews on the same topic to be inconsistent, and it is widely recognised that the results of the reviews are not always effectively communicated to healthcare professionals and the lay public. This manuscript proposes a strategy to summarise and communicate the findings of previous systematic reviews and meta-analyses to wider audiences. The proposed approach couples the findings of umbrella reviews with the creation of open-access online platforms that present the results of these umbrella reviews in an accessible way to various stakeholders. The key potential methodological avenues of this approach are presented, and specific examples from the author's own works and those from other teams are provided. An accompanying website (https://u-reach.org/) has been designed to present this Umbrella-Review, Evaluation, Analysis, and Communication Hub (U-REACH) approach and to overcome the technical challenges associated with this type of project (by sharing the code used to build existing U-REACH projects). The present document is intended to serve as a methodological and technical guide for the creation of large-scale projects designed to synthesise and disseminate scientific information to a broad audience.

The optimal approach for retrieving systematic reviews was achieved when searching MEDLINE and Epistemonikos in addition to reference checking: a methodological validation study

BACKGROUND

Systematic reviews (SRs) are used to inform clinical practice guidelines and healthcare decision making by synthesising the results of primary studies. Efficiently retrieving as many relevant SRs as possible is challenging with a minimum number of databases, as there is currently no guidance on how to do this optimally. In a previous study, we determined which individual databases contain the most SRs, and which combination of databases retrieved the most SRs. In this study, we aimed to validate those previous results by using a different, larger, and more recent set of SRs.

METHODS

We obtained a set of 100 Overviews of Reviews that included a total of 2276 SRs. SR inclusion was assessed in MEDLINE, Embase, and Epistemonikos. The mean inclusion rates (% of included SRs) and corresponding 95% confidence intervals were calculated for each database individually, as well as for combinations of MEDLINE with each other database and reference checking. Features of SRs not identified by the best database combination were reviewed qualitatively.

RESULTS

Inclusion rates of SRs were similar in all three databases (mean inclusion rates in % with 95% confidence intervals: 94.3 [93.9-94.8] for MEDLINE, 94.4 [94.0-94.9] for Embase, and 94.4 [93.9-94.9] for Epistemonikos). Adding reference checking to MEDLINE increased the inclusion rate to 95.5 [95.1-96.0]. The best combination of two databases plus reference checking consisted of MEDLINE and Epistemonikos (98.1 [97.7-98.5]). Among the 44/2276 SRs not identified by this combination, 34 were published in journals from China, four were other journal publications, three were health agency reports, two were dissertations, and one was a preprint. When discounting the journal publications from China, the SR inclusion rate in the recommended combination (MEDLINE, Epistemonikos and reference checking) was even higher than in the previous study (99.6 vs. 99.2%).

CONCLUSIONS

A combination of databases and reference checking was the best approach to searching for biomedical SRs. MEDLINE and Epistemonikos, complemented by checking the references of the included studies, was the most efficient and produced the highest recall. However, our results point to the presence of geographical bias, because some publications in journals from China were not identified.

STUDY REGISTRATION

https://doi.org/10.17605/OSF.IO/R5EAS (Open Science Framework).

Clinical guidelines for children and adolescents experiencing gender dysphoria or incongruence: a systematic review of guideline quality (part 1)

BACKGROUND

Increasing numbers of children and adolescents experiencing gender dysphoria/incongruence are being referred to specialist gender services. There are various guidelines outlining approaches to the clinical care of these children and adolescents.

AIM

To examine the quality and development of published guidelines or clinical guidance containing recommendations for managing gender dysphoria/incongruence in children and/or adolescents (age 0-18). A separate paper reports the synthesis of guideline recommendations.

METHODS

A systematic review and narrative synthesis. Databases (Medline, Embase, CINAHL, PsycINFO, Web of Science) were searched to April 2022 and web-based searches and contact with international experts continued to December 2022, with results assessed independently by two reviewers. The Appraisal of Guidelines for Research and Evaluation tool was used to examine guideline quality.

RESULTS

Twenty-three guidelines/clinical guidance publications (1998-2022) were identified (4 international, 3 regional and 16 national). The quality and methods reporting in these varied considerably. Few guidelines systematically reviewed empirical evidence, and links between evidence and recommendations were often unclear. Although most consulted with relevant stakeholders, including 10 which involved service users or user representatives, it was often unclear how this influenced recommendations and only two reported including children/adolescents and/or parents. Guidelines also lacked clarity about implementation. Two international guidelines (World Professional Association for Transgender Health and Endocrine Society) formed the basis for most other guidance, influencing their development and recommendations.

CONCLUSIONS

Most clinical guidance for managing children/adolescents experiencing gender dysphoria/incongruence lacks an independent and evidence-based approach and information about how recommendations were developed. This should be considered when using these to inform service development and clinical practice.

PROSPERO REGISTRATION NUMBER

CRD42021289659.

Surf therapy for people with mental health disorders: a systematic review of randomized and non-randomized controlled trials

BACKGROUND

Surf therapy is gaining popularity for improving mental health. However, there is limited research evidence to substantiate these claims. Therefore, this systematic review aimed to assess randomized or non-randomized studies analyzing the efficacy of surf therapy in improving symptoms of mental illness compared to non-exercising controls and/or alternative intervention, and to identify evidential gaps to inform future research.

METHODS

PRISMA 2020 reporting guidelines were followed. Eligibility criteria included participants of any age and sex, explicitly diagnosed with any mental health disorder, while exposed to surf therapy and compared to non-exercising controls and/or alternative interventions. The primary outcome consisted of changes in symptoms of mental illness scored from baseline to post-intervention. Any randomized or non-randomized trial design was considered. We searched Cochrane Library, CINAHL, EMBASE, PubMed, Scopus, SPORTDiscus and Web of Science databases (December 7, 2023), without language or publication date restrictions and without filters. Risk of bias was assessed using RoB 2. A meta-analysis could not be conducted due to heterogeneity of the studies; therefore, a narrative synthesis of individual study results was performed.

RESULTS

Of 5,666 records, three randomized controlled studies were included in the review. Overall, the findings of the three studies suggest no robust consistent evidence of improvement in mental health symptoms when comparing surf therapy to wait-list control groups or other nature-based exercise interventions (hike therapy). Certainty of evidence should be considered very low, as it is based on three randomized controlled trials.

CONCLUSION

Although we believe that surf therapy provides an interesting approach, robust evidence is scarce. Routes for future well-designed, controlled studies are discussed.

OTHER

No funding to report.

REGISTRATION

PROSPERO CRD42021277060.

Analysis of risk of bias assessments in a sample of intervention systematic reviews, part I: many aspects of conduct and reporting need improvement

OBJECTIVES

Current standards for systematic reviews (SRs) require adequate conduct and complete reporting of risk of bias (RoB) assessments of the individual studies included in the review. We investigated the conduct and reporting of RoB assessments reported in a sample of SRs of interventions for persons with cerebral palsy (CP).

STUDY DESIGN AND SETTING

We included SRs published from 2014 to 2021. Authors worked in pairs to independently extract data on the characteristics of the SRs and to rate their conduct and reporting. The conduct of RoB assessment was appraised with the three AMSTAR-2 items related to RoB assessment. Reporting completeness was evaluated using the two items related to RoB assessment within studies in the PRISMA 2020 guidelines. We use descriptive statistics to report the consensus data, in accordance with our protocol.

RESULTS

We included 145 SRs. Among the 128 (88.3%) SRs that assessed RoB, the standards for AMSTAR-2 item 9 (use of an adequate RoB tool) were partially or fully satisfied in 73 (57.0%). Across the 128 SRs that assessed RoB, 46 (35.9%) accounted for RoB in interpreting the SR's findings and, of the 49 that included a meta-analysis, 11 (22.4%) discussed the impact of RoB on this. 123 (96.1%) of the 128 SRs named the RoB tool that was used for at least one of the study designs they included, 96 (75.0%) specified the RoB items assessed and 89 (69.5%) reported the findings for each item, 81 (63.2%) fully reported the processes for RoB assessment, 68 (53.1%) reported how an overall RoB judgment was reached, and 74 (57.8%) reported an overall RoB assessment for every study.

CONCLUSION

The selection and application of RoB tools in this sample of SRs about interventions for CP are comparable to those reported in other recent studies. However, most SRs in this sample did not fully meet the appraisal standards of AMSTAR-2 regarding the adequacy of the RoB tool applied and other aspects of RoB assessment conduct; Cochrane SRs were a notable exception. Overall, reporting of RoB assessments was somewhat better than conduct, perhaps reflecting the more widespread uptake of the PRISMA guidelines. Our findings may be generalizable to some extent, considering the extensive literature reporting widespread inadequacies in health care-related intervention SRs and reports from other specialties that document similar RoB assessment deficiencies. As such, this study should remind authors, peer reviewers, and journal editors to follow the RoB assessment reporting guidelines of PRISMA 2020 and to understand the corresponding critical appraisal standards of AMSTAR-2. We recommend a shift of focus from the documentation of inadequate RoB assessments and well-known deficiencies in other components of SRs towards the implementation of changes to address these problems along with plans to evaluate their effectiveness.

Analysis of risk of bias assessments in a sample of intervention systematic reviews, Part II: focus on risk of bias tools reveals few meet current appraisal standards

OBJECTIVES

Risk of bias (RoB) assessment is a critical part of any systematic review (SR). There are multiple tools available for assessing RoB of the studies included in a SR. The conduct of these assessments in intervention SRs are addressed by three items in AMSTAR-2, considered the preferred tool for critically appraising an intervention SR. This study focuses attention on item 9, which assesses the ability of a RoB tool to adequately address sources of bias, particularly in randomized trials (RCTs) and nonrandomized studies of interventions (NRSI). Our main objective is to report the detailed results of our examination of both Cochrane and non-Cochrane RoB tools and distinguish those that meet AMSTAR-2 item 9 appraisal standards.

STUDY DESIGN AND SETTING

We identified critical appraisal tools reported in a sample of 126 SRs reporting on interventions for persons with cerebral palsy published from 2014 to 2021. Eligible tools were those that had been used to assess the primary studies included in these SRs and for which assessment results were reported in enough detail to allow appraisal of the tool. We identified the version of the tool applied as original, modified, or novel and established the applicable study designs as intended by the tools' developers. We then evaluated the potential ability of these tools to assess the four sources of bias specified in AMSTAR-2 item 9 for RCTs and NRSI. We adapted item 9 to appraise tools applied to single-case experimental designs, which we also encountered in this sample of SRs.

RESULTS

Most of the eligible tools are recognized by name in the published literature and were applied in the original or modified form. Modifications were applied with considerable variability across the sample. Of the 37 tools we examined, those judged to fully meet the appraisal standards for RCTs included all the Cochrane tools, the original and modified Downs and Black Checklist, and the quality assessment standard for a cross-over study by Ding et al; for NRSI, these included all the Cochrane tools, the original and modified Downs and Black Checklist, and the Research Triangle Institute item bank on Risk of Bias and Precision of Observational Studies for NRSI. In general, tools developed for a specific study design were judged to meet the appraisal standards fully or partially for that design. These results suggest it is unlikely that a single tool will be adequate by AMSTAR-2 item 9 appraisal standards for an intervention SR that includes studies of various designs.

CONCLUSION

To our knowledge, this is the first resource providing SR authors with practical information about the appropriateness and adequacy of RoB tools by the appraisal standards specified in AMSTAR-2 item 9 for RCTs and NRSI. We propose similar methods for appraisal of tools applied to single-case experimental design. We encourage authors to seek contemporary RoB tools developed for use in healthcare-related intervention SRs and designed to evaluate relevant study design features. The tools should address attributes unique to the review topic and research question but not be subjected to unjustified and excessive modifications. We promote recognition of the potential shortcomings of both Cochrane and non-Cochrane RoB tools, even those that perform well by AMSTAR-2 item 9 appraisal standards.

Stakeholders' perspectives on patient involvement in systematic reviews - Results of a World Café in Germany

INTRODUCTION

Patient involvement (PI) in systematic reviews (SRs) can help to improve the quality of SRs and enhance the credibility of the research process. At the same time, PI in SRs poses challenges such as the need for extra time. While several organizations and working groups from English-speaking countries provide recommendations for PI in SRs, there is a lack of current insights from stakeholders in Germany, including researchers and patients. Eliciting their perspectives is indicated, as PI in SRs in Germany might differ due to language barriers and organizational dissimilarities. For sharing and discussing stakeholders' experiences in Germany, a workshop was facilitated. This paper summarizes the results of the workshop to elucidate stakeholders' perspectives on key aspects of PI in SRs in Germany.

METHODS

A World Café was conducted at the 2023 conference of the Network for Evidence-based Medicine. Participants at all levels of experience could take part without prior registration. The data obtained was summarized narratively in an iterative process, and a framework of the topics discussed was developed.

RESULTS

22 participants, predominantly researchers, took part. Participants formulated several general conditions for PI in SRs such as time and transparency. The majority of the tasks described referred to the application phase and the initial phase of a SR. The development of training and information materials in plain German language was deemed essential. The application phase of an externally funded SR and patient recruitment were considered as particularly challenging.

DISCUSSION

Several of the formulated aspects such as time and transparency are consistent with earlier work. The project start of a SR, however, has so far not been explicitly described in the literature as being of particular importance. This phase might be even more crucial to SR projects in Germany since researchers are expected to develop information materials for patients. Both the application phase and patient recruitment could be considered particularly challenging due to a lower degree of organisation of PI in Germany.

CONCLUSION

World Café participants described many aspects referring to the project start of a SR. This underlines that PI in SRs needs to be described as a process. A process model intertwining the phases of a SR with the respective phases of PI, ideally including best practices for each phase, could be of great value. With respect to the specific context in Germany, a greater degree of organization of PI, i.e. coordinated by an institution, could help to manage challenges such as patient recruitment.

A systematic review for the evidence of recommendations and guidelines in hybrid nuclear cardiovascular imaging

OBJECTIVES

This study aimed to evaluate the level of evidence of expert recommendations and guidelines for clinical indications and procedurals in hybrid nuclear cardiovascular imaging.

METHODS

From inception to August 2023, a PubMed literature analysis of the latest version of guidelines for clinical hybrid cardiovascular imaging techniques including SPECT(/CT), PET(/CT), and PET(/MRI) was performed in two categories: (1) for clinical indications for all-in primary diagnosis; subgroup in prognosis and therapy evaluation; and for (2) imaging procedurals. We surveyed to what degree these followed a standard methodology to collect the data and provide levels of evidence, and for which topic systematic review evidence was executed.

RESULTS

A total of 76 guidelines, published between 2013 and 2023, were included. The evidence of guidelines was based on systematic reviews in 7.9% of cases, non-systematic reviews in 47.4% of cases, a mix of systematic and non-systematic reviews in 19.7%, and 25% of guidelines did not report any evidence. Search strategy was reported in 36.8% of cases. Strengths of recommendation were clearly reported in 25% of guidelines. The notion of external review was explicitly reported in 23.7% of cases. Finally, the support of a methodologist was reported in 11.8% of the included guidelines.

CONCLUSION

The use of evidence procedures for developing for evidence-based cardiovascular hybrid imaging recommendations and guidelines is currently suboptimal, highlighting the need for more standardized methodological procedures.

Clinical Practice Guideline Development in Pediatric Emergency Medicine Departments Across the United States: A Cross-sectional Study

OBJECTIVE

This cross-sectional study aimed to assess the process of clinical practice guideline (CPG) development in pediatric emergency medicine departments (PEDs) across the United States, with a focus on identifying areas for improvement to enhance the quality of CPGs.

METHODS

An electronic survey was distributed to PEDs with pediatric emergency medicine fellowship programs. Respondents were asked about their CPG development processes (ie, guideline committee composition, consideration of conflicts of interest, grading recommendations, guideline training opportunities) based on the Appraisal of Guidelines, Research, and Evaluation (AGREE II) tool as well as implementation and monitoring. Univariate analysis was used to assess associations between the number of AGREE II elements adhered to and annual patient volume.

RESULTS

Of the 84 PEDs surveyed, 44 (52.4%) responded, representing institutions from 28 states. Most PEDs (97.7%, n = 43) reported developing their own guidelines, citing the need to improve care quality and standardize patient care as primary reasons. Although most guideline committees included multidisciplinary team members (74.4%, n = 32), routine patient involvement in committees was rare (11.6%, n = 5), and barriers such as time and resource constraints hindered their inclusion. Formal training for committee members in guideline development was uncommon (18.6%, n = 8). There was an association between higher annual patient volume and the number of AGREE II elements adhered to (P = 0.03). Few PEDs considered potential conflicts of interest among committee members (13.6%, n = 6). Over half of surveyed PEDs (54.5%, n = 24) lacked a systematic approach to grading recommendations. Educational materials and workshops were the most common guideline implementation strategies.

CONCLUSIONS

Our findings highlight need for improvement in the CPG development process in PEDs. Including patients in committees, providing formal training for committee members, and adopting a rigorous approach to grading recommendations are crucial steps toward enhancing guideline quality. Emphasizing these improvements has the potential to improve the quality of CPGs for pediatric emergency care.

Consensus methods in patellofemoral pain: how rigorous are they? A scoping review

OBJECTIVE

Clinicians treating patients with patellofemoral pain (PFP) rely on consensus statements to make the best practice recommendations in the absence of definitive evidence on how to manage PFP. However, the methods used to generate and assess agreement for these recommendations have not been examined. Our objective was to map the methods used to generate consensus-based recommendations for PFP and apply four novel questions to assess the rigour of consensus development.

DESIGN

Scoping review.

DATA SOURCES

We searched Medline, SPORTDiscus, CINAHL and Embase from inception to May 2022 to identify consensus-derived statements or practice guidelines on PFP. The Joanna Briggs Institute Manual for Evidence Synthesis was followed to map the existing evidence. We measured the consensus methods based on four sets of questions addressing the panel composition, application of the consensus method chosen, agreement process and the use of evidence mapping.

ELIGIBILITY CRITERIA

All consensus statements or clinical guidelines on PFP were considered.

RESULTS

Twenty-two PFP consensus statements were identified. Panel composition: 3 of the 22 (14%) consensus groups reported the panellists' experience, 2 (9%) defined a desired level of expertise, 10 (45%) reported panellist sex and only 2 (9%) included a patient. Consensus method: 7 of 22 (32%) reported using an established method of consensus measurement/development. Agreement process: 10 of 22 (45%) reported their consensus threshold and 2 (9%) acknowledged dissenting opinions among the panel. Evidence mapping: 6 of 22 (27%) reported using systematic methods to identify relevant evidence gaps.

CONCLUSIONS

PFP consensus panels have lacked diversity and excluded key partners including patients. Consensus statements on PFP frequently fail to use recognised consensus methods, rarely describe how 'agreement' was defined or measured and often neglect to use systematic methods to identify evidence gaps.

A critical appraisal of acute sore throat guidelines using the AGREE II instrument: a scoping review

BACKGROUND

Conflicting international guidelines exist on the management of sore throat by antibiotics.

OBJECTIVES

To assess with the Appraisal of Guidelines for Research and Evaluation II (AGREE) instrument the quality of guidelines for uncomplicated acute group A beta-haemolytic streptococcal (GABHS) sore-throat. To make a sensitivity analysis restricted to guidelines with a rigour of development score higher than 60% and to describe their recommendations on scores, tests, and antibiotic therapy, including their justification.

METHODS

A guideline literature review of acute GABHS sore throat, published between January 2000 and December 2019 in primary care and secondary care. The PubMed database, the Canadian Medical Association Infobase on Clinical Practice Guidelines and the International Network Guidelines were used. The quality of guidelines was assessed using the AGREE II instrument. The guidelines were classified into 2 categories: high-quality guidelines had to rate >60% for the rigour of development score, the others were classified as low-quality guidelines.

RESULTS

Significant heterogeneity between the 15 guidelines concerned the scores of the 6 assessment domains. Among them, 6 guidelines presented a score above 60% with regards to the rigor of development domain and used a systematic literature search method, citing meta-analyses of recent randomised clinical trials. Most of the 6 high-quality guidelines no longer recommended the systematic use of diagnostic scores and tests, nor antibiotic therapy to prevent acute rheumatic fever or loco-regional complications, except for high-risk patients.

CONCLUSION

Major discrepancies emphasise the need for only high-quality guidelines, based on adequately assessed evidence. Restricted antibiotic prescriptions to severe cases or high-risk patients would avoid antibiotic resistance.

Appraisal of quality and analysis of the similarities and differences between osteoarthritis Clinical Practice Guideline recommendations: A systematic review

OBJECTIVE

Clinical Practice Guidelines (CPGs) aim to support management of hip and knee osteoarthritis (OA), but recommendations are often conflicting and implementation is poor, contributing to evidence-to-practice gaps. This systematic review investigated the contextual and methodological factors contributing to conflicting recommendations for hip and knee OA.

METHOD

Our systematic review appraised CPGs for managing hip and knee OA in adults ≥18 years (PROSPERO CRD42021276635). We used AGREE-II and AGREE-REX to assess quality and extracted data on treatment gaps, conflicts, biases, and consensus. Heterogeneity of recommendations was determined using Weighted Fleiss Kappa (K). The relationship between (K) and AGREE-II/AGREE-REX scores was explored.

RESULTS

We identified 25 CPGs across eight countries and four international organisations. The ACR, EULAR, NICE, OARSI and RACGP guidelines scored highest for overall AGREE-II quality (83%). The highest overall AGREE-REX scores were for BMJ Arthroscopy (80%), RACGP (78%) and NICE (76%). CPGs with the least agreement for pharmacological recommendations were ESCEO and NICE (-0.14), ACR (-0.08), and RACGP (-0.01). The highest agreements were between RACGP and NICE (0.53), RACGP and ACR (0.61), and NICE and ACR (0.91). Decreased internal validity determined by low-quality AGREE scores(<60%) in editorial independence were associated with less agreement for pharmacological recommendations.

CONCLUSION

There were associations between guideline quality and agreement scores. Future guideline development should be informed by robust evidence, editorial independence and methodological rigour to ensure a harmonisation of recommendations. End-users of CPGs must recognise the contextual factors associated with the development of OA CPGs and balance these factors with available evidence.

Quality analysis of the clinical practice guidelines for management of impacted maxillary central incisors: a systematic review

PURPOSE

To assess the quality of clinical practice guidelines (CPG) for management of impacted central incisors.

METHODS

Search was performed in PubMed, LILACS, Web of Science, EMBASE, Scopus, and Cochrane databases, and guideline-focused databases/repositories on 15-09-2022 without any limitations and was updated on 15-07-2023. Grey literature search was also performed. Two independent reviewers were involved in the study selection and data extraction. Quality assessment of the included CPG was performed by four independent appraisers using the AGREE-II instrument. The degree of agreement among the appraisers was calculated using the intraclass correlation coefficient (ICC).

RESULTS

Five CPG were included in the review. The Ministry of Health, Malaysia (MHM) guideline obtained the highest scores in all six domains of AGREE-II and an overall score of 73% demonstrating the "highest" quality. The remaining four guidelines obtained overall "low-quality" scores ranging from 34.57-37.52%. The ICC scores ranged from 0.530 to 0.990 for various domains of AGREE-II.

CONCLUSION

MHM guidelines demonstrated high-quality scores in domains of 'scope and purpose', 'clarity of presentation', 'applicability domain', and 'editorial independence', while others were found to have moderate or low quality. This review identified areas that can be addressed by future guideline developers to avoid these discrepancies.

Systematic review of clinical practice guidelines and systematic reviews: A method for conducting comprehensive analysis

A systematic review (SR) is a research method for synthesizing evidence on a specific topic. Among the various types of systematic reviews, there are SRs of guidelines (CPGs) and SRs of SRs. Traditionally, they are limited to just one type of secondary evidence. This paper introduces an innovative SR methodology that combines CPGs and SRs to improve evidence synthesis and overcome the limitations of isolated use. Essential steps that should always precede the actual research process include registering the research protocol, formulating research questions and setting inclusion/exclusion criteria. Using the PRISMA protocol for comprehensive database searches, it's crucial to combine keywords with boolean operators and remove duplicates. The eligibility of studies should be assessed by selecting potentially relevant articles through an initial screening of titles and abstracts, followed by a meticulous analysis of the full-texts. Rigorous evidence evaluation tools, such as AGREE II for CPGs and AMSTAR 2 for SRs, and the double reviewer approach ensure high-quality selections. Additionally, converting summarized results into percentages and applying statistical analyses facilitate interpretation and improve the reliability of rater assessments. A further characteristic of this methodology is its adaptability to the evolution of healthcare research.

Development of a Novel Evidence-Based Practice-Specific Competency for Doctor of Physical Therapy Students in Clinical Education: A Modified Delphi Approach

INTRODUCTION

Evidence-based practice (EBP) results in high-quality care and decreases unwarranted variation in practice.

REVIEW OF THE LITERATURE

Few performance criteria related to EBP are included in physical therapy clinical education (CE) performance measures, despite EBP requirements in Doctor of Physical Therapy education. The purpose of this study was to develop EBP-specific competencies that may be used for Doctor of Physical Therapy students for use throughout CE.

SUBJECTS

Thirteen subject matter experts (SME) participated in this study.

METHODS

Subject matter experts were asked to rank each core EBP competency, from a previously described framework, using a 3-point Likert scale, which included "Not Essential," "Essential," and "Not Sure." A consensus of 70% or greater for the "Essential" rating advanced the competency to the final Delphi round, whereas a consensus of 70% or greater for the "Not Essential" rating was required for competency elimination. Subject matter experts voted to either "Accept" or "Modify" the competencies that had reached the inclusion consensus threshold. All competencies that reached consensus for inclusion after all 3 rounds were included in the final EBP Domain of Competence.

RESULTS

Consensus was achieved in round one for 38% (n = 26) of items. In round 2, a consensus was achieved for 20% (n = 8) of items. Of the items remaining after rounds 1 and 2, 6 overarching competencies were identified, and all remaining items served as descriptions and specifications in the final EBP Domain of Competence.

DISCUSSION AND CONCLUSIONS

The 6 competencies developed from this study constitute the EBP Domain of Competence and may be used throughout CE to assess students' EBP competency in clinical practice.

The quality of COVID-19 systematic reviews during the coronavirus 2019 pandemic: an exploratory comparison

BACKGROUND

The unprecedented volume and speed at which COVID-19-related systematic reviews (SRs) may have been produced has raised questions regarding the quality of this evidence. It is feasible that pandemic-related factors may have led to an impairment in quality (reduced internal validity, increased risk of bias [RoB]). This may have serious implications for decision-making related to public health and individual healthcare.

OBJECTIVE

The primary objective was to compare the quality of SRs published during the pandemic that were related to COVID-19 with SRs published during the pandemic that were unrelated to COVID-19 (all of which were fully appraised in the KSR Evidence database of SRs in healthcare). Our secondary objective was to compare the quality of SRs published during the pandemic (regardless of research topic), with SRs published pre-pandemic.

METHODS

We compared all SRs related to COVID-19 to all SRs unrelated to COVID-19 that (i) were published during the pandemic (between 1st March 2020 and September 14, 2022), (ii) were included in KSR Evidence, and (iii) had been appraised using the ROBIS tool. We then compared all SRs published during the pandemic (regardless of research topic) with a pre-pandemic sample of SRs.

RESULTS

For SRs published during the pandemic, we found there was no statistically significant difference in quality between those SRs tagged as being related to COVID-19 and those that were not [relative risk (RR) of low RoB for COVID-19 versus COVID-19-unrelated reviews: 0.94; 95% confidence interval (CI): 0.66 to 1.34]. Generally, COVID-19 SRs and COVID-19-unrelated SRs were both of low quality with only 10% of COVID-19 reviews and 11% of COVID-19-unrelated reviews rated as low RoB. However, SRs (regardless of topic) published during the pandemic were of lower quality than those published pre-pandemic (RR for low RoB for 'during pandemic' versus 'pre-pandemic': 0.30; 95% CI: 0.26 to 0.34) with 11% of pandemic and 36% of pre-pandemic SRs rated as low RoB.

CONCLUSION

These results suggest COVID-19 and COVID-19-unrelated SRs published during the pandemic are equally of low quality. SRs published during the pandemic were generally lower quality compared with SRs published pre-pandemic irrespective of COVID-19 focus. Moreover, SR quality in general is seriously lacking, and considerable efforts need to be made to substantially improve the quality and rigour of the SR process.

Same evidence different recommendations: a methodological assessment of transatlantic guidelines for the management of valvular heart disease

OBJECTIVES

The aim of this study was to identify methodological variations leading to varied recommendations between the American College of Cardiology (ACC)/American Heart Association (AHA) and the European Society of Cardiology (ESC)/European Association for Cardio-Thoracic Surgery (EACTS) valvular heart disease guidelines and to suggest foundational steps towards standardizing guideline development.

METHODS

An in-depth analysis was conducted to evaluate the methodologies used in developing the transatlantic guidelines for managing valvular heart disease. The evaluation was benchmarked against the standards proposed by the Institute of Medicine.

RESULTS

Substantial discrepancies were noted in the methodologies utilized in development processes, including Writing Committee composition, evidence evaluation, conflict of interest management and voting processes. Furthermore, despite their mutual differences, both methodologies demonstrate notable deviations from the Institute of Medicine standards in several essential areas, including literature review and evidence grading. These dual variances likely influenced divergent treatment recommendations. For example, the ESC/EACTS recommends transcatheter edge-to-edge repair for patients with chronic severe mitral regurgitation ineligible for mitral valve surgery, while the ACC/AHA recommends transcatheter edge-to-edge repair based on anatomy, regardless of surgical risk. ESC/EACTS guidelines recommend a mechanical aortic prosthesis for patients under 60, while ACC/AHA guidelines recommend it for patients under 50. Notably, the ACC/AHA and ESC/EACTS guidelines have differing age cut-offs for surgical over transcatheter aortic valve replacement (<65 and <75 years, respectively).

CONCLUSIONS

Variations in methodologies for developing clinical practice guidelines have resulted in different treatment recommendations that may significantly impact global practice patterns. Standardization of essential processes is vital to increase the uniformity and credibility of clinical practice guidelines, ultimately improving healthcare quality, reducing variability and enhancing trust in modern medicine.

Adequacy of clinical guideline recommendations for patients with low-risk cancer managed with monitoring: systematic review

OBJECTIVES

The aim of this systematic review was to summarize national and international guidelines that made recommendations for monitoring patients diagnosed with low-risk cancer. It appraised the quality of guidelines and determined whether the guidelines adequately identified patients for monitoring, specified which tests to use, defined monitoring intervals, and stated triggers for further intervention. It then assessed the evidence to support each recommendation.

STUDY DESIGN AND SETTING

Following the Preferred Reporting Items for Systematic Reviews and Meta-analyses, we searched PubMed and Turning Research into Practice databases for national and international guidelines' that were written in English and developed or updated between 2012 and 2023. Quality of individual guidelines was assessed using the AGREE II tool.

RESULTS

Across the 41 published guidelines, 48 different recommendations were identified: 15 (31%) for prostate cancer, 11 (23%) for renal cancer, 6 (12.5%) for thyroid cancer, and 10 (21%) for blood cancer. The remaining 6 (12.5%) were for brain, gastrointestinal, oral cavity, bone and pheochromocytoma and paraganglioma cancer. When combining all guidelines, 48 (100%) stated which patients qualify for monitoring, 31 (65%) specified which tests to use, 25 (52%) provided recommendations for surveillance intervals, and 23 (48%) outlined triggers to initiate intervention. Across all cancer sites, there was a strong positive trend with higher levels of evidence being associated with an increased likelihood of a recommendation being specific (P = 0.001) and the evidence for intervals was based on expert opinion or other guidance.

CONCLUSION

With the exception of prostate cancer, the evidence base for monitoring low-risk cancer is weak and consequently recommendations in clinical guidelines are inconsistent. There is a lack of direct evidence to support monitoring recommendations in the literature making guideline developers reliant on expert opinion, alternative guidelines, or indirect or nonspecific evidence.

Obesity intervention evidence synthesis: Where are the gaps and which should we address first?

Health professionals and policymakers rely on evidence synthesized from high quality research studies. Yet, there remain unanswered questions about how to prevent and treat obesity. In this research project, international practice guidelines and Cochrane systematic reviews were examined in order to identify gaps in the synthesized obesity intervention evidence base. One hundred and forty-two partial or complete gaps were found. Systematic review questions to address these gaps were formulated and subjected to a prioritization consultation process with 36 international obesity expert stakeholders. Forty-three review questions were priority-assessed. The top 10 ranked review questions received support from at least 75.0% of stakeholders. The leading questions focused on preventive and community-based approaches, including those delivered through primary-care. Children within the context of their families were a highly-prioritized target group, as were persons with diabetes or disabilities. Experts also prioritized reviews to determine which elements of programs are the most effective, and by which mode they are best delivered. Experts recommended that negative, psycho-social, and longer-term outcomes be captured in reviews. We request reviewers and funders to strongly consider addressing the top 10 leading prioritized review questions presented here.

Newborn and child health national and provincial clinical practice guidelines in South Africa, Nigeria and Malawi: a scoping review

BACKGROUND

Low and middle-income countries remain disproportionately affected by high rates of child mortality. Clinical practice guidelines are essential clinical tools supporting implementation of effective, safe, and cost-effective healthcare. High-quality evidence-based guidelines play a key role in improving clinical management to impact child mortality. We aimed to identify and assess the quality of guidelines for newborn and child health published in South Africa, Nigeria and Malawi in the last 5 years (2017-2022).

METHODS

We searched relevant websites (June-July 2022), for publicly available national and subnational de novo or adapted guidelines, addressing newborn and child health in the three countries. Pairs of reviewers independently extracted information from eligible guidelines (scope, topic, target population and users, responsible developers, stakeholder consultation process, adaptation description, assessment of evidence certainty). We appraised guideline quality using the Appraisal of Guidelines for Research & Evaluation (AGREE II) instrument.

RESULTS

We identified 40-guidelines from the three countries. Of these, 8/40 reported being adopted from a parent guideline. More guidelines (n = 19) provided guidance on communicable diseases than on non-communicable diseases (n = 8). Guidelines were most often developed by national health ministries (n = 30) and professional societies (n = 14). Eighteen guidelines reported on stakeholder consultation; with Nigeria (10/11) and Malawi (3/6) faring better than South Africa (5/23) in reporting this activity. The Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach was used in 1/7 guidelines that reported assessing certainty of evidence. Overall guidelines scored well on two AGREE II domains: scope and purpose median (IQR) score 68% (IQR 47-83), and clarity of presentation 81% (67-94). Domains critical for ensuring credible guidance scored below 20%: rigour of development 11% (4-32) and editorial independence 6% (0-27).

CONCLUSION

National ministries and professional societies drive guideline activities in Malawi, Nigeria and South Arica. However, the methods and reporting do not adhere to global standards. We found low AGREE II scores for rigour of guideline development and editorial independence and limited use of GRADE or adaptation methods. This undermines the credibility of available guidelines to support evidence-informed care. Our findings highlight the importance of ongoing efforts to strengthen partnerships, capacity, and support for guideline development.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

The Reporting and Methodological Quality of Systematic Reviews Underpinning Clinical Practice Guidelines Focused on the Management of Cutaneous Melanoma: Cross-Sectional Analysis

BACKGROUND

Clinical practice guidelines (CPGs) inform evidence-based decision-making in the clinical setting; however, systematic reviews (SRs) that inform these CPGs may vary in terms of reporting and methodological quality, which affects confidence in summary effect estimates.

OBJECTIVE

Our objective was to appraise the methodological and reporting quality of the SRs used in CPGs for cutaneous melanoma and evaluate differences in these outcomes between Cochrane and non-Cochrane reviews.

METHODS

We conducted a cross-sectional analysis by searching PubMed for cutaneous melanoma guidelines published between January 1, 2015, and May 21, 2021. Next, we extracted SRs composing these guidelines and appraised their reporting and methodological rigor using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and AMSTAR (A Measurement Tool to Assess Systematic Reviews) checklists. Lastly, we compared these outcomes between Cochrane and non-Cochrane SRs. All screening and data extraction occurred in a masked, duplicate fashion.

RESULTS

Of the SRs appraised, the mean completion rate was 66.5% (SD 12.29%) for the PRISMA checklist and 44.5% (SD 21.05%) for AMSTAR. The majority of SRs (19/50, 53%) were of critically low methodological quality, with no SRs being appraised as high quality. There was a statistically significant association (P<.001) between AMSTAR and PRISMA checklists. Cochrane SRs had higher PRISMA mean completion rates and higher methodological quality than non-Cochrane SRs.

CONCLUSIONS

SRs supporting CPGs focused on the management of cutaneous melanoma vary in reporting and methodological quality, with the majority of SRs being of low quality. Increasing adherence to PRISMA and AMSTAR checklists will likely increase the quality of SRs, thereby increasing the level of evidence supporting cutaneous melanoma CPGs.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Systematic Review and Critical Appraisal of Urticaria Clinical Practice Guidelines: A Global Guidelines in Dermatology Mapping Project (GUIDEMAP)

BACKGROUND

Management of urticaria can be optimized with clinical practice guidelines (CPGs). However, the quality of recent urticaria CPGs remains unclear.

OBJECTIVE

To identify and appraise urticaria CPGs worldwide published in the last 5 years.

METHODS

A search for relevant urticaria CPGs was conducted between January 1, 2017, and May 31, 2022, using the following databases: MEDLINE, Embase, National Institute for Health and Care Excellence (NICE) Evidence Search, Guidelines International Network, ECRI Guidelines Trust, Australian Clinical Practice Guidelines, Trip Medical Database, and DynaMed. The included CPGs were critically appraised using the Appraisal of Guidelines for Research and Evaluation (AGREE) II instrument, Lenzer et al's red flags, and the Institute of Medicine (IOM) criteria of trustworthiness.

RESULTS

We included 21 urticaria CPGs. Most guidelines reviewed treatment recommendations of chronic spontaneous urticaria. The majority of guidelines were from European and Asian countries with high and high-middle sociodemographic index, written in English, and openly accessible. Seventeen guidelines (81%) had at least 1 AGREE II domain rated poor quality. Applicability, rigor of development, and stakeholder involvement were the 3 AGREE II domains that scored the lowest across guidelines. Appraisal with Lenzer et al's red flags showed that 18 guidelines (86%) raised at least 1 red flag indicating potential bias. The top 3 domains raising red flags were: no inclusion of nonphysician experts/patient representative/community stakeholders, no or limited involvement of a methodologist in the evaluation of evidence, and lack of external review. Based on IOM's criteria of trustworthiness, 20 guidelines (95%) had 1 or more criteria that did not meet best practice standards. The 3 domains with the highest number of best practice standards not met were updating procedures, rating strength of recommendations, and external review. Guidelines scored highest for the AGREE II domains of defining scope and purpose and clarity of presentation, and had the most fully met IOM's best practice standard for articulation of recommendations. However, only 1 urticaria CPG by NICE was identified as rigorously developed across all 3 appraisal tools.

CONCLUSIONS

The quality of urticaria CPGs in the last 5 years varied widely. Only the NICE urticaria guideline consistently demonstrated excellent quality, high trustworthiness, and low risk of bias. Use of a rigorous framework to rate certainty of evidence and grade strength of recommendation, involvement of methodologists, stakeholder engagement with external review, and clear guidance for updating can help improve the quality of future CPGs.

Analysis and comparison of clinical practice guidelines regarding treatment recommendations for chronic tinnitus in adults: a systematic review

OBJECTIVES

To determine if, and to what extent, published clinical practice guidelines for the treatment of chronic tinnitus vary in their recommendations.

DESIGN

Systematic review of guidelines.

DATA SOURCES

PubMed, EMBASE and GIN electronic databases were searched in March 2022 and the search was updated in June 2023.

ELIGIBILITY CRITERIA

We included clinical practice guidelines that gave recommendations on the treatment of tinnitus. No language restrictions were applied.

DATA EXTRACTION AND SYNTHESIS

Two independent reviewers extracted the data and used the AGREE checklist to report on reporting.

RESULTS

A total of 10 guidelines were identified and included, published between 2011 and 2021. Recommendations for 13 types of tinnitus treatments were compared. Large differences in guideline development and methodology were found. Seven of the 10 guidelines included a systematic search of the literature to identify the available evidence. Six of the 10 guidelines used a framework for the development of the guideline. Reporting was poor in multiple guidelines. Counselling and cognitive behavioural therapy were the only treatments that were recommended for treating tinnitus associated distress by all guidelines that reported on these topics. Tinnitus retraining therapy, sound therapy, hearing aids and cochlear implantation were not unanimously recommended either due to the lack of evidence, a high risk of bias or judgement of no beneficial effect of the specific treatment.

CONCLUSIONS

There were notable differences with respect to whether guidelines considered the available evidence sufficient enough to make a recommendation. Notably, we identified substantial differences in the rigour of guideline design and development. Reporting was poor in many guidelines. Future guidelines could benefit from the use of reporting tools to improve reporting and transparency and the inclusion of guideline experts and patients to improve the quality of clinical practice guidelines on tinnitus.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Clinical practice guidelines on the use of deep brain stimulation for the treatment of obsessive-compulsive disorder: systematic review

BACKGROUND

Deep brain stimulation (DBS) has been proposed to improve symptoms of obsessive-compulsive disorder (OCD) but is not yet an established therapy.

AIMS

To identify relevant guidelines and assess their recommendations for the use of DBS in OCD.

METHOD

Medline, Embase, American Psychiatric Association PsycInfo and Scopus were searched, as were websites of relevant societies and guideline development organisations. The review was based on the PRISMA recommendations, and the search strategy was verified by a medical librarian. The protocol was developed and registered with PROSPERO (CRD42022353715). The guidelines were assessed for quality using the AGREE II instrument.

RESULTS

Nine guidelines were identified. Three guidelines scored >80% on AGREE II. 'Scope and Purpose' and 'Editorial Independence' were the highest scoring domains, but 'Applicability' scores were low. Eight guidelines recommended that DBS is used after all other treatment options have failed to alleviate OCD symptoms. One guideline did not recommend DBS beyond a research setting. Only one guideline performed a cost-effectiveness analysis; the other eight did not provide details on safe or effective DBS protocols.

CONCLUSION

Despite a very limited evidence base, eight of the nine identified guidelines supported the use of DBS for OCD as a last line of therapy; however, multiple aspects of DBS provision were not addressed.

Overviews of reviews in the cardiovascular field underreported critical methodological and transparency characteristics: a methodological study based on the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement

OBJECTIVES

This study aimed to evaluate the epidemiology, reporting characteristics, and adherence to the Preferred Reporting Items for Overviews of Reviews (PRIOR) statement of overviews of reviews (overviews) of interventions in the cardiovascular field.

STUDY DESIGN AND SETTING

MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched from January 1, 2000, to October 15, 2020. An updated search was performed in MEDLINE, Epistemonikos, and Google Scholar up to August 25, 2022. Overviews of interventions published in English and primarily considering populations, interventions, and outcomes pertinent to the cardiovascular field were eligible. Study selection, data extraction, and PRIOR adherence assessment were performed by two authors independently.

RESULTS

We analyzed 96 overviews. Almost half (43/96 [45%]) were published between 2020 and 2022 and contained a median of 15 systematic reviews (SRs) (interquartile range, 9-28). The commonest title terminology was "overview of (systematic) reviews" (38/96 [40%]). Methods for handling SR overlap were reported in 24/96 (25%), methods for assessing primary study overlap in 18/96 (19%), handling of discrepant data in 11/96 (11%), and methods for methodological quality or risk of bias assessment of the primary studies within SRs in 23/96 (24%). Authors included data sharing statements in 28/96 (29%), complete funding disclosure in 43/96 (45%), protocol registration in 43/96 (45%), and conflict of interest statement in 82/96 (85%) overviews.

CONCLUSION

Insufficient reporting was identified in methodological characteristics unique in overviews' conduct and most transparency markers. Adoption of PRIOR from the research community could ameliorate overviews' reporting.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to Best Tools and Practices for Systematic Reviews

» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Differential impact of transfusion guidelines on blood transfusion practices within a health network

Whether clinical practice guidelines have a significant impact on practice is unclear. The effect of guideline recommendations on clinical practice often a lags behind the date of publication. We evaluated by means of a data-driven approach if and when the guidelines on red blood cell transfusions (RBCTs) issued by Swiss Smarter Medicine in 2016 had an impact on RBCTs practice within a hospital network, where awareness of guidelines was promoted mainly among internal medicine specialties. Data on RBCTs performed in a Swiss hospital network from January 2014 to April 2021 were analyzed by hospital site and specialty to assess whether guidelines led to a decrease in inappropriate RBCTs. RBCTs were defined as "inappropriate" if patients had a hemoglobin level ≥ 70 g/L without or ≥ 80 g/L with significant cardiovascular comorbidities. Changes in the rate of inappropriate transfusions were analyzed with an advanced statistical approach that included generalized additive models. Overall prior to March 2017 there were more inappropriate than appropriate RBCTs, but after October 2017 the opposite could be observed. A change-point in the time trend was estimated from transfusion data to occur in the time interval between March and October 2017. This change was mainly driven by practice changes in the medical wards, while no significant change was observed in the critical care, surgical and oncology wards. Change in practice varied by hospital site. In conclusion, our results show that a significant change in the RBCTs practice at the hospital level occurred approximately 18 months after national guidelines were issued.

The utilization of systematic review evidence in formulating India's National Health Programme guidelines between 2007 and 2021

Evidence-informed policymaking integrates the best available evidence on programme outcomes to guide decisions at all stages of the policy process and its importance becomes more pronounced in resource-constrained settings. In this paper, we have reviewed the use of systematic review evidence in framing National Health Programme (NHP) guidelines in India. We searched official websites of the different NHPs, linked to the main website of the Ministry of Health and Family Welfare (MoHFW), in December 2020 and January 2021. NHP guideline documents with systematic review evidence were identified and information on the use of this evidence was extracted. We classified the identified systematic review evidence according to its use in the guideline documents and analysed the data to provide information on the different factors and patterns linked to the use of systematic review evidence in these documents. Systematic reviews were mostly visible in guideline documents addressing maternal and newborn health, communicable diseases and immunization. These systematic reviews were cited in the guidelines to justify the need for action, to justify recommendations for action and opportunities for local adaptation, and to highlight implementation challenges and justify implementation strategies. Guideline documents addressing implementation cited systematic reviews about the problems and policy options more often than citing systematic reviews about implementation. Systematic reviews were linked directly to support statements in few guideline documents, and sometimes the reviews were not appropriately cited. Most of the systematic reviews providing information on the nature and scale of the policy problem included Indian data. It was seen that since 2014, India has been increasingly using systematic review evidence for public health policymaking, particularly for some of its high-priority NHPs. This complements the increasing investment in research synthesis centres and procedures to support evidence-informed decision making, demonstrating the continued evolution of India's evidence policy system.

Recommendations for Improving the Quality of Prehospital Evidence-Based Guidelines

Evidence-based guidelines that provide recommendations for clinical care or operations are increasingly being published to inform the EMS community. The quality of evidence evaluation and methodological rigor undertaken to develop and publish these recommendations vary. This can negatively affect dissemination, education, and implementation efforts. Guideline developers and end users could be better informed by efforts across medical specialties to improve the quality of guidelines, including the use of specific criteria that have been identified within the highest quality guidelines. In this special contribution, we aim to describe the current state of published guidelines available to the EMS community informed by two recent systematic reviews of existing prehospital evidenced based guidelines (EBGs). We further aim to provide a description of key elements of EBGs, methods that can be used to assess their quality, and concrete recommendations for guideline developers to improve the quality of evidence evaluation, guideline development, and reporting. Finally, we outline six key recommendations for improving prehospital EBGs, informed by systematic reviews of prehospital guidelines performed by the Prehospital Guidelines Consortium.

Guidance to best tools and practices for systematic reviews1

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Evidence-basing for quality improvement; bringing clinical practice guidelines closer to their promise of improving care practices

BACKGROUND

Clinical practice guidelines (CPGs) have become central to efforts to change clinical practice and improve the quality of health care. Despite growing attention for rigorous development methodologies, it remains unclear what contribution CPGs make to quality improvement.

AIM

This mixed methods study examines guideline quality in relation to the availability of certain types of evidence and reflects on the implications of CPGs' promise to improve the quality of care practices.

METHODS

The quality of 62 CPGs was assessed with the Appraisal of Guidelines, Research, and Evaluation (AGREE) instrument. Findings were discussed in 19 follow-up interviews to examine how different quality aspects were considered during development.

RESULTS

The AGREE assessment showed that while some quality criteria were met, CPGs have limited coverage of domains such as stakeholder involvement and applicability, which generally lack a 'strong' evidence base (e.g., randomized controlled trials [RCT]). Qualitative findings uncovered barriers that impede the consolidation of evidence-based guideline development and quality improvement including guideline scoping based on the patient-intervention-comparison-outcome (PICO) question format and a lack of clinical experts involved in evidence appraisal. Developers used workarounds to include quality considerations that lack a strong base of RCT evidence, which often ended up in separate documents or appendices.

CONCLUSION

Findings suggest that CPGs mostly fail to integrate different epistemologies needed to inform the quality improvement of clinical practice. To bring CPGs closer to their promise, guideline scoping should maintain a focus on the most pertinent quality issues that point developers toward the most fitting knowledge for the question at hand, stretching beyond the PICO format. To address questions that lack a strong evidence base, developers actually need to appeal to other sources of knowledge, such as quality improvement, expert opinion, and best practices. Further research is needed to develop methods for the robust inclusion of other types of knowledge.

Project rebuild the evidence base (REB): A method to interpret randomised clinical trials and their meta-analysis to present solid benefit-risk assessments to patients

Evidence-based medicine is the cornerstone of shared-decision making in healthcare today. The public deserves clear, transparent and trust-worthy information on drug efficacy. Yet today, many drugs are prescribed and used without solid evidence of efficacy. Clinical trials and randomised clinical trials (RCTs) are the best method to evaluate drug efficacy and side effects. In a shared medical decision-making approach, general practitioners need drug assessment based on patient-important outcomes. The aim of project rebuild the evidence base (REB) is to bridge the gap between the data needed in clinical practice and the data available from clinical research. The drugs will be assessed on clinical patient important outcomes and for a population. Using the Cochrane tools, we propose to analyse for each population and outcome: 1) a meta-analysis based on RCTs with a low risk of bias overall; 2) an evaluation of results of confirmatory RCTs; 3) a statistical analysis of heterrogeneity between RCTs and 4) an analysis of publication bias. Depending on the results of these analyses, the evidence will be categorized in 4 different levels: firm evidence, evidence (to be confirmed), signal or absence of evidence. Project REB proposes a method for reading and interpreting RCTs and their meta-analysis to produce quality data for general practitioners to focus on risk-benefit assessment in the interest of patients. If this data does not exist, it could enable clinical research to better its aim.

Basics of Sustainable Diets and Tools for Assessing Dietary Sustainability: A Primer for Researchers and Policy Actors

Climate change can have economic consequences, affecting the nutritional intake of populations and increasing food insecurity, as it negatively affects diet quality parameters. One way to mitigate these consequences is to change the way we produce and consume our food. A healthy and sustainable diet aims to promote and achieve the physical, mental, and social well-being of the populations at all life stages, while protecting and safeguarding the resources of the planet and preserving biodiversity. Over the past few years, several indexes have been developed to evaluate dietary sustainability, most of them based on the EAT-Lancet reference diet. The present review explains the problems that arise in human nutrition as a result of climate change and presents currently available diet sustainability indexes and their applications and limitations, in an effort to aid researchers and policy actors in identifying aspects that need improvement in the development of relevant indexes. Overall, great heterogeneity exists among the indicators included in the available indexes and their methodology. Furthermore, many indexes do not adequately account for the diets’ environmental impact, whereas others fall short in the economic impact domain, or the ethical aspects of sustainability. The present review reveals that the design of one environmentally friendly diet that is appropriate for all cultures, populations, patients, and geographic locations is a difficult task. For this, the development of sustainable and healthy diet recommendations that are region-specific and culturally specific, and simultaneously encompass all aspects of sustainability, is required.