What does expert opinion in guidelines mean? a meta-epidemiological study

Interventions to support children after a parental acquired brain injury: a scoping review

OBJECTIVE

This scoping review aimed to identify manualised programs and practice suggestions to support children's health literacy, behaviors and emotions after a parental acquired brain injury.

METHODS

A systematic search of five scientific databases (PsychINFO, MEDLINE, ProQuest, Scopus, Cochrane) and gray literature occurred. Inclusion criteria included: studies and gray literature published 1989 to 2023, in English, child populations with relationship to parental acquired brain injury, identifying manualised programs or practice suggestions via content analysis approach.

ETHICAL CONSIDERATIONS

No data were collected from human participants. All included studies, where relevant, demonstrated consent and/or ethical processes.

RESULTS

Sixteen relevant studies and three gray literature resources (n = 19) were identified, including two studies that detailed manualised programs, and fifteen studies and two resource packs that included practice suggestions. Five common domains within practice suggestions were identified: systemic commitment (n = 17); family-centered approaches (n = 16); child-centered practices (n = 15); structured programs (n = 9); and peer support (n = 8).

CONCLUSIONS

More rigorous evaluation is required to test the potential benefits of manualised programs and practice suggestions. A systemic commitment at clinical and organizational levels to provide child and family-centered practices, structured programs, and access to peer support, early and throughout adult-health care settings, may help to meet the support needs of children.

Utilization of expert opinion in infectious diseases clinical guidelines-A meta-epidemiological study

INTRODUCTION

Expert opinion is widely used in clinical guidelines. No research has ever been conducted investigating the use of expert opinion in international infectious disease guidelines. This study aimed to create an analytical map by describing the prevalence and utilization of expert opinion in infectious disease guidelines and analyzing the methodological aspects of these guidelines.

METHODS

In this meta-epidemiological study, systematic searches in PubMed and Trip Medical Database were performed to identify clinical guidelines on infectious diseases, published between January 2018 and May 2023 in English, by international organizations. Data extracted included guideline characteristics, expert opinion utilization, and methodological details. Prevalence and rationale of expert opinion use were analyzed descriptively. Methodological differences between groups were analyzed with Chi-square and Mann-Whitney U Test.

RESULTS

The analysis covered 66 guidelines with 2296 recommendations, published/endorsed by 136 organizations. Most guidelines (79%) used systematic literature searches, 42% provided search strategies, and 38% presented screening flow diagrams and conducted risk of bias assessments. 48.5% of the guidelines allowed expert opinion, most of which included expert opinion as part of the evidence hierarchy within the grading system. Guidelines allowing expert opinion, compared to those which do not, issued more recommendations per guideline (48.82 vs.19.13, p<0.001), and reported fewer screening flow diagrams (25% vs. 65%, p = 0.002), and less risk of bias assessments (19% vs.78%, p<0.001).

CONCLUSIONS

Expert opinion is utilized in half of assessed guidelines, often integrated into the evidence hierarchy within the grading system. Its utilization varies considerably in methodology, form, and terminology between guidelines. These findings highlight a pressing need for additional research and guidance, to improve and advance the standardization of infectious disease guidelines.

Hierarchy of Evidence: An Appraisal Tool for Weighting the Evidence in Healthcare Design Research Based on Internal Validity

PURPOSE

This inquiry aims to develop an appraisal tool to offer greater granularity in weighing evidence in the field of healthcare design research.

BACKGROUND

In evidence-based design (EBD), the goal is to implement interventions that result in a meaningful and optimal effect based on current best evidence. Although multiple appraisal tools (many adopted from medical disciplines) have been instrumental in evaluating studies in the field of healthcare design research to identify the best evidence, they do not necessarily consider the unique contexts of healthcare design research, and methodologies appropriate to the field.

METHODS

Five basic types of studies are ranked based on the level of confidence that they offer regarding the estimate of an effect: (1) meta-analysis studies, (2) causal studies, (3) correlational studies, (4) descriptive studies, and (5) anecdotal evidence. Causal studies are further divided into four levels based on the interaction of two factors: (1) type of intervention and (2) groups' equivalency and extraneous variable control.

RESULTS

An eight-level hierarchy of evidence for healthcare design research is proposed that is expected to improve upon previous hierarchies in three major ways: (a) including research methods that are more relevant to healthcare design research, (b) enhancing evaluation accuracy and reliability by providing a clearer definition of studies based on their key components rather than using study labels alone, and (c) distinguishing different levels of evidence, particularly in causal studies.

CONCLUSIONS

The proposed appraisal tool is developed specifically for EBD by reflecting on the unique context of healthcare design research and practice.

Comparison of the design and methodology of Phase 3 clinical trials of bictegravir/emtricitabine/tenofovir alafenamide (BIC/FTC/TAF) and dolutegravir-based dual therapy (DTG) in HIV: a systematic review of the literature

INTRODUCTION

Current recommended antiretroviral regimens include a combination of two (dual; DT) or three (triple; TT) antiretroviral drugs. This study aims to determine whether the quality of evidence from clinical trials of dolutegravir (dolutegravir/lamivudine [DTG/3TC] or dolutegravir/rilpivirine [DTG/RPV]) is methodologically comparable to that of clinical trials conducted with bictegravir/emtricitabine/tenofovir alafenamide (BIC/FTC/TAF).

AREAS COVERED

A systematic review of the medical literature was carried out in PubMed without date or language restrictions, following the PRISMA guidelines. All aspects of the methodological design of phase 3 randomized clinical trials (RCTs) of DT and TT, evaluated by the European Medicines Agency (registration trials), were reviewed. The quality of clinical trials was assessed using the Jadad scale.

EXPERT OPINION

The search identified 5, 3 and 2 phase 3 RCTs with BIC/FTC/TAF, DTG/3TC and DTG/RPV, respectively, that met the inclusion criteria. The designs would not be comparable due to differences in pre-randomization losses, blinding, patient recruitment, as well as differences in methodological quality, with the average score of the RCTs conducted with BIC/FTC/TAF, DTG/3TC and DTG/RPV being 4.2 (high quality), 3.0 (medium quality) and 3.0 (medium quality), respectively. Due to methodological differences between the BIC/FTC/TAF, DTG/3TC and DTG/RPV RCTs, the results of these are not comparable.

Which actionable statements qualify as good practice statements In Covid-19 guidelines? A systematic appraisal

OBJECTIVES

To evaluate the development and quality of actionable statements that qualify as good practice statements (GPS) reported in COVID-19 guidelines.

DESIGN AND SETTING

Systematic review . We searched MEDLINE, MedSci, China National Knowledge Infrastructure (CNKI), databases of Grading of Recommendations Assessment, Development and Evaluation (GRADE) Guidelines, NICE, WHO and Guidelines International Network (GIN) from March 2020 to September 2021. We included original or adapted recommendations addressing any COVID-19 topic.

MAIN OUTCOME MEASURES

We used GRADE Working Group criteria for assessing the appropriateness of issuing a GPS: (1) clear and actionable; (2) rationale necessitating the message for healthcare practice; (3) practicality of systematically searching for evidence; (4) likely net positive consequences from implementing the GPS and (5) clear link to the indirect evidence. We assessed guideline quality using the Appraisal of Guidelines for Research and Evaluation II tool.

RESULTS

253 guidelines from 44 professional societies issued 3726 actionable statements. We classified 2375 (64%) as GPS; of which 27 (1%) were labelled as GPS by guideline developers. 5 (19%) were labelled as GPS by their authors but did not meet GPS criteria. Of the 2375 GPS, 85% were clear and actionable; 59% provided a rationale necessitating the message for healthcare practice, 24% reported the net positive consequences from implementing the GPS. Systematic collection of evidence was deemed impractical for 13% of the GPS, and 39% explained the chain of indirect evidence supporting GPS development. 173/2375 (7.3%) statements explicitly satisfied all five criteria. The guidelines' overall quality was poor regardless of the appropriateness of GPS development and labelling.

CONCLUSIONS

Statements that qualify as GPS are common in COVID-19 guidelines but are characterised by unclear designation and development processes, and methodological weaknesses.

Updated guideline on equipment to manage difficult airways: Australian and New Zealand College of Anaesthetists

The Australian and New Zealand College of Anaesthetists (ANZCA) recently reviewed and updated the guideline on equipment to manage a difficult airway. An ANZCA-established document development group, which included representatives from the Australasian College for Emergency Medicine and the College of Intensive Care Medicine of Australia and New Zealand, performed the review, which is based on expert consensus, an extensive literature review, and bi-nationwide consultation. The guideline (PG56(A) 2021, https://www.anzca.edu.au/getattachment/02fe1a4c-14f0-4ad1-8337-c281d26bfa17/PS56-Guideline-on-equipment-to-manage-difficult-airways) is accompanied by a detailed background paper (PG56(A)BP 2021, https://www.anzca.edu.au/getattachment/9ef4cd97-2f02-47fe-a63a-9f74fa7c68ac/PG56(A)BP-Guideline-on-equipment-to-manage-difficult-airways-Background-Paper), from which the current recommendations are reproduced on behalf of, and with the permission of, ANZCA. The updated 2021 guideline replaces the 2012 version and aims to provide an updated, objective, informed, transparent, and evidence-based review of equipment to manage difficult airways.

No Recommendation Is (at Least Presently) the Best Recommendation: An Updating Quality Appraisal of Recommendations on Screening for Scoliosis

Recommendations addressing screening for scoliosis differ substantially. Systematically developed guidelines are confronted by consensus and opinion-based statements. This paper elaborates on the issue of the standards of development and reporting of current guidelines and recommendation statements, as well as on the methodological quality of the corresponding evidence syntheses. The SORT classification and the AMSTAR 2 tool were used for those purposes, respectively. Publications included in the analysis differed substantially in terms of their methodological quality. Based on the SORT and AMSTAR 2 scores, the 2018 US PSTF recommendation statement and systematic review on screening for scoliosis are trustworthy and high-quality sources of evidence and aid for decision making. The recommendation statement on insufficient evidence to formulate any recommendations is, paradoxically, very informative. Significantly, updated opinion-based position statements supporting screening for scoliosis acknowledged the importance of research evidence as a basis for recommendation formulation and are more cautious and balanced than formerly. Expert opinions, not built on properly presented analyses of evidence, are at odds with evidence-based practice. Nonetheless, contemporary principles of screening programs, especially those addressing people's values and preferences, and the possible harms of screening, remain underrepresented in both research and recommendations addressing screening for scoliosis.

Wording of American Urological Association Guideline Recommendations Does Not Signal the Strength of Recommendation

OBJECTIVE

To systematically analyze the wording of American Urological Association (AUA) clinical practice guidelines' recommendations.

MATERIALS AND METHODS

We systematically identified all AUA guideline documents that used the current AUA framework and extracted the individual recommendations, their statement type, and their corresponding evidence level (if applicable), and analyzed the recommendation wording separating out the modal verbs (eg, should, may, or must) as well as the main verbs (eg, recommend, treat, perform). We performed descriptive statistics using SPSS Version 27.

RESULTS

We included 18 documents with 754 distinct recommendations; the median number of recommendations per guideline was 36 (interquartile range: 29; 48.5). Expert opinion was the most used statement type (193; 25.6%), followed by clinical principle (156; 20.7%). Four-hundred-four recommendations were either strong, moderate, or conditional: 135 (17.9%), 187 (24.8%) and 83 (11.0%), respectively. Most recommendation statements (701; 93.0%) used modal verbs to express a level of obligation. Overall, "should" was the most employed modal verb used (73.9%), followed by "may" (17.6%) and "must" (1.5%). Both "may" and "should" were used in conjunction with all five statement types, whereas "must" was limited to strong recommendations and clinical principles.

CONCLUSION

The wording of AUA recommendation statements does not signal the strength of recommendation and the intended level of obligation, which may represent a barrier to guideline implementation. More consistent wording of recommendations by strength may improve guideline understanding, uptake, and adherence.

Evidence-based urology: understanding GRADE methodology

Grading of Recommendations Assessment, Development, and Evaluation (GRADE) is a methodologically rigorous and transparent framework for grading the certainty of evidence and moving from evidence to recommendations in guidelines. Certainty of evidence can be rated as high, moderate, low, or very low, and some domains may lead to an increase or decrease in the certainty of evidence. Once evidence is summarized, the certainty, balance of desirable and undesirable outcomes, patients' values and preferences, equity, acceptability, and feasibility are taken into consideration in order to give a strong or weak recommendation for or against an intervention. In this review we summarize the GRADE approach using urology-related examples. PATIENT SUMMARY: GRADE is a well-established system for determining how much confidence we can place in research evidence and how guideline panels can make recommendations for patient care and health care policy. Recommendations can be strong or weak for or against a treatment strategy. Guideline developers should consider the certainty of the evidence, the balance of desirable and undesirable outcomes, patients' values and preferences, and resource use.

GRADE notes: How to use GRADE when there is "no" evidence? A case study of the expert evidence approach

OBJECTIVES

One essential requirement of trustworthy guidelines is that they should be based on systematic reviews of the best available evidence. The GRADE Working Group has provided guidance for evaluating the certainty of evidence based on several domains. However, for many clinical questions, published evidence may be limited, too indirect or simply not exist. In this brief report (GRADE notes), we describe our method of developing evidence-based recommendations when publisheddirect evidence was lacking.

STUDY DESIGN AND SETTING

When direct published literature was absent, an expert evidence survey was administered to panel members about their unpublished observations and case series. Focus was on collecting data about cases and outcome, not panel opinions.

RESULTS

Out of 26 questions prioritized by the panel for pediatric venous thromboembolism, 12 had no, very limited, or very low certainty of evidence to inform them. The panel survey was administered for these questions.

CONCLUSIONS

Areas of sparse evidence often reflect key questions that are critical to address in clinical practice guidelines due to the uncertainty among health care providers. The expert evidence approach used in this study is one method for panels totransparently deal with the lack of published evidence to directly inform recommendations.

Determining the level of evidence: Nonresearch evidence

ABSTRACT

To support evidence-based nursing practice, the authors provide guidelines for nonresearch evidence, which includes clinical practice guidelines, consensus or position statements, literature review, expert opinion, organizational experience, case reports, community standards, clinician experience, and consumer preferences. This is the third in a three-part series.

Worldwide management of donors after neurological death: a systematic review and narrative synthesis of guidelines

OBJECTIVE

The objectives of this study were to systematically identify and describe guidelines for the care of neurologically deceased donors and to evaluate their methodological quality, with the aim of informing and supporting the new Canadian guidelines for the management of organ donors.

METHODOLOGY

Following a systematic search, we included any document endorsed by an organ donation organization, a professional society, or a government, that aims to direct the medical management of adult, neurologically deceased, multi-organ donors. We extracted recommendations pertaining to six domains: the autonomic storm, hemodynamic instability, hormone supplementation, ventilation, blood product transfusions, and general intensive care unit (ICU) care. Methodological quality of the guidelines was assessed by the validated AGREE-II tool.

MAIN FINDINGS

This review includes 27 clinical practice guidelines representing 26 countries published between 1993 and 2019. Using the AGREE-II validated tool for the evaluation of guidelines' quality, documents generally scored well on their scope and clarity of presentation. Nevertheless, quality was limited in terms of the scientific rigor of guideline development. Recommendations varied substantially across the domains of managing the autonomic storm, subsequent management of hemodynamic instability, hormone therapy, mechanical ventilation, blood product transfusion, and general ICU care. We found consistent recommendations for low tidal volume ventilation subsequent to the publication of a landmark clinical trial.

CONCLUSION

Highly inconsistent recommendations for deceased donor care summarized in this review likely reflect the relatively slow emergence of high-quality clinical research in this field, as well as a late uptake of recent validated guideline methodology. Even in this context of few randomized-controlled trials, our group supported the need for new Canadian guidelines for the management of organ donors that follow rigorous recognized methodology and grading of the evidence.

TRIAL REGISTRATION

PROSPERO (CRD42018084012); registered 25 February 2016.

Evaluation of evidence grades in psychiatry and psychotherapy guidelines

BACKGROUND

Information regarding the distribution of evidence grades in psychiatry and psychotherapy guidelines is lacking. Based on the German evidence- and consensus- based (S3) psychiatry and psychotherapy and the Scottish Intercollegiate Guidelines Network (SIGN) treatment guidelines, we aimed to specify how guideline recommendations are composed and to what extent recommendations are evidence-based.

METHODS

Data was collected from all published evidence- and consensus-based S3-classified psychiatry and psychotherapy guidelines. As control conditions, data from German neurology S3-classified guidelines as well as data from recent SIGN guidelines of mental health were extracted. Two investigators reviewed the selected guidelines independently, extracted and analysed the numbers and levels of recommendations.

RESULTS

On average, 45.1% of all recommendations are not based on strong scientific evidence in German guidelines of psychiatry and psychotherapy. A related pattern can be confirmed for SIGN guidelines, where the mean average of recommendations with lacking evidence is 33.9%. By contrast, in the German guidelines of neurology the average of such recommendations is 16.5%. A total of 24.5% of all recommendations in the guidelines of psychiatry and psychotherapy are classified as level A recommendations, compared to 31.6% in the field of neurology and 31.1% in the SIGN guidelines. Related patterns were observed for B and 0 level recommendations.

CONCLUSION

Guidelines should be practical tools to simplify the decision-making process based on scientific evidence. Up to 45% of all recommendations in the investigated guidelines of psychiatry and psychotherapy are not based on strong scientific evidence. The reasons for this high number remain unclear. Possibly, only a limited number of studies answer clinically relevant questions. Our findings thereby question whether guidelines should include non-evidence-based recommendations to be methodologically stringent and whether specific processes to develop expert-opinion statements must be implemented.

Selecting Evidence-Based Content for Inclusion in Self-Management Apps for Pressure Injuries in Individuals With Spinal Cord Injury: Participatory Design Study

BACKGROUND

Technological solutions, particularly mobile health (mHealth), have been shown to be potentially viable approaches for sustaining individuals' self-management of chronic health conditions. Theory-based interventions are more successful, as evidence-based information is an essential prerequisite for appropriate self-management. However, several reviews have shown that many existing mobile apps fail to be either theoretically grounded or based on evidence. Although some authors have attempted to address these two issues by focusing on the design and development processes of apps, concrete efforts to systematically select evidence-based content are scant.

OBJECTIVE

The objective of this study was to present a procedure for the participatory identification of evidence-based content to ground the development of a self-management app.

METHODS

To illustrate the procedure, we focused on the prevention and management of pressure injuries (PIs) in individuals with spinal cord injury (SCI). The procedure involves the following three steps: (1) identification of existing evidence through review and synthesis of existing recommendations on the prevention and self-management of PIs in SCI; (2) a consensus meeting with experts from the field of SCI and individuals with SCI to select the recommendations that are relevant and applicable to community-dwelling individuals in their daily lives; and (3) consolidation of the results of the study.

RESULTS

In this case study, at the end of the three-step procedure, the content for an mHealth intervention was selected in the form of 98 recommendations.

CONCLUSIONS

This study describes a procedure for the participatory identification and selection of disease-specific evidence and professional best practices to inform self-management interventions. This procedure might be especially useful in cases of complex chronic health conditions, as every recommendation in these cases needs to be evaluated and considered in light of all other self-management requirements. Hence, the agreement of experts and affected individuals is essential to ensure the selection of evidence-based content that is considered to be relevant and applicable.

A Theoretical Framework and Competency-Based Approach to Training in Guideline Development

BACKGROUND

There is increasing requirement to develop guidelines using transparent, standardized, and rigorous methods. Consequently, a better understanding of the knowledge, skills, and expertise necessary for guideline development is needed. The aim of this manuscript is to describe a theoretical framework of knowledge and skills that are required for individuals to serve on a guideline panel in varying capacities.

METHODS

Based on an iterative process and review of published manuscripts focused on guideline development, we identified competencies, subcompetencies, and milestones.

RESULTS

Using a competency-based approach to training and the Dreyfus model of skill acquisition, we identified three core competencies: (1) facilitate the development of guideline structure and setup, (2) make judgments about the quality or certainty of the evidence, and (3) transform evidence to a recommendation. Level 1 focuses on recognizing and acknowledging the importance of a specific skill or behavior. Levels 2 and 3 require learners to demonstrate progressive acquisition of knowledge and application to specific behaviors. Level 4 represents the individual who has acquired the requisite knowledge and can function independently, while level 5 represents the mastery/aspirational level.

DISCUSSION

We propose a preliminary competency-based education framework that will (1) help standardize the qualifications needed for individuals to serve on guideline panels in varying capacities or (2) help with curricula development for teaching and training of guideline panel members. This framework can also help enable guideline-producing organizations to identify guideline methodologists with the relevant and appropriate level of knowledge and skills to lead guidelines. Validation of the framework and further refinement of the competencies and milestones will be required before widespread adoption.

Using systematic reviews in guideline development: the GRADE approach

Systematic reviews are essential to produce trustworthy guidelines. To assess the certainty of a body of evidence included in a systematic review the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group has developed an approach that is currently used by over 100 organisations, including the World Health Organization and the Cochrane Collaboration. GRADE provides operational definitions and instructions to rate the certainty of the evidence for each outcome in a review as high, moderate, low, or very low for the effects of interventions, prognostic estimates, values and preferences, test accuracy and resource utilization. The assessment includes assessing risk of bias, imprecision, inconsistency, indirectness, and publication bias, the magnitude of effects, dose-response relations and the impact of residual confounding and bias. Summary statistical information and assessments of certainty are presented in GRADE evidence summary tables, which can be produced using GRADE's official GRADEpro software tool (www.gradepro.org/). The evidence summary tables feed into the GRADE Evidence to Decision frameworks which guideline panels can use to produce recommendations.

Methodological quality and synthesis of case series and case reports

Case reports and case series are uncontrolled study designs known for increased risk of bias but have profoundly influenced the medical literature and continue to advance our knowledge. In this guide, we present a framework for appraisal, synthesis and application of evidence derived from case reports and case series. We propose a tool to evaluate the methodological quality of case reports and case series based on the domains of selection, ascertainment, causality and reporting and provide signalling questions to aid evidence-based practitioners and systematic reviewers in their assessment. We suggest using evidence derived from case reports and case series to inform decision-making when no other higher level of evidence is available.

Methodological quality and synthesis of case series and case reports

Case reports and case series are uncontrolled study designs known for increased risk of bias but have profoundly influenced the medical literature and continue to advance our knowledge. In this guide, we present a framework for appraisal, synthesis and application of evidence derived from case reports and case series. We propose a tool to evaluate the methodological quality of case reports and case series based on the domains of selection, ascertainment, causality and reporting and provide signalling questions to aid evidence-based practitioners and systematic reviewers in their assessment. We suggest using evidence derived from case reports and case series to inform decision-making when no other higher level of evidence is available.