Synthesis, grading, and presentation of evidence in guidelines: article 7 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report

Invited review: Ketone biology-The shifting paradigm of ketones and ketosis in the dairy cow

Ketosis is currently regarded as a major metabolic disorder of dairy cows, reflective of the animal's efforts to adapt to energy deficit while transitioning into lactation. Currently viewed as a pathology by some, ketosis is associatively implicated in milk production losses and peripartal health complications that increase the risk of early removal of cows from the herd, thus carrying economic losses for dairy farmers and jeopardizing the sustainability of the dairy industry. Despite decades of intense research in the mitigation of ketosis and its sequelae, our ability to lessen its purported effects remains limited. Moreover, the association of ketosis to reduced milk production and peripartal disease is often erratic and likely mired by concurrent potential confounders. In this review, we discuss the potential reasons for these apparent paradoxes in the light of currently available evidence, with a focus on the limitations of observational research and the necessary steps to unambiguously identify the effects of ketosis on cow health and performance via controlled randomized experimentation. A nuanced perspective is proposed that considers the dissociation of ketosis-as a disease-from healthy hyperketonemia. Furthermore, in consideration of a growing body of evidence that highlights positive roles of ketones in the mitigation of metabolic dysfunction and chronic diseases, we consider the hypothetical functions of ketones as health-promoting metabolites and ponder on their potential usefulness to enhance dairy cow health and productivity.

Stem Cell-Based Therapy for Experimental Ischemic Stroke: A Preclinical Systematic Review

Stem cell transplantation offers promise in the treatment of ischemic stroke. Here we utilized systematic review, meta-analysis, and meta-regression to study the biological effect of stem cell treatments in animal models of ischemic stroke. A total of 98 eligible publications were included by searching PubMed, EMBASE, and Web of Science from inception to August 1, 2020. There are about 141 comparisons, involving 5,200 animals, that examined the effect of stem cell transplantation on neurological function and infarct volume as primary outcome measures in animal models for stroke. Stem cell-based therapy can improve both neurological function (effect size, −3.37; 95% confidence interval, −3.83 to −2.90) and infarct volume (effect size, −11.37; 95% confidence interval, −12.89 to −9.85) compared with controls. These results suggest that stem cell therapy could improve neurological function deficits and infarct volume, exerting potential neuroprotective effect for experimental ischemic stroke, but further clinical studies are still needed.

Informing Healthcare Decisions with Observational Research Assessing Causal Effect. An Official American Thoracic Society Research Statement

Rationale: Decisions in medicine are made on the basis of knowledge and reasoning, often in shared conversations with patients and families in consideration of clinical practice guideline recommendations, individual preferences, and individual goals. Observational studies can provide valuable knowledge to inform guidelines, decisions, and policy.

Objectives: The American Thoracic Society (ATS) created a multidisciplinary ad hoc committee to develop a research statement to clarify the role of observational studies—alongside randomized controlled trials (RCTs)—in informing clinical decisions in pulmonary, critical care, and sleep medicine.

Methods: The committee examined the strengths of observational studies assessing causal effects, how they complement RCTs, factors that impact observational study quality, perceptions of observational research, and, finally, the practicalities of incorporating observational research into ATS clinical practice guidelines.

Measurements and Main Results: There are strengths and weakness of observational studies as well as RCTs. Observational studies can provide evidence in representative and diverse patient populations. Quality observational studies should be sought in the development of ATS clinical practice guidelines, and medical decision-making in general, when 1) no RCTs are identified or RCTs are appraised as being of low- or very low-quality (replacement); 2) RCTs are of moderate quality because of indirectness, imprecision, or inconsistency, and observational studies mitigate the reason that RCT evidence was downgraded (complementary); or 3) RCTs do not provide evidence for outcomes that a guideline committee considers essential for decision-making (e.g., rare or long-term outcomes; “sequential”).

Conclusions: Observational studies should be considered in developing clinical practice guidelines and in making clinical decisions.

Medication Discontinuation in Adults With COPD Discharged From the Hospital: A Population-Based Cohort Study

BACKGROUND

Patients admitted to the hospital with COPD are commonly managed with inhaled short-acting bronchodilators, sometimes in lieu of the long-acting bronchodilators they take as outpatients. If held on admission, these long-acting inhalers should be re-initiated upon discharge; however, health-care transitions sometimes result in unintentional discontinuation.

RESEARCH QUESTION

What is the risk of unintentional discontinuation of long-acting muscarinic antagonist (LAMA) and long-acting beta-agonist and inhaled corticosteroid (LABA-ICS) combination medications following hospital discharge in older adults with COPD?

STUDY DESIGN AND METHODS

A retrospective cohort study was conducted by using health administrative data from 2004 to 2016 from Ontario, Canada. Adults with COPD aged ≥ 66 years who had filled prescriptions for a LAMA or LABA-ICS continuously for ≥ 1 year were included. Log-binomial regression models were used to determine risk of medication discontinuation following hospitalization in each medication cohort.

RESULTS

Of the 27,613 hospitalization discharges included in this study, medications were discontinued 1,466 times. Among 78,953 patients with COPD continuously taking a LAMA or LABA-ICS, those hospitalized had a higher risk of having medications being discontinued than those who remained in the community (adjusted risk ratios of 1.50 [95% CI, 1.34-1.67; P < .001] and 1.62 [95% CI, 1.39, 1.90; P < .001] for LAMA and LABA-ICS, respectively). Crude rates of discontinuation for people taking LAMAs were 5.2% in the hospitalization group and 3.3% in the community group; for people taking LABA-ICS, these rates were 5.5% in the hospitalization group and 3.1% in the community group.

INTERPRETATION

In an observational study of highly compliant patients with COPD, hospitalization was associated with an increased risk of long-acting inhaler discontinuation. These Results suggest a likely larger discontinuation problem among less adherent patients and should be confirmed and quantified in a prospective cohort of patients with COPD and average compliance. Quality improvement efforts should focus on safe transitions and patient medication reconciliation following discharge.

Quality standards in respiratory real-life effectiveness research: the REal Life EVidence AssessmeNt Tool (RELEVANT): report from the Respiratory Effectiveness Group-European Academy of Allergy and Clinical Immunology Task Force

INTRODUCTION

A Task Force was commissioned jointly by the European Academy of Allergy and Clinical Immunology (EAACI) and the Respiratory Effectiveness Group (REG) to develop a quality assessment tool for real-life observational research to identify high-quality real-life asthma studies that could be considered within future guideline development.

METHODS

The resulting REal Life EVidence AssessmeNt Tool (RELEVANT) was achieved through an extensive analysis of existing initiatives in this area. The first version was piloted among 9 raters across 6 articles; the revised, interim, version underwent extensive testing by 22 reviewers from the EAACI membership and REG collaborator group, leading to further revisions and tool finalisation. RELEVANT was validated through an analysis of real-life effectiveness studies identified via systematic review of Medline and Embase databases and relating to topics for which real-life studies may offer valuable evidence complementary to that from randomised controlled trials. The topics were selected through a vote among Task Force members and related to the influence of adherence, smoking, inhaler device and particle size on asthma treatment effectiveness.

RESULTS

Although highlighting a general lack of high-quality real-life effectiveness observational research on these clinically important topics, the analysis provided insights into how identified observational studies might inform asthma guidelines developers and clinicians. Overall, RELEVANT appeared reliable and easy to use by expert reviewers.

CONCLUSIONS

Using such quality appraisal tools is mandatory to assess whether specific observational real-life effectiveness studies can be used to inform guideline development and/or decision-making in clinical practice.

Extending the Reach of Evidence-Based Medicine: A Proposed Categorization of Lower-Level Evidence

Clinical practice involves making many treatment decisions for which only limited formal evidence exists. While the methodology of evidence-based medicine (EBM) has evolved tremendously, there is a need to better characterize lower-level evidence. This should enhance the ability to appropriately weigh the evidence against other considerations, and counter the temptation to think it is more robust than it actually is. A framework to categorize lower-level evidence is proposed, consisting of nonrandomized comparisons, extrapolation using indirect evidence, rationale, and clinical experience (ie, an accumulated general impression). Subtypes are recognized within these categories, based on the degree of confounding in nonrandomized comparisons, the uncertainty involved in extrapolation from indirect evidence, and the plausibility of a rationale. Categorizing the available evidence in this way can promote a better understanding of the strengths and limitations of using such evidence as the basis for treatment decisions in clinically relevant areas that are devoid of higher-level evidence.

A Comparative Analysis of Pulmonary and Critical Care Medicine Guideline Development Methodologies

RATIONALE

The Institute of Medicine (IOM) standards for guideline development have had unintended negative consequences. A more efficient approach is desirable.

OBJECTIVES

To determine whether a modified Delphi process early during guideline development discriminates recommendations that should be informed by a systematic review from those that can be based upon expert consensus.

METHODS

The same questions addressed by IOM-compliant pulmonary or critical care guidelines were addressed by expert panels using a modified Delphi process, termed the Convergence of Opinion on Recommendations and Evidence (CORE) process. The resulting recommendations were compared. Concordance of the course of action, strength of recommendation, and quality of evidence, as well as the duration of recommendation development, were measured.

MEASUREMENTS AND MAIN RESULTS

When 50% agreement was required to make a recommendation, all questions yielded recommendations, and the recommended courses of action were 89.6% concordant. When 70% agreement was required, 17.9% of questions did not yield recommendations, but for those that did, the recommended courses of action were 98.2% concordant. The time to completion was shorter for the CORE process (median, 19.3 vs. 1,309.0 d; P = 0.0002).

CONCLUSIONS

We propose the CORE process as an early step in guideline creation. Questions for which 70% agreement on a recommendation cannot be achieved should go through an IOM-compliant process; however, questions for which 70% agreement on a recommendation can be achieved can be accepted, avoiding a lengthy systematic review.

Idiopathic pulmonary fibrosis - clinical management guided by the evidence-based GRADE approach: what arguments can be made against transparency in guideline development?

Evidence-based guidelines have undergone an incredible transformation over the last number of years. Significant advances include explicit linkages of systematic evidence summaries to the strength and direction of recommendations, consideration of all patient-important factors, transparent reporting of the recommendation generation process including conflict of interest management strategies and the production of clinical practice guidelines which use simple and clear language. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology provides a framework for guideline development and was employed to produce the recently published ATS/ERS/JRS/ALAT update on treatment for idiopathic pulmonary fibrosis (IPF). Herein we discuss the advantages of using an evidence-based approach for guideline development using the IPF process and resultant document as an example.

Quality standards for real-world research. Focus on observational database studies of comparative effectiveness

Real-world research can use observational or clinical trial designs, in both cases putting emphasis on high external validity, to complement the classical efficacy randomized controlled trials (RCTs) with high internal validity. Real-world research is made necessary by the variety of factors that can play an important a role in modulating effectiveness in real life but are often tightly controlled in RCTs, such as comorbidities and concomitant treatments, adherence, inhalation technique, access to care, strength of doctor-caregiver communication, and socio-economic and other organizational factors. Real-world studies belong to two main categories: pragmatic trials and observational studies, which can be prospective or retrospective. Focusing on comparative database observational studies, the process aimed at ensuring high-quality research can be divided into three parts: preparation of research, analyses and reporting, and discussion of results. Key points include a priori planning of data collection and analyses, identification of appropriate database(s), proper outcomes definition, study registration with commitment to publish, bias minimization through matching and adjustment processes accounting for potential confounders, and sensitivity analyses testing the robustness of results. When these conditions are met, observational database studies can reach a sufficient level of evidence to help create guidelines (i.e., clinical and regulatory decision-making).

Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise

BACKGROUND

Although several tools to evaluate the credibility of health care guidelines exist, guidance on practical steps for developing guidelines is lacking. We systematically compiled a comprehensive checklist of items linked to relevant resources and tools that guideline developers could consider, without the expectation that every guideline would address each item.

METHODS

We searched data sources, including manuals of international guideline developers, literature on guidelines for guidelines (with a focus on methodology reports from international and national agencies, and professional societies) and recent articles providing systematic guidance. We reviewed these sources in duplicate, extracted items for the checklist using a sensitive approach and developed overarching topics relevant to guidelines. In an iterative process, we reviewed items for duplication and omissions and involved experts in guideline development for revisions and suggestions for items to be added.

RESULTS

We developed a checklist with 18 topics and 146 items and a webpage to facilitate its use by guideline developers. The topics and included items cover all stages of the guideline enterprise, from the planning and formulation of guidelines, to their implementation and evaluation. The final checklist includes links to training materials as well as resources with suggested methodology for applying the items.

INTERPRETATION

The checklist will serve as a resource for guideline developers. Consideration of items on the checklist will support the development, implementation and evaluation of guidelines. We will use crowdsourcing to revise the checklist and keep it up to date.

A guide to guidelines for pulmonary, sleep, and critical care medicine clinicians

Both the Institute of Medicine and the Guidelines International Network have recently published standards for trustworthy guidelines. The standards address multiple aspects of guideline development, including being transparent about funding and methodology, minimizing bias related to conflicts of interest, assembling writing committees with broad stakeholder representation, using rigorous and systematic methods to synthesize evidence and formulate recommendations, and periodically assessing guidelines for currency and updating them as required. In this article, we present the perspective of the Documents Development and Implementation Committee of the American Thoracic Society (ATS) on these and other guideline-related topics of relevance to ATS members. In addition, we summarize the many important take-home messages from a workshop that was jointly sponsored by the ATS and the European Respiratory Society, and attempt to place these messages in the context of a methodology that is rapidly evolving and a landscape in which clinical practice guidelines are subjected to ever-increasing scrutiny by clinicians, patients, and other third-party stakeholders.

Reporting and publishing guidelines: article 12 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report

INTRODUCTION

Professional societies, like many other organizations around the world, have recognized the need to use rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the twelfth of a series of 14 articles that were prepared to advise guideline developers in respiratory and other diseases. This article discusses the reporting and publishing of guidelines.

METHODS

The authors formulated and discussed the following questions on the reporting and publishing of guidelines. (1) What should be reported in guidelines? (2) How should guidelines be written? (3) How should the bottom-line message be conveyed? (4) How should guidelines be packaged? (5) Where should guidelines be published? (6) Who benefits from the publication of guidelines? (7) What information should be vetted by the editor(s)? (8) How should guidelines be peer reviewed? We conducted a review of the literature, looking for systematic reviews and methodological research that addressed these questions, but we did not conduct a full systematic review. Our conclusions are based on the available evidence from the published literature and logical arguments from experienced guideline developers.

RESULTS AND DISCUSSION

There is little empirical evidence that addresses the reporting and publishing of guidelines. A standard format for reporting guidelines is desirable to ensure that guidelines are comprehensive and that all of the information necessary to judge their quality is presented. In addition, guidelines should contain concise evidence-based recommendations. To facilitate the use of guidelines by consumers, it is preferable to publish them in journals that serve the target audience and to package them in multiple ways. Editors and peer reviewers should ensure that reporting standards have been met, potential conflicts of interest have been adequately addressed and made public, and that the recommendations address important clinical questions.

Stakeholder involvement: how to do it right: article 9 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report

INTRODUCTION

Professional societies, like many other organizations around the world, have recognized the need to use more rigorous processes to ensure that healthcare recommendations are informed by the best available research evidence with input from appropriate stakeholders. This is the ninth of a series of 14 articles that were prepared by an international panel to advise guideline developers in respiratory and other diseases on approaches for guideline development. We updated a review of the literature on stakeholder involvement, focusing on six key questions.

METHODS

In this review we addressed the following questions. (1) What are "stakeholders"? (2) Why involve stakeholders in guidelines? (3) At what stage should stakeholders contribute to guidelines? (4) What are the potential barriers to integrating stakeholder involvement? (5) How can stakeholders be involved effectively? (6) Should anyone be excluded from the process? We searched PubMed and other databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct our own systematic reviews. Our conclusions are based on available evidence, the experience of guideline developers, and workshop discussions.

RESULTS AND DISCUSSION

Stakeholders are all those who have a legitimate interest in a guideline. They include healthcare professionals, patients and caregivers, public and private funding bodies, managers, employers, and manufacturers. Their engagement is justified for several reasons, including limitations of evidence, principles of transparency and democracy, ownership, and potential policy implications. They have a role to play at different points of guideline development, but their involvement can be complex. To be successful, stakeholder engagement needs to be inclusive, equitable, and adequately resourced.