1
|
Djulbegovic B, Hozo I, Guyatt G. Decision theoretical foundations of clinical practice guidelines: an extension of the ASH thrombophilia guidelines. Blood Adv 2024; 8:3596-3606. [PMID: 38625997 DOI: 10.1182/bloodadvances.2024012931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2024] [Revised: 03/29/2024] [Accepted: 03/29/2024] [Indexed: 04/18/2024] Open
Abstract
ABSTRACT Decision analysis can play an essential role in informing practice guidelines. The American Society of Hematology (ASH) thrombophilia guidelines have made a significant step forward in demonstrating how decision modeling integrated within Grading of Recommendations Assessment, Developing, and Evaluation (GRADE) methodology can advance the field of guideline development. Although the ASH model was transparent and understandable, it does, however, suffer from certain limitations that may have generated potentially wrong recommendations. That is, the panel considered 2 models separately: after 3 to 6 months of index venous thromboembolism (VTE), the panel compared thrombophilia testing (A) vs discontinuing anticoagulants (B) and testing (A) vs recommending indefinite anticoagulation to all patients (C), instead of considering all relevant options simultaneously (A vs B vs C). Our study aimed to avoid what we refer to as the omitted choice bias by integrating 2 ASH models into a single unifying threshold decision model. We analyzed 6 ASH panel's recommendations related to the testing for thrombophilia in settings of "provoked" vs "unprovoked" VTE and low vs high bleeding risk (total 12 recommendations). Our model disagreed with the ASH guideline panels' recommendations in 4 of the 12 recommendations we considered. Considering all 3 options simultaneously, our model provided results that would have produced sounder recommendations for patient care. By revisiting the ASH guidelines methodology, we have not only improved the recommendations for thrombophilia but also provided a method that can be easily applied to other clinical problems and promises to improve the current guidelines' methodology.
Collapse
Affiliation(s)
- Benjamin Djulbegovic
- Division of Medical Hematology and Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC
| | - Iztok Hozo
- Department of Mathematics, Indiana University Northwest, Gary, IN
| | - Gordon Guyatt
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| |
Collapse
|
2
|
Kepp KP, Aavitsland P, Ballin M, Balloux F, Baral S, Bardosh K, Bauchner H, Bendavid E, Bhopal R, Blumstein DT, Boffetta P, Bourgeois F, Brufsky A, Collignon PJ, Cripps S, Cristea IA, Curtis N, Djulbegovic B, Faude O, Flacco ME, Guyatt GH, Hajishengallis G, Hemkens LG, Hoffmann T, Joffe AR, Klassen TP, Koletsi D, Kontoyiannis DP, Kuhl E, La Vecchia C, Lallukka T, Lambris J, Levitt M, Makridakis S, Maltezou HC, Manzoli L, Marusic A, Mavragani C, Moher D, Mol BW, Muka T, Naudet F, Noble PW, Nordström A, Nordström P, Pandis N, Papatheodorou S, Patel CJ, Petersen I, Pilz S, Plesnila N, Ponsonby AL, Rivas MA, Saltelli A, Schabus M, Schippers MC, Schünemann H, Solmi M, Stang A, Streeck H, Sturmberg JP, Thabane L, Thombs BD, Tsakris A, Wood SN, Ioannidis JPA. Panel stacking is a threat to consensus statement validity. J Clin Epidemiol 2024; 173:111428. [PMID: 38897481 DOI: 10.1016/j.jclinepi.2024.111428] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2024] [Revised: 06/09/2024] [Accepted: 06/10/2024] [Indexed: 06/21/2024]
Abstract
Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members toward one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but nonfinancial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analyzing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated toward COVID-19 elimination (Zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases toward advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.
Collapse
Affiliation(s)
- Kasper P Kepp
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA
| | - Preben Aavitsland
- Pandemic Centre, University of Bergen, Bergen, Norway; Norwegian Institute of Public Health, Oslo, Norway
| | - Marcel Ballin
- Centre for Epidemiology and Community Medicine, Region Stockholm, Stockholm, Sweden; Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden
| | | | - Stefan Baral
- Department of Epidemiology, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of International Health, Johns Hopkins School of Public Health, Baltimore, MD, USA; Department of Health, Policy, and Management, Johns Hopkins School of Public Health, Baltimore, MD, USA
| | - Kevin Bardosh
- School of Public Health, University of Washington, Seattle, WA, USA; Edinburgh Medical School, University of Edinburgh, Edinburgh, UK
| | - Howard Bauchner
- Department of Pediatrics, Boston University School of Medicine, Boston, MA, USA
| | - Eran Bendavid
- Department of Medicine (Primary Care and Population Health), Stanford University School of Medicine, Stanford, CA, USA; Department of Health Policy, Stanford University School of Medicine, Stanford, CA, USA; Freeman Spogli Institute for International Studies, Stanford University, Stanford, CA, USA
| | - Raj Bhopal
- Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Daniel T Blumstein
- Department of Ecology & Evolutionary Biology, Institute of the Environment & Sustainability, University of California Los Angeles, Los Angeles, CA, USA
| | - Paolo Boffetta
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy; Stony Brook Cancer Center, Stony Brook University, Stony Brook, NY, USA
| | | | - Adam Brufsky
- Division of Hematology-Oncology, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, USA
| | - Peter J Collignon
- Department of Infectious Diseases and Microbiology, Canberra Hospital, Garran, Australian Capital Territory, Australia; Department of Infectious Disease, Medical School, Australian National University, Acton, Australian Capital Territory, Australia
| | - Sally Cripps
- Human Technology Institute, University of Technology Sydney, Sydney, Australia
| | - Ioana A Cristea
- Department of General Psychology, University of Padova, Padova, Italy
| | - Nigel Curtis
- Department of Paediatrics, The University of Melbourne, Parkville, Australia; Infectious Diseases Research Group, Murdoch Children's Research Institute, Parkville, Australia; Infectious Diseases Unit, The Royal Children's Hospital Melbourne, Parkville, Australia
| | - Benjamin Djulbegovic
- Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA
| | - Oliver Faude
- Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
| | - Maria Elena Flacco
- Department of Environmental and Prevention Sciences, University of Ferrara, Ferrara, Italy
| | - Gordon H Guyatt
- Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Faculty of Health Sciences, Department of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - George Hajishengallis
- Department of Basic and Translational Sciences, Penn Dental Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Lars G Hemkens
- Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland
| | - Tammy Hoffmann
- Faculty of Health Sciences and Medicine, Institute for Evidence-Based Healthcare, Bond University, Robina, Queensland, Australia
| | - Ari R Joffe
- Department of Pediatrics and John Dossetor Health Ethics Center, University of Alberta, Edmonton, Alberta, Canada
| | - Terry P Klassen
- Children's Hospital Research Institute of Manitoba, Department of Pediatrics and Child Health, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Manitoba, Canada
| | - Despina Koletsi
- Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland
| | - Dimitrios P Kontoyiannis
- Division of Internal Medicine, Department of Infectious Diseases, Infection Control, and Employee Health, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Ellen Kuhl
- Departments of Mechanical Engineering and of Bioengineering, Stanford University, Stanford, CA, USA
| | - Carlo La Vecchia
- Department of Clinical Sciences and Community Health, Università degli Studi di Milano, Milano, Italy
| | - Tea Lallukka
- Department of Public Health, University of Helsinki, Helsinki, Finland
| | - John Lambris
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael Levitt
- Department of Structural Biology, Stanford University School of Medicine, Stanford, CA, USA
| | - Spyros Makridakis
- Institute For the Future (IFF), University of Nicosia, Nicosia, Cyprus
| | - Helena C Maltezou
- Directorate of Research, Studies and Documentation, National Public Health Organization, Athens, Greece
| | - Lamberto Manzoli
- Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy
| | - Ana Marusic
- Department of Research in Biomedicine and Health and Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia
| | - Clio Mavragani
- Department of Physiology, Medical School, National and Kapodistrian University of Athens, Athens, Greece
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Ben W Mol
- Department of Obstetrics and Gynaecology, Monash University, Clayton, Australia
| | | | - Florian Naudet
- Research Institute for Environmental and Occupational Health (IRSET, UMR_S INSERM 1085), University of Rennes, Rennes, France; Institut Universitaire de France, Paris, France; Clinical Investigation Center (INSERM CIC 1414) and Adult Psychiatry Department, Rennes University Hospital, Rennes, France
| | - Paul W Noble
- Department of Medicine, Women's Guild Lung Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Anna Nordström
- Department of Public Health and Clinical Medicine, Umeå University, Umeå, Sweden; School of Sport Sciences, UiT the Arctic University of Norway, Tromsø, Norway; Department of Health Sciences, The Swedish Winter Sport Research Centre, Mid Sweden University, Östersund, Sweden
| | - Peter Nordström
- Department of Public Health and Caring Sciences, Clinical Geriatrics, Uppsala University, Uppsala, Sweden
| | - Nikolaos Pandis
- Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland
| | - Stefania Papatheodorou
- Department of Biostatistics and Epidemiology, Rutgers School of Public Health, Piscataway, NJ, USA; Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Chirag J Patel
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| | - Irene Petersen
- Department of Primary Care and Population Health, University College London, London, UK
| | - Stefan Pilz
- Division Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, Graz, Austria
| | - Nikolaus Plesnila
- Institute for Stroke and Dementia Research (ISD), Ludwig-Maximilians-University Munich, Munich, Germany; Munich Cluster for Systems Neurology (Synergy), Munich, Germany
| | - Anne-Louise Ponsonby
- The Florey Institute of Neuroscience and Mental Health, Melbourne, Australia; Murdoch Children's Research Institute, Royal Children's Hospital, Melbourne, Australia; Centre of Epidemiology and Biostatistics, School of Population and Global Health, University of Melbourne, Melbourne, Australia
| | - Manuel A Rivas
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA
| | - Andrea Saltelli
- UPF Barcelona School of Management, Barcelona, Spain; Centre for the Study of the Sciences and the Humanities, University of Bergen, Bergen, Norway
| | - Manuel Schabus
- Department of Psychology, University of Salzburg, Salzburg, Austria
| | - Michaéla C Schippers
- Department of Organisation and Personnel Management, Erasmus University Rotterdam, Rotterdam, Netherlands
| | - Holger Schünemann
- Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Marco Solmi
- Department of Psychiatry, University of Ottawa, Ottawa, Ontario, Canada; Department of Child and Adolescent Psychiatry, Charité Universitätsmedizin, Berlin, Germany
| | - Andreas Stang
- Institute of Medical Informatics, Biometry and Epidemiology, University Hospital Essen, Essen, Germany
| | - Hendrik Streeck
- Faculty of Medicine, Institute of Virology, University of Bonn, Bonn, Germany
| | - Joachim P Sturmberg
- College of Health, Medicine and Wellbeing, University of Newcastle, Holgate, New South Wales, Australia; International Society for Systems and Complexity Sciences for Health, Waitsfield, Vermont, USA
| | - Lehana Thabane
- Faculty of Health Sciences, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Brett D Thombs
- Lady Davis Institute for Medical Research, Jewish General Hospital, Montreal, Quebec, Canada; Department of Psychiatry, McGill University, Montreal, Quebec, Canada; Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada; Department of Medicine, McGill University, Montreal, Quebec, Canada; Biomedical Ethics Unit, McGill University, Montreal, Quebec, Canada; Department of Psychology, McGill University, Montreal, Quebec, Canada
| | - Athanasios Tsakris
- Department of Microbiology, Medical School, University of Athens, Athens, Greece
| | - Simon N Wood
- Chair of Computational Statistics, School of Mathematics, University of Edinburgh, Edinburgh, UK
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, USA; Department of Medicine (Stanford Prevention Research Center), Stanford University School of Medicine, Stanford, CA, USA; Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA.
| |
Collapse
|
3
|
Djulbegovic B, Hozo I, Cuker A, Guyatt G. Improving methods of clinical practice guidelines: From guidelines to pathways to fast-and-frugal trees and decision analysis to develop individualised patient care. J Eval Clin Pract 2024; 30:393-402. [PMID: 38073027 DOI: 10.1111/jep.13953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/11/2023] [Revised: 11/16/2023] [Accepted: 11/20/2023] [Indexed: 01/30/2024]
Abstract
BACKGROUND Current methods for developing clinical practice guidelines have several limitations: they are characterised by the "black box" operation-a process with defined inputs and outputs but an incomplete understanding of its internal workings; they have "the integration problem"-a lack of framework for explicitly integrating factors such as patient preferences and trade-offs between benefits and harms; they generate one recommendation at a time that typically are not connected in a coherent analytical framework; and they apply to "average" patients, while clinicians and their patients seek advice tailored to individual circumstances. METHODS We propose augmenting the current guideline development method by converting evidence-based pathways into fast-and-frugal decision trees (FFTs) and integrating them with generalised decision curve analysis to formulate clear, individualised management recommendations. RESULTS We illustrate the process by developing recommendations for the management of heparin-induced thrombocytopenia (HIT). We converted evidence-based pathways for HIT, developed by the American Society of Hematology, into an FFT. Here, we consider only thrombotic complications and major bleeding. We leveraged the predictive potential of FFTs to compare the effects of argatroban, bivalirudin, fondaparinux, and direct oral anticoagulants (DOACs) using generalised decision curve analysis. We found that DOACs were superior to other treatments if the FFT-predicted probability of HIT exceeded 3%. CONCLUSIONS The proposed analytical framework connects guidelines, pathways, FFTs, and decision analysis, offering risk-tailored personalised recommendations and addressing current guideline development critiques.
Collapse
Affiliation(s)
- Benjamin Djulbegovic
- Division of Medical Hematology and Oncology, Medical University of South Carolina, Charleston, South Carolina, USA
| | - Iztok Hozo
- Department of Mathematics, Indiana University Northwest, Gary, Indiana, USA
| | - Adam Cuker
- Department of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Gordon Guyatt
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
| |
Collapse
|
4
|
Gao Y, Liu Z, Cao R, Liao Y, Feng Y, Su C, Guan X, Fang R, Deng Y, Xiang W, Liu J, Li Y, Fei Y. Emphasis should be placed on identifying and reporting research priorities to increase research value: An empirical analysis. PLoS One 2024; 19:e0300841. [PMID: 38517858 PMCID: PMC10959327 DOI: 10.1371/journal.pone.0300841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 03/04/2024] [Indexed: 03/24/2024] Open
Abstract
OBJECTIVES To compared the presentation of research priorities in the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) clinical practice guidelines (CPGs) developed under the guidance of the GRADE working group or its two co-chair, and the Chinese CPGs. METHODS This was a methodological empirical analysis. We searched PubMed, Embase, and four Chinese databases (Wanfang, VIP Database for Chinese Technical Periodicals, China National Knowledge Infrastructure and Chinese Biomedical Literature Database) and retrieved nine Chinese guideline databases or Society websites as well as GRADE Pro websites. We included all eligible GRADE CPGs and a random sample of double number of Chinese CPGs, published 2018 to 2022. The reviewers independently screened and extracted the data, and we summarized and analyzed the reporting on the research priorities in the CPGs. RESULTS Of the 135 eligible CPGs (45 GRADE CPGs and 90 Chinese CPGs), 668, 138 research priorities were identified respectively. More than 70% of the research priorities in GRADE CPGs and Chinese CPGs had population and intervention (PI) structure. 99 (14.8%) of GRADE CPG research priorities had PIC structures, compared with only 4(2.9%) in Chinese. And 28.4% (190) GRADE CPG research priorities reflected comparisons between PICO elements, approximately double those in Chinese. The types of research priorities among GRADE CPGs and Chinese CPGs were mostly focused on the efficacy of interventions, and the type of comparative effectiveness in the GRADE research priorities was double those in Chinese. CONCLUSIONS There was still considerable room for improvement in the developing and reporting of research priorities in Chinese CPGs. Key PICO elements were inadequately presented, with more attention on intervention efficacy and insufficient consideration given to values, preferences, health equity, and feasibility. Identifying and reporting of research priorities deserves greater effort in the future.
Collapse
Affiliation(s)
- Yicheng Gao
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Zhihan Liu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Rui Cao
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Yingdi Liao
- Kunming Traditional Chinese Medicine Hospital, Kunming, China
| | - Yuting Feng
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Chengyuan Su
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Xinmiao Guan
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Rui Fang
- Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Wulumuqi, China
| | - Yingjie Deng
- Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Wulumuqi, China
| | - Wenyuan Xiang
- Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Wulumuqi, China
| | - Junchang Liu
- Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Wulumuqi, China
| | - Yuanyuan Li
- Affiliated Hospital of Traditional Chinese Medicine, Xinjiang Medical University, Wulumuqi, China
| | - Yutong Fei
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| |
Collapse
|
5
|
Li SA, Guyatt GH, Yao L, Donn G, Wang Q, Zhu Y, Yan L, Djulbegovic B. Guideline panel social dynamics influence the development of clinical practice recommendations: a mixed-methods systematic review. J Clin Epidemiol 2024; 166:111224. [PMID: 38036187 DOI: 10.1016/j.jclinepi.2023.111224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 11/07/2023] [Accepted: 11/26/2023] [Indexed: 12/02/2023]
Abstract
OBJECTIVES To synthesize empirical studies that investigate the cognitive and social processes involved in the deliberation process of guideline development meetings and determine the distribution of deliberated topics. STUDY DESIGN AND SETTING We conducted a mixed-method systematic review using a convergent segregated approach. We searched for empirical studies that investigate the intragroup dynamics of guideline development meetings indexed in bibliographic databases. RESULTS Of the 5,899 citations screened, 12 studies from six countries proved eligible. Chairs, cochairs, and methodologists contributed to at least one-third of the discussion time in guideline development meetings; patient partners contributed the least. In interdisciplinary groups, male gender and occupation as a physician were positively associated with the amount of contribution. Compared to groups that used the Grading of Recommendations Assessment, Development and Evaluation approach, for groups that did not, when faced with insufficient or low-quality evidence, relied more on their clinical experience. The presence of a cognitive "yes" bias was apparent in meetings: panelists tended to acquiesce with positive statements that required less cognitive effort than negative statements. CONCLUSION The social dynamics of the discussions were linked to each panelist's activity role, professional background, and gender, all of which influenced the level of contributions they made in guideline development meetings.
Collapse
Affiliation(s)
- Shelly-Anne Li
- Department of Family & Community Medicine, University Health Network, 440 Bathurst Street, Toronto, Ontario M6T 2S6, Canada.
| | - Gordon H Guyatt
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4K1, Canada
| | - Liang Yao
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4K1, Canada
| | - Gemma Donn
- Department of Curriculum and Pedagogy, Ontario Institute for Studies in Education, 252 Bloor St W, Toronto, Ontario M5S 1V6, Canada
| | - Qi Wang
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4K1, Canada
| | - Ying Zhu
- Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4K1, Canada
| | - Lijiao Yan
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing 100029, China
| | - Benjamin Djulbegovic
- Department of Computational & Quantitative Medicine, Beckman Research Institute, City of Hope, 1500 E Duarte Rd, Duarte, CA 91010, USA
| |
Collapse
|
6
|
Gao YC, Cao R, Liu ZH, Liao YD, Tao LY, Feng YT, Chai QY, Luo MJ, Fei YT. Comprehensive consideration of multiple determinants from evidence to recommendations in guidelines for most traditional Chinese medicine was suboptimal: a systematic review. BMC Complement Med Ther 2024; 24:19. [PMID: 38178118 PMCID: PMC10765706 DOI: 10.1186/s12906-023-04321-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2023] [Accepted: 12/20/2023] [Indexed: 01/06/2024] Open
Abstract
BACKGROUND The overall comprehensive consideration of the factors influencing the recommendations in the traditional Chinese medicine (TCM) guidelines remains poorly studied. This study systematically evaluate the factors influencing recommendations formation in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) clinical practice guidelines (CPGs) and TCM CPGs. METHODS This was a methodological review in which we searched six databases and multiple related websites. The GRADE CPGs were identified as the guidelines developed by the GRADE Working Group or the two Co-Chairs. For the TCM CPGs, we randomly selected guidelines that were published by the TCM or integrative medicine academic societies from China mainland (published by the TCM or integrative medicine academic societies of China mainland). Two reviewers independently screened and extracted data. We included CPGs published in 2018-2022. We extracted information on the influencing factors of evidence to recommendation and conducted the analyses using descriptive statistics and calculated the proportion of relevant items by IBM SPSS Statistics and Microsoft Excel to compare the differences between the GRADE CPGs and the TCM CPGs. RESULTS Forty-five GRADE CPGs (including 912 recommendations) and 88 TCM CPGs (including 2452 recommendations) were included. TCM recommendations mainly considered the four key determinants of desirable anticipated effects, undesirable anticipated effects, balance between desirable and undesirable effects, certainty of evidence, with less than 20% of other dimensions. And TCM CPGs presented more strong recommendations (for or against) and inappropriate discordant recommendations than GRADE CPGs. GRADE CPGs were more comprehensive considered about the factors affecting the recommendations, and considered more than 70% of all factors in the evidence to recommendation. CONCLUSIONS The TCM CPGs lack a comprehensive consideration of multiple influencing determinants from evidence to recommendations. In the future, the correct application of the GRADE approaches should be emphasized.
Collapse
Affiliation(s)
- Yi-Cheng Gao
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Rui Cao
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Zhi-Han Liu
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
- Kunming Traditional Chinese Medicine Hospital, Kunming, China
| | - Ying-Di Liao
- Kunming Traditional Chinese Medicine Hospital, Kunming, China
| | - Li-Yuan Tao
- Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
| | - Yu-Ting Feng
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Qian-Yun Chai
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Min-Jing Luo
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China
- Beijing GRADE Centre, Beijing, China
| | - Yu-Tong Fei
- Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China.
- Institute of Excellence in Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China.
- Beijing GRADE Centre, Beijing, China.
| |
Collapse
|
7
|
Becker M, Hansen U, Eikermann M. [Are the assessments of individual out-of-pocket health services of the IGeL-Monitor in line with clinical guidelines?]. DAS GESUNDHEITSWESEN 2023; 85:1192-1199. [PMID: 38081174 PMCID: PMC10713336 DOI: 10.1055/a-2158-8869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
OBJECTIVES The IGeL-Monitor of the Federal Medical Advisory Service in Germany evaluates the benefits and harms of individual out-of-pocket health services (in German: Individuelle Gesundheitsleistungen / IGeL). The aim of the analysis was to systematically compare IGeL-assessements with the recommendations from evidence-based guidelines. METHOD To identify guidelines, we conducted searches in guidelines databases (AWMF, Guidelines International Network and Trip database) and the websites of guideline organisations (February/March 2022). We included guidelines that were not older than 5 years. The methodological quality of the guidelines was assessed using the AGREE II instrument. We compared the recommendations with the IGeL-assessments in terms of content and grade of recommendation. RESULTS We identified 41 guidelines covering 24 IGeL-assessements. 19 (79%) assessments (nearly) were in agreement with the guideline recommendations. No comparison was possible for 5 IGeL-assessements, because, for example, the recommendations were more specific. Ten of the 13 IGeL that were rated (tendentially) negatively were also not recommended in the guidelines. CONCLUSION Overall, the IGeL-assessments were consistent with the recommendations of current guidelines. Accordingly, guideline groups seem to assess the evidence similarly to the IGeL-Monitor team. Insured persons should be informed honestly about the evidence, particularly for the (tendentially) negatively evaluated IGeL that are not recommended even in guidelines.
Collapse
Affiliation(s)
- Monika Becker
- Bereich Evidenzbasierte Medizin, Medizinischer Dienst Bund, Essen,
Germany
| | - Ute Hansen
- Bereich Evidenzbasierte Medizin, Medizinischer Dienst Bund, Essen,
Germany
| | - Michaela Eikermann
- Bereich Evidenzbasierte Medizin, Medizinischer Dienst Bund, Essen,
Germany
| |
Collapse
|
8
|
Byrne M, Mattison R, Bercovitz R, Lottenberg R, Rezende SM, Silverstein R, Terrell D, Kunkle R, Smith D, Bollard C, Haberichter S, Holter-Chakrabarty J, Pai M, Cheung M, Cuker A, Seftel M, Djulbegovic B. Identifying experts for clinical practice guidelines: perspectives from the ASH Guideline Oversight Subcommittee. Blood Adv 2023; 7:4323-4326. [PMID: 37186271 PMCID: PMC10424133 DOI: 10.1182/bloodadvances.2023010039] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 04/17/2023] [Accepted: 04/17/2023] [Indexed: 05/17/2023] Open
Affiliation(s)
| | - Ryan Mattison
- Department of Medicine, Division of Hematology/Oncology, University of Wisconsin-Madison, Madison, WI
| | | | - Richard Lottenberg
- Department of Medicine, Division of Hematology/Oncology, University of Florida, Gainesville, FL
| | - Suely M. Rezende
- Department of Internal Medicine, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
| | - Roy Silverstein
- Department of Medicine, Medical College of Wisconsin, Milwaukee, WI
| | - Deirdra Terrell
- Department of Biostatistics and Epidemiology, University of Oklahoma Health Sciences Center, Oklahoma City, OK
| | - Rob Kunkle
- American Society of Hematology, Washington, DC
| | - Deion Smith
- American Society of Hematology, Washington, DC
| | - Catherine Bollard
- Children's National Medical/George Washington University, Washington, DC
| | | | - Jennifer Holter-Chakrabarty
- Department of Biostatistics and Epidemiology, University of Oklahoma Health Sciences Center, Oklahoma City, OK
| | - Menaka Pai
- Department of Medicine, Division of Hematology and Thromboembolism, McMaster University, Hamilton, ON, Canada
| | | | - Adam Cuker
- Department of Pathology/Laboratory Medicine, University of Pennsylvania, Philadelphia, PA
| | - Matthew Seftel
- Department of Medicine, Division of Hematology, University of British Columbia and Canadian Blood Services, Vancouver, BC, Canada
| | - Benjamin Djulbegovic
- Department of Medicine, Division of Hematology/Medical Oncology, Medical University of South Carolina, Charleston, SC
| |
Collapse
|
9
|
Brun C, Zerhouni O, Akinyemi A, Houtin L, Monvoisin R, Pinsault N. Impact of uncertainty intolerance on clinical reasoning: A scoping review of the 21st-century literature. J Eval Clin Pract 2023; 29:539-553. [PMID: 36071694 DOI: 10.1111/jep.13756] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 08/11/2022] [Accepted: 08/16/2022] [Indexed: 12/29/2022]
Abstract
UNLABELLED RATIONALE, AIMS AND OBJECTIVES: Clinical reasoning is currently extensively studied to find out how to make proper diagnoses. Literature indicates that intolerance of uncertainty (IU) may have a strong negative impact on clinical reasoning. We summarize the various consequences of IU on clinical reasoning. METHODS A scoping review was conducted using relevant keywords to scientific databases (i.e., Google Scholar, Medline, PsycINFO and PBSC) from September to November 2021. Complementary research included relevant articles and articles retrieved through Google Scholar's alert system. We included articles about healthcare professionals as defined by the French Public Health Code (As defined here: https://www.vie-publique.fr/fiches/37855-categories-de-professionnels-de-sante-code-se-la-sante-publique), and articles reporting on the impact of IU or uncertainty management on clinical reasoning. RESULTS We retrieved 1853 articles, of which 8 were kept for final analysis considering our inclusion criteria. Two behaviour categories were affected by uncertainty intolerance: investigative and prescriptive behaviours. Regarding the investigation process, mismanagement of uncertainty appeared to lead to reasoning bias, potentially resulting in diagnostic errors. IU was associated with withholding information, more referrals to peers and less use of new medical interventions. Regarding prescription behaviours, IU among health professionals could lead to overprescribing unnecessary or dangerous tests. IU was also associated with more antibiotic prescriptions for conditions where antibiotics are to be used carefully. CONCLUSION Few studies have yet addressed the impact of IU on clinical reasoning. IU's influence is primarily observed on investigative and prescribing behaviours. More studies are needed to fully understand the impact of IU on clinical reasoning itself, and not only on practical consequences.
Collapse
Affiliation(s)
- Clémence Brun
- Université Grenoble Alpes, TIMC-IMAG UMR CNRS 5525, ThEMAS Team, Domaine de la Merci, La Tronche, France
| | - Oulmann Zerhouni
- Laboratoire Parisien de Psychologie Sociale, EA 4386 (équipe PS2C), Nanterre, France
| | - Alexis Akinyemi
- Laboratoire Parisien de Psychologie Sociale, EA 4386 (équipe PS2C), Nanterre, France
| | - Laurène Houtin
- Laboratoire Parisien de Psychologie Sociale, EA 4386 (équipe PS2C), Nanterre, France
| | - Richard Monvoisin
- Université Grenoble Alpes, TIMC-IMAG UMR CNRS 5525, ThEMAS Team, Domaine de la Merci, La Tronche, France
| | - Nicolas Pinsault
- Université Grenoble Alpes, TIMC-IMAG UMR CNRS 5525, ThEMAS Team, Domaine de la Merci, La Tronche, France
| |
Collapse
|
10
|
Chong MC, Sharp MK, Smith SM, O'Neill M, Ryan M, Lynch R, Mahtani KR, Clyne B. Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines. BMC Med Res Methodol 2023; 23:68. [PMID: 36966277 PMCID: PMC10039768 DOI: 10.1186/s12874-023-01895-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2023] [Accepted: 03/18/2023] [Indexed: 03/27/2023] Open
Abstract
BACKGROUND Clinical guidelines should be based on a thorough evaluation of the evidence and generally include a rating of the quality of evidence and assign a strength to recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against making strong recommendations when the certainty of the evidence is low or very low, but has identified five paradigmatic situations (e.g. life-threatening situations) where this may be justified. AIMS AND OBJECTIVES We aimed to characterize the strength of recommendations and certainty of the evidence in Irish National Clinical Guidelines using the GRADE approach. METHODS All National Clinical Guidelines from the National Clinical Effectiveness Committee (NCEC) website using the GRADE approach (fully or partially) were included. All recommendations and their corresponding certainty of the evidence, strength of recommendations and justifications were extracted. Authors classified instances of strong recommendations with low certainty evidence (referred to as discordant recommendations) into one of the five paradigmatic situations. Descriptive statistics were calculated. RESULTS From the 29 NCEC Clinical Guidelines available at the time of analysis, we identified 8 guidelines using GRADE with a total of 240 recommendations; 38 recommendations did not use the GRADE approach and were excluded. Half of the included guidelines focused on emergency situations. In the final dataset of 202 recommendations, 151 (74.7%) were classified as strong and 51 (25.3%) as conditional. Of the 151 strong recommendations, 55 (36.4%) were supported by high or moderate certainty evidence and 96 (63.6%) by low or very low certainty evidence and were considered discordant. Of these 96 discordant recommendations, 55 (73.7%) were consistent with one of the five paradigmatic situations. However, none were specifically described as such within the guidelines. CONCLUSIONS The proportion of discordant recommendations identified in this analysis was higher than some previous international studies (range of all strong recommendations being discordant 30-50%), but similar to other guidelines focused on emergency situations. The majority of discordant recommendations could be mapped to one of the five situations, but no National Clinical Guideline explicitly referenced this. Guideline developers require further guidance to enable greater transparency in the reporting of the reasons for discordant recommendations.
Collapse
Affiliation(s)
- Ming Chuen Chong
- Department of General Practice, RCSI University of Medicine and Health Sciences, Dublin, Dublin 2, Ireland
| | - Melissa K Sharp
- Department of General Practice, RCSI University of Medicine and Health Sciences, Dublin, Dublin 2, Ireland
| | - Susan M Smith
- Department of Public Health and Primary Care, School of Medicine, Trinity College Dublin, Dublin, Dublin 2, Ireland
| | - Michelle O'Neill
- Health Information and Quality Authority, George's Court, George's Lane, Dublin, Dublin 7, Ireland
| | - Máirín Ryan
- Health Information and Quality Authority, George's Court, George's Lane, Dublin, Dublin 7, Ireland
| | - Rosarie Lynch
- Department of Health, Clinical Effectiveness and Antimicrobial Resistance Unit, National Patient Safety Office, Dublin, Ireland
| | - Kamal R Mahtani
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, England
| | - Barbara Clyne
- Department of General Practice, RCSI University of Medicine and Health Sciences, Dublin, Dublin 2, Ireland.
| |
Collapse
|
11
|
Abstract
Today, every country struggles to provide adequate health care to its citizens. Globally, an average of $8.3 trillion or 10% of gross domestic product (GDP) is annually spent on health services. In 2019, the USA spent nearly 18% ($3.2 trillion) of its GDP on health care, projected to reach $6.2 trillion by 2028.
Collapse
Affiliation(s)
- Benjamin Djulbegovic
- Hematology Stewardship Program, Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA.
| | - Iztok Hozo
- Department of Mathematics, Indiana University Northwest, Gary, IN, USA
| |
Collapse
|
12
|
Djulbegovic B, Ahmed MM, Hozo I, Koletsi D, Hemkens L, Price A, Riera R, Nadanovsky P, Dos Santos APP, Melo D, Pathak R, Pacheco RL, Fontes LE, Miranda E, Nunan D. High quality (certainty) evidence changes less often than low-quality evidence, but the magnitude of effect size does not systematically differ between studies with low versus high-quality evidence. J Eval Clin Pract 2022; 28:353-362. [PMID: 35089627 PMCID: PMC9305903 DOI: 10.1111/jep.13657] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Revised: 12/22/2021] [Accepted: 01/03/2022] [Indexed: 11/29/2022]
Abstract
RATIONALE, AIMS, AND OBJECTIVES It is generally believed that evidence from low quality of evidence generate inaccurate estimates about treatment effects more often than evidence from high (certainty) quality evidence (CoE). As a result, we would expect that (a) estimates of effects of health interventions initially based on high CoE change less frequently than the effects estimated by lower CoE (b) the estimates of magnitude of effect size differ between high and low CoE. Empirical assessment of these foundational principles of evidence-based medicine has been lacking. METHODS We reviewed the Cochrane Database of Systematic Reviews from January 2016 through May 2021 for pairs of original and updated reviews for change in CoE assessments based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) method. We assessed the difference in effect sizes between the original versus updated reviews as a function of change in CoE, which we report as a ratio of odds ratio (ROR). We compared ROR generated in the studies in which CoE changed from very low/low (VL/L) to moderate/high (M/H) versus M/H to VL/L. Heterogeneity and inconsistency were assessed using the tau and I2 statistic. We also assessed the change in precision of effect estimates (by calculating the ratio of standard errors) (seR), and the absolute deviation in estimates of treatment effects (aROR). RESULTS Four hundred and nineteen pairs of reviews were included of which 414 (207 × 2) informed the CoE appraisal and 384 (192 × 2) the assessment of effect size. We found that CoE originally appraised as VL/L had 2.1 [95% confidence interval (CI): 1.19-4.12; p = 0.0091] times higher odds to be changed in the future studies than M/H CoE. However, the effect size was not different (p = 1) when CoE changed from VL/L → M/H [ROR = 1.02 (95% CI: 0.74-1.39)] compared with M/H → VL/L (ROR = 1.02 [95% CI: 0.44-2.37]). Similar overlap in aROR between the VL/L → M/H versus M/H → VL/L subgroups was observed [median (IQR): 1.12 (1.07-1.57) vs. 1.21 (1.12-2.43)]. We observed large inconsistency across ROR estimates (I2 = 99%). There was larger imprecision in treatment effects when CoE changed from VL/L → M/H (seR = 1.46) than when it changed from M/H → VL/L (seR = 0.72). CONCLUSIONS We found that low-quality evidence changes more often than high CoE. However, the effect size did not systematically differ between the studies with low versus high CoE. The finding that the effect size did not differ between low and high CoE indicate urgent need to refine current EBM critical appraisal methods.
Collapse
Affiliation(s)
- Benjamin Djulbegovic
- Department of Computational & Quantitative Medicine, Beckman Research Institute, City of Hope, Duarte, California, USA
| | - Muhammad Muneeb Ahmed
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Iztok Hozo
- Department of Mathematics, Indiana University Northwest, Gary, Indiana, USA
| | - Despina Koletsi
- Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland
| | - Lars Hemkens
- Department of Clinical Research, University of Basel, Basel Institute for Clinical Epidemiology & Biostatistics, University Hospital Basel, Basel, Switzerland.,Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA.,Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany
| | - Amy Price
- Anesthesia Informatics and Media Lab, Stanford University, Stanford, California, USA
| | - Rachel Riera
- Universidade Federal de São Paulo, Escola Paulista de Medicina, Brazil (Unifesp), São Paulo, Brazil
| | - Paulo Nadanovsky
- Department of Epidemiology and Quantitative Methods in Health, National School of Public Health, Fundação Oswaldo Cruz (FIOCRUZ) - Department of Epidemiology, Institute of Social Medicine, Universidade do Estado do Rio de Janeiro (UERJ), Rio de Janeiro, Brazil
| | - Ana Paula Pires Dos Santos
- Department of Pharmaceutical Sciences, Universidade Federal de São Paulo (Unifesp), Rio de Janeiro, Brazil
| | - Daniela Melo
- Department of Community and Preventive Dentistry, Faculty of Dentistry, Universidade do Estado do Rio de Janeiro (UERJ), Rio de Janeiro, Brazil
| | - Ranjan Pathak
- Department of Medical Oncology and Therapeutics Research, City of Hope, Duarte, California, USA
| | - Rafael Leite Pacheco
- Centro Universitário São Camilo, Researcher at the Center of Health Technology Assessment, Hospital Sirio-Libanês, São Paulo, Brazil
| | - Luis Eduardo Fontes
- Centro Universitário São Camilo, Researcher at the Center of Health Technology Assessment, Hospital Sirio-Libanês, São Paulo, Brazil.,Department of Intensive Care, and Emergency Medicine at Faculdade de Medicina de Petrópolis, in Petrópolis, Rio de Janeiro, Brazil
| | - Enderson Miranda
- Department of Intensive Care, and Emergency Medicine at Faculdade de Medicina de Petrópolis, in Petrópolis, Rio de Janeiro, Brazil
| | - David Nunan
- Department of Intensive Care, and Emergency Medicine at Faculdade de Medicina de Petrópolis, in Petrópolis, Rio de Janeiro, Brazil.,Kellogg College, University of Oxford, Oxford, UK
| |
Collapse
|
13
|
Yao L, Ahmed MM, Guyatt GH, Yan P, Hui X, Wang Q, Yang K, Tian J, Djulbegovic B. Discordant and inappropriate discordant recommendations in consensus and evidence based guidelines: empirical analysis. BMJ 2021; 375:e066045. [PMID: 34824101 PMCID: PMC8613613 DOI: 10.1136/bmj-2021-066045] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
OBJECTIVE To investigate whether alignment of strength of recommendations with quality of evidence differs in consensus based versus evidence based guidelines. DESIGN Empirical analysis. DATA SOURCE Guidelines developed by the American College of Cardiology and the American Heart Association (ACC/AHA) and the American Society of Clinical Oncology (ASCO) up to 27 March 2021. STUDY SELECTION Recommendations were clearly categorised as consensus or evidence based, were separated from the remainder of the text, and included both the quality of evidence and the strength of the recommendations. DATA EXTRACTION Paired authors independently extracted the recommendation characteristics, including type of recommendation (consensus or evidence based), grading system used for developing recommendations, strength of the recommendation, and quality of evidence. The study team also calculated the number of discordant recommendations (strong recommendations with low quality evidence) and inappropriate discordant recommendations (those that did not meet grading of recommendations assessment, development, and evaluation criteria of appropriateness). RESULTS The study included 12 ACC/AHA guidelines that generated 1434 recommendations and 69 ASCO guidelines that generated 1094 recommendations. Of the 504 ACC/AHA recommendations based on low quality evidence, 200 (40%) proved to be consensus based versus 304 (60%) evidence based; of the 404 ASCO recommendations based on low quality evidence, 292 (72%) were consensus based versus 112 (28%) that were evidence based. In both ACC/AHA and ASCO guidelines, the consensus approach yielded more discordant recommendations (ACC/AHA: odds ratio 2.1, 95% confidence interval 1.5 to 3.1; ASCO: 2.9, 1.1 to 7.8) and inappropriate discordant recommendations (ACC/AHA: 2.6, 1.7 to 3.7; ASCO: 5.1, 1.6 to 16.0) than the evidence based approach. CONCLUSION Consensus based guidelines produce more recommendations violating the evidence based medicine principles than evidence based guidelines. Ensuring appropriate alignment of quality of evidence with the strength of recommendations is key to the development of "trustworthy" guidelines.
Collapse
Affiliation(s)
- Liang Yao
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | | | - Gordon H Guyatt
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Department of Medicine, McMaster University, Hamilton, ON, Canada
| | - Peijing Yan
- Department of Epidemiology and Health Statistics, West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, Sichuan, China
| | - Xu Hui
- Evidence Based Medicine Centre, Lanzhou University, Lanzhou, Gansu, China
| | - Qi Wang
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
| | - Kehu Yang
- Evidence Based Medicine Centre, Lanzhou University, Lanzhou, Gansu, China
| | - Jinhui Tian
- Evidence Based Medicine Centre, Lanzhou University, Lanzhou, Gansu, China
| | - Benjamin Djulbegovic
- Beckman Research Institute, Department of Computational and Quantitative Medicine, City of Hope, Duarte, CA, USA
| |
Collapse
|
14
|
Schneider CR, Freeman ALJ, Spiegelhalter D, van der Linden S. The effects of quality of evidence communication on perception of public health information about COVID-19: Two randomised controlled trials. PLoS One 2021; 16:e0259048. [PMID: 34788299 PMCID: PMC8598038 DOI: 10.1371/journal.pone.0259048] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Accepted: 10/11/2021] [Indexed: 12/21/2022] Open
Abstract
Background The quality of evidence about the effectiveness of non-pharmaceutical health interventions is often low, but little is known about the effects of communicating indications of evidence quality to the public. Methods In two blinded, randomised, controlled, online experiments, US participants (total n = 2140) were shown one of several versions of an infographic illustrating the effectiveness of eye protection in reducing COVID-19 transmission. Their trust in the information, understanding, feelings of effectiveness of eye protection, and the likelihood of them adopting it were measured. Findings Compared to those given no quality cues, participants who were told the quality of the evidence on eye protection was ‘low’, rated the evidence less trustworthy (p = .001, d = 0.25), and rated it as subjectively less effective (p = .018, d = 0.19). The same effects emerged compared to those who were told the quality of the evidence was ‘high’, and in one of the two studies, those shown ‘low’ quality of evidence said they were less likely to use eye protection (p = .005, d = 0.18). Participants who were told the quality of the evidence was ‘high’ showed no statistically significant differences on these measures compared to those given no information about evidence quality. Conclusions Without quality of evidence cues, participants responded to the evidence about the public health intervention as if it was high quality and this affected their subjective perceptions of its efficacy and trust in the provided information. This raises the ethical dilemma of weighing the importance of transparently stating when the evidence base is actually low quality against evidence that providing such information can decrease trust, perception of intervention efficacy, and likelihood of adopting it.
Collapse
Affiliation(s)
- Claudia R. Schneider
- Winton Centre for Risk and Evidence Communication, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
- * E-mail:
| | - Alexandra L. J. Freeman
- Winton Centre for Risk and Evidence Communication, University of Cambridge, Cambridge, United Kingdom
| | - David Spiegelhalter
- Winton Centre for Risk and Evidence Communication, University of Cambridge, Cambridge, United Kingdom
| | - Sander van der Linden
- Winton Centre for Risk and Evidence Communication, University of Cambridge, Cambridge, United Kingdom
- Department of Psychology, University of Cambridge, Cambridge, United Kingdom
| |
Collapse
|
15
|
Li SA, Yousefi-Nooraie R, Guyatt G, Talwar G, Wang Q, Zhu Y, Hozo I, Djulbegovic B. A few panel members dominated guideline development meeting discussions: Social network analysis. J Clin Epidemiol 2021; 141:1-10. [PMID: 34555427 DOI: 10.1016/j.jclinepi.2021.09.023] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Revised: 09/05/2021] [Accepted: 09/15/2021] [Indexed: 02/07/2023]
Abstract
OBJECTIVES To identify patterns of interactions that may influence guideline panels' decision-making. STUDY DESIGN AND SETTING Social network analysis (SNA) to describe the conversation network in a guideline development meeting in United States. RESULTS We analyzed one two-day guideline panel meeting that included 20 members who developed a guideline using the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. The conversation pattern of the guideline panel indicated a well-connected network (density=0.59, clustering coefficient=0.82). GRADE topics on quality of evidence and benefits versus harms accounted for 46%; non-GRADE factors accounted for 30% of discussion. The chair, co-chair and methodologist initiated 53% and received 60% of all communications in the meeting; 42% of their communications occurred among themselves. SNA metrics (eigenvector, betweenness and closeness) indicated that these individuals also exerted highest influence on discussion, controlled information flow and were at the center of all communications. Members were more likely to continue previous discussion with the same individuals after both morning breaks (r=0.54, P<0.005; r=0.17, P=0.04), and after the last break on day 2 (r=0.44, P=0.015). CONCLUSION Non-GRADE factors such as breaks, and the members' roles, affect guideline development more than previously recognized. Collectively, the chair, co-chair and methodologist dominated the discussion.
Collapse
Affiliation(s)
- Shelly-Anne Li
- Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Ontario, Canada.
| | | | - Gordon Guyatt
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Ontario, Canada
| | - Gaurav Talwar
- Michael G DeGroote School of Medicine, McMaster University, Ontario, Canada
| | - Qi Wang
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Ontario, Canada
| | - Ying Zhu
- Department of Health Research Methods, Evidence and Impact, Faculty of Health Sciences, McMaster University, Ontario, Canada
| | - Iztok Hozo
- Department of Mathematics, Indiana University, IN, USA
| | - Benjamin Djulbegovic
- Department of Computational & Quantitative Medicine, Beckman Research Institute, City of Hope, Duarte, CA, USA
| |
Collapse
|
16
|
Aminoshariae A, Donaldson M, Horan M, Kulild JC, Baur D. Perioperative Antiplatelet and Anticoagulant Management with Endodontic Microsurgical Techniques. J Endod 2021; 47:1557-1565. [PMID: 34265324 DOI: 10.1016/j.joen.2021.07.006] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 06/23/2021] [Accepted: 07/01/2021] [Indexed: 11/28/2022]
Abstract
INTRODUCTION The purpose of this study was to review evidence-based recommendations for the safe perioperative management of patients undergoing endodontic microsurgery who are currently taking antiplatelet or anticoagulant medications. Using the PICO (Population, Intervention, Comparison, Outcome) format, the following scientific question was asked: In patients taking anticoagulant or antiplatelet agents, what is the available evidence in the management of endodontic microsurgery? METHODS MEDLINE, Scopus, Cochrane Library, and ClinicalTrials.gov databases were searched to identify current recommendations regarding the management of antiplatelet and anticoagulant medications in the context of outpatient dental surgical procedures. Additionally, the authors hand searched the bibliographies of all relevant articles, the gray literature, and textbooks. Because of the lack of clinical studies and evidence on this subject, articles and guidelines from other organizations and association position statements were included. RESULTS Because any minor surgery can become a major surgery, the treating doctor needs to best assess the risk of bleeding, especially if the surgery is anticipated to take longer than 45 minutes. Every patient should be stratified on a case-by-case basis. Consultation with the patient's physician is highly recommended. CONCLUSIONS In order to maximize the effects of these medications (to prevent thrombosis) while minimizing the potential risks (procedural hemorrhage), clinicians should be aware of the best available evidence when considering continuation or discontinuation of antiplatelet and anticoagulant agents perioperatively for endodontic microsurgery. Ideally, a joint effort from an expert panel for microsurgery would be warranted.
Collapse
Affiliation(s)
- Anita Aminoshariae
- Departments of Endodontics, Case Western Reserve University, School of Dental Medicine, Cleveland, Ohio.
| | - Mark Donaldson
- School of Pharmacy, University of Montana, Missoula, Montana; School of Dentistry, Oregon Health and Sciences University, Portland, Oregon
| | - Michael Horan
- Oral and Maxillofacial Surgery, Case Western Reserve University, School of Dental Medicine, Cleveland, Ohio
| | - James C Kulild
- UKMC Dental School, University of Missouri-Kansas City, Kansas City, Missouri
| | - Dale Baur
- Oral and Maxillofacial Surgery, Case Western Reserve University, School of Dental Medicine, Cleveland, Ohio
| |
Collapse
|