The methodological quality is insufficient in clinical practice guidelines in the context of COVID-19: systematic review

Utilization of expert opinion in infectious diseases clinical guidelines-A meta-epidemiological study

INTRODUCTION

Expert opinion is widely used in clinical guidelines. No research has ever been conducted investigating the use of expert opinion in international infectious disease guidelines. This study aimed to create an analytical map by describing the prevalence and utilization of expert opinion in infectious disease guidelines and analyzing the methodological aspects of these guidelines.

METHODS

In this meta-epidemiological study, systematic searches in PubMed and Trip Medical Database were performed to identify clinical guidelines on infectious diseases, published between January 2018 and May 2023 in English, by international organizations. Data extracted included guideline characteristics, expert opinion utilization, and methodological details. Prevalence and rationale of expert opinion use were analyzed descriptively. Methodological differences between groups were analyzed with Chi-square and Mann-Whitney U Test.

RESULTS

The analysis covered 66 guidelines with 2296 recommendations, published/endorsed by 136 organizations. Most guidelines (79%) used systematic literature searches, 42% provided search strategies, and 38% presented screening flow diagrams and conducted risk of bias assessments. 48.5% of the guidelines allowed expert opinion, most of which included expert opinion as part of the evidence hierarchy within the grading system. Guidelines allowing expert opinion, compared to those which do not, issued more recommendations per guideline (48.82 vs.19.13, p<0.001), and reported fewer screening flow diagrams (25% vs. 65%, p = 0.002), and less risk of bias assessments (19% vs.78%, p<0.001).

CONCLUSIONS

Expert opinion is utilized in half of assessed guidelines, often integrated into the evidence hierarchy within the grading system. Its utilization varies considerably in methodology, form, and terminology between guidelines. These findings highlight a pressing need for additional research and guidance, to improve and advance the standardization of infectious disease guidelines.

Capturing Emerging Experiential Knowledge for Vaccination Guidelines Through Natural Language Processing: Proof-of-Concept Study

BACKGROUND

Experience-based knowledge and value considerations of health professionals, citizens, and patients are essential to formulate public health and clinical guidelines that are relevant and applicable to medical practice. Conventional methods for incorporating such knowledge into guideline development often involve a limited number of representatives and are considered to be time-consuming. Including experiential knowledge can be crucial during rapid guidance production in response to a pandemic but it is difficult to accomplish.

OBJECTIVE

This proof-of-concept study explored the potential of artificial intelligence (AI)-based methods to capture experiential knowledge and value considerations from existing data channels to make these insights available for public health guideline development.

METHODS

We developed and examined AI-based methods in relation to the COVID-19 vaccination guideline development in the Netherlands. We analyzed Dutch messages shared between December 2020 and June 2021 on social media and on 2 databases from the Dutch National Institute for Public Health and the Environment (RIVM), where experiences and questions regarding COVID-19 vaccination are reported. First, natural language processing (NLP) filtering techniques and an initial supervised machine learning model were developed to identify this type of knowledge in a large data set. Subsequently, structural topic modeling was performed to discern thematic patterns related to experiences with COVID-19 vaccination.

RESULTS

NLP methods proved to be able to identify and analyze experience-based knowledge and value considerations in large data sets. They provide insights into a variety of experiential knowledge that is difficult to obtain otherwise for rapid guideline development. Some topics addressed by citizens, patients, and professionals can serve as direct feedback to recommendations in the guideline. For example, a topic pointed out that although travel was not considered as a reason warranting prioritization for vaccination in the national vaccination campaign, there was a considerable need for vaccines for indispensable travel, such as cross-border informal caregiving, work or study, or accessing specialized care abroad. Another example is the ambiguity regarding the definition of medical risk groups prioritized for vaccination, with many citizens not meeting the formal priority criteria while being equally at risk. Such experiential knowledge may help the early identification of problems with the guideline's application and point to frequently occurring exceptions that might initiate a revision of the guideline text.

CONCLUSIONS

This proof-of-concept study presents NLP methods as viable tools to access and use experience-based knowledge and value considerations, possibly contributing to robust, equitable, and applicable guidelines. They offer a way for guideline developers to gain insights into health professionals, citizens, and patients' experience-based knowledge, especially when conventional methods are difficult to implement. AI-based methods can thus broaden the evidence and knowledge base available for rapid guideline development and may therefore be considered as an important addition to the toolbox of pandemic preparedness.

A Fuzzy Hybrid MCDM Approach for Assessing the Emergency Department Performance during the COVID-19 Outbreak

The use of emergency departments (EDs) has increased during the COVID-19 outbreak, thereby evidencing the key role of these units in the overall response of healthcare systems to the current pandemic scenario. Nevertheless, several disruptions have emerged in the practical scenario including low throughput, overcrowding, and extended waiting times. Therefore, there is a need to develop strategies for upgrading the response of these units against the current pandemic. Given the above, this paper presents a hybrid fuzzy multicriteria decision-making model (MCDM) to evaluate the performance of EDs and create focused improvement interventions. First, the intuitionistic fuzzy analytic hierarchy process (IF-AHP) technique is used to estimate the relative priorities of criteria and sub-criteria considering uncertainty. Then, the intuitionistic fuzzy decision making trial and evaluation laboratory (IF-DEMATEL) is employed to calculate the interdependence and feedback between criteria and sub-criteria under uncertainty, Finally, the combined compromise solution (CoCoSo) is implemented to rank the EDs and detect their weaknesses to device suitable improvement plans. The aforementioned methodology was validated in three emergency centers in Turkey. The results revealed that the most important criterion in ED performance was ER facilities (14.4%), while Procedures and protocols evidenced the highest positive D + R value (18.239) among the dispatchers and is therefore deemed as the main generator within the performance network.

Evaluation of the Quality of COVID-19 Guidance Documents in Anaesthesia using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) Instrument

Background

Methods

Results

Conclusions

Possible long COVID healthcare pathways: a scoping review

Background

Individuals of all ages and with all degrees of severity of the coronavirus disease (COVID) can suffer from persisting or reappearing symptoms called long COVID. Long COVID involves various symptoms, such as shortness of breath, fatigue, or organ damage. The growing number of long COVID cases places a burden on the patients and the broader economy and, hence, has gained more weight in political decisions. This scoping review aimed to give an overview of recommendations about possible long COVID healthcare pathways and requirements regarding decision-making and communication for healthcare professionals.

Methods

A systematic search in four databases and biweekly update-hand searches were conducted. In addition to guidelines and reviews, expert opinions in consensus statements or clinical perspectives were also considered. Data were systematically extracted and subsequently narratively and graphically summarised.

Results

Fourteen references, five guidelines, four reviews, one consensus paper, and four clinical perspectives were included. The evidence recommended that most long COVID-related healthcare should be in primary care. Patients with complex symptoms should be referred to specialized long COVID outpatient assessment clinics. In contrast, patients with one dominant symptom should be directed to the respective specialist for a second assessment. Depending on the patients’ needs, further referral options include, e.g. rehabilitation or non-medical health services. Self-management and good communication between healthcare professionals and patients are crucial aspects of the long COVID management recommendations.

Conclusions

The quality of the included guidelines and reviews is limited in the methods applied due to the novelty of this topic and the associated urgency for research. Hence, an update review with more rigorous data is recommended. Furthermore, the systematic collection of real-world data on long COVID surveillance needs to be set up soon to gather further information on the duration and severity of long COVID and thereby facilitate long COVID care planning.

Systematic Identification and Critical Appraisal of Pediatric COVID-19 Guidelines Applicable in India

OBJECTIVE

To systematically identify and critically appraise the methodological quality of pediatric guidelines applicable to management of COVID-19 in India.

METHODS

Pediatric COVID-19 guidelines applicable to India, published until 30 April 2021, were identified through a systematic search across ten databases. Each was critically appraised for methodological quality using the AGREE-II tool, by at least two appraisers. Median (interquartile range) of the total score and domain-wise scores were calculated, and compared for Indian vs. foreign guidelines, updated vs. original versions of guidelines, and those developed earlier vs. later in the pandemic.

RESULTS

A total of 62 guidelines was identified. Only 8 (12.9%) were published in India. The overall AGREE-II score ranged from 4.7% to 72.8%; with median (IQR) 37.9% (29.4, 48.6). This suggested overall low(er) methodological quality. The median (IQR) domain-wise scores were as follows: Scope and Purpose 66.7% (58.3, 83.3), Stakeholder Involvement 41.7% (30.6, 83.3), Rigor of Development 23.4% (14.8, 37.5), Clarity of Presentation 59.7% (50.0, 75.0), Applicability 27.1% (18.8, 33.3), and Editorial Independence 8.3% (0.0, 45.8). This suggested diversity in quality of different aspects of the guidelines, with very low quality in the critical domain of methodological rigor. There were no statistically significant differences in the overall scores of Indian vs. foreign guidelines, updated versions vs. original versions, and those developed earlier vs. later in the pandemic.

CONCLUSION

The currently available pediatric COVID-19 guidelines have low methodological quality, adversely affecting their credibility, validity, and applicability. Urgent corrective strategies are presented for consideration.

A Comparison and Evaluation of International Guidelines on the Treatment of Severe SARS-CoV-2 Infection

BACKGROUND

When the SARS-CoV-2 pandemic began, no uniform treatment and care strategies for critically ill COVID-19 patients were yet available. National and international treatment recommendations were formulated under time pressure, initially on the basis of indirect evidence from the treatment of similar diseases. In this article, we give an overview of the content, currency, and methodological quality of the existing national and international guidelines, with special attention to the care of critically ill patients.

METHODS

Guidelines were identified by a comprehensive search, the included guidelines were assessed in standardized fashion with the AGREE II guideline assessment instrument and according to the AMWF rulebook criteria, and the core recommendations of the included and methodologically high-quality guidelines were compared.

RESULTS

Nine of the 97 guidelines that were identified fulfilled the content criteria for inclusion, and 6 of these fulfilled the qualitative criteria; these 6 guidelines still differed, however, in the topics to which they devoted the most attention, as well as in their methodological quality and currency. The treatment strategies for patients with severe respiratory failure (lung-protective ventilation strategies and rescue measures) deviated little from established standards. Uniform recommendations were made, among other things, for the administration of dexamethasone, which was recommended in all of the guidelines for patients requiring oxygen treatment, as well as for antithrombotic drug prophylaxis and for the prone positioning of ventilated patients. Many recommendations were based on insufficient evidence, and some were contradictory, e.g., those regarding antibiotic treatment or the choice between high-flow oxygen administration via nasal canula (HFNC) and noninvasive ventilation (NIV).

CONCLUSION

The consultation of multiple high-quality international guidelines and guideline recommendations shared in online portals such as MagicApp are helpful sources of information for clinicians. In view of the continuing lack of strong evidence, further research on intensive care treatments is needed (aspects of ventilation, positioning therapy, and the role of extracorporeal membrane oxygenation [ECMO]).

Recommendations for the development of Clinical Practice Guidelines

Clinical practice guidelines are statements that include recommendations intended to optimize patient care, are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options, and ensure that the best available clinical knowledge is used to provide effective and quality care. They can reduce inappropriate care and variability in clinical practice and can support the translation of new research knowledge into clinical practice. Recommendations from clinical practice guidelines can support health professionals by facilitating the decision-making process, empowering them to make more informed health care choices, clarifying which interventions should be priorities based on a favorable trade-off, and discouraging the use of those that have proven ineffective, dangerous, or wasteful. This review aims to summarize the key components of high-quality and trustworthy guidelines. Articles were retrieved from various libraries, databases, and search engines using free-text term searches adapted for different databases, and selected according to author discretion. Clinical practice guidelines in geriatrics can have a major impact on prevention, diagnosis, treatment, rehabilitation, health care, and the management of diseases and conditions, but they should only be implemented when they have high-quality, rigorous, and unbiased methodologies that consider older adult priorities and provide valid recommendations.

COVID-19 vaccine guidelines was numerous in quantity but many lack transparent reporting of methodological practices

AIM

To describe the current status of COVID-19 vaccine guidelines.

STUDY DESIGN AND SETTING

We searched databases, Google and guideline platforms to retrieve COVID-19 vaccine guidelines published between January 1, 2020 and July 8, 2021. We worked in pairs to identify the eligible guidelines and extract data of whether the methodology, funding, and conflict of interests were assessed/reported, and so on. Results were presented descriptively.

RESULTS

A total of 106 COVID-19 vaccine guidelines were included. In the first half of 2021, on average 15 guidelines were published every month. Fifty (47.2%) guidelines addressed the vaccination of people with specific medical conditions, and 18 (17.0%) guidelines focused on adverse effects after vaccination. Only 28 (26.4%) guidelines reported the methodology they used. Four (3.8%) of guidelines assessed both the quality of evidence and strength of recommendations; 42 (39.6%) and 65 (61.3%) guidelines reported their funding sources and conflict of interest, respectively. Most guidelines were published in English (n=92, 86.8%).

CONCLUSION

A high number of guidelines on COVID-19 vaccines have been published in the recent months, but most of them lack clear and transparent reporting of methodology, funding, and conflicts of interest. Rigorous methodological and reporting quality evaluation of these guidelines is needed.

Evaluation of biochemical characteristics of 183 COVID-19 patients: A retrospective study

Introduction and aim

Coronavirus disease 2019 (COVID-19), with a high mortality rate, has caught the eyes of researchers worldwide and placed a heavy burden on the health care system. Accordingly, this study aimed to evaluate the values of biochemical parameters on the outcomes of COVID-19 patients in Golestan, Iran.

Materials and methods

This retrospective study was conducted on 183 COVID-19 patients (i.e., 94 males and 89 females) between March and September 2020. The biochemical parameters and demographic data of the patients (including age, sex, urea, creatinine [Cr], lactate dehydrogenase [LDH], and creatine kinase [CK]) were obtained from electrical medical records. According to the outcome of COVID-19, the patients were categorized into two groups (i.e., death [n = 63] and survival [n = 120] groups), and the biochemical parameters and outcomes of COVID-19 were analyzed.

Results

Of the 183 patients, 120 (65.5%) had a non-severe type and recovered from COVID-19, and 63 (34.4%) developed into a critically severe type and died. The mean age of all patients was 56.5 years old. The highest mortality was observed in patients with LDH ≥280. The data obtained by the one-sample t-test showed that there were significantly higher mean values of urea, Cr, CK, and LDH in COVID-19 patients when compared to their reference intervals (P˂0.0001 for all).

Conclusions

Some biochemical parameters are effective in the evaluation of dynamic variations in COVID-19 patients. It can be concluded from the results that biochemical parameters and reinforce LDH may be useful for the evaluation of the COVID-19 outcome.

Adherence of Clinical Practice Guidelines for Pharmacologic Treatments of Hospitalized Patients With COVID-19 to Trustworthy Standards: A Systematic Review

IMPORTANCE

The COVID-19 pandemic created the need for rapid and urgent guidance for clinicians to manage COVID-19 among patients and prevent transmission.

OBJECTIVE

To appraise the quality of clinical practice guidelines (CPGs) using the National Academy of Medicine (NAM) criteria.

EVIDENCE REVIEW

A search of MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials to December 14, 2020, and a search of related articles to February 28, 2021, that included CPGs developed by societies or by government or nongovernment organizations that reported pharmacologic treatments of hospitalized patients with COVID-19. Teams of 2 reviewers independently abstracted data and assessed CPG quality using the 15-item National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) instrument.

FINDINGS

Thirty-two CPGs were included in the review. Of these, 25 (78.1%) were developed by professional societies and emanated from a single World Health Organization (WHO) region. Overall, the CPGs were of low quality. Only 7 CPGs (21.9%) reported funding sources, and 12 (37.5%) reported conflicts of interest. Only 5 CPGs (15.6%) included a methodologist, described a search strategy or study selection process, or synthesized the evidence. Although 14 CPGs (43.8%) made recommendations or suggestions for or against treatments, they infrequently rated confidence in the quality of the evidence (6 of 32 [18.8%]), described potential benefits and harms (6 of 32 [18.8%]), or graded the strength of the recommendations (5 of 32 [15.6%]). External review, patient or public perspectives, or a process for updating were rare. High-quality CPGs included a methodologist and multidisciplinary collaborations involving investigators from 2 or more WHO regions.

CONCLUSIONS AND RELEVANCE

In this review, few COVID-19 CPGs met NAM standards for trustworthy guidelines. Approaches that prioritize engagement of a methodologist and multidisciplinary collaborators from at least 2 WHO regions may lead to the production of fewer, high-quality CPGs that are poised for updates as new evidence emerges.

TRIAL REGISTRATION

PROSPERO Identifier: CRD42021245239.

The methodology of a "living" COVID-19 registry development in a clinical context

OBJECTIVE

The aim of this study was to describe an innovative methodology of a registry development, constantly updated for the scientific assessment and analysis of the health status of the population with COVID-19.

STUDY DESIGN AND SETTING

A methodological study design to develop a multi-site, Living COVID-19 Registry of COVID-19 patients admitted in Fondazione Don Gnocchi centres started in March 2020.

RESULTS

The integration of the living systematic reviews and focus group methodologies led to a development of a registry which includes 520 fields filled in for 748 COVID-19 patients recruited from 17 Fondazione Don Gnocchi centres. The result is an evidence and experience-based registry, according to the evolution of a new pathology which was not known before outbreak of March 2020 and with the aim of building knowledge to provide a better quality of care for COVID-19 patients.

CONCLUSION

A Living COVID-19 Registry is an open, living and up to date access to large-scale patient-level data sets that could help identifying important factors and modulating variable for recognising risk profiles and predicting treatment success in COVID-19 patients hospitalized. This innovative methodology might be used for other registries, to be sure which the data collected is an appropriate means of accomplishing the scientific objectives planned.

CLINICAL TRIAL REGISTRATION NUMBER

not applicable.

Methodological assessment of systematic reviews and meta-analyses on COVID-19: A meta-epidemiological study

RATIONALE, AIMS, AND OBJECTIVES

COVID-19 has caused an ongoing public health crisis. Many systematic reviews and meta-analyses have been performed to synthesize evidence for better understanding this new disease. However, some concerns have been raised about rapid COVID-19 research. This meta-epidemiological study aims to methodologically assess the current systematic reviews and meta-analyses on COVID-19.

METHODS

We searched in various databases for systematic reviews with meta-analyses published between 1 January 2020 and 31 October 2020. We extracted their basic characteristics, data analyses, evidence appraisal, and assessment of publication bias and heterogeneity.

RESULTS

We identified 295 systematic reviews on COVID-19. The median time from submission to acceptance was 33 days. Among these systematic reviews, 73.9% evaluated clinical manifestations or comorbidities of COVID-19. Stata was the most used software programme (43.39%). The odds ratio was the most used effect measure (34.24%). Moreover, 28.14% of the systematic reviews did not present evidence appraisal. Among those reporting the risk of bias results, 14.64% of studies had a high risk of bias. Egger's test was the most used method for assessing publication bias (38.31%), while 38.66% of the systematic reviews did not assess publication bias. The I² statistic was widely used for assessing heterogeneity (92.20%); many meta-analyses had high values of I² . Among the meta-analyses using the random-effects model, 75.82% did not report the methods for model implementation; among those meta-analyses reporting implementation methods, the DerSimonian-Laird method was the most used one.

CONCLUSIONS

The current systematic reviews and meta-analyses on COVID-19 might suffer from low transparency, high heterogeneity, and suboptimal statistical methods. It is recommended that future systematic reviews on COVID-19 strictly follow well-developed guidelines. Sensitivity analyses may be performed to examine how the synthesized evidence might depend on different methods for appraising evidence, assessing publication bias, and implementing meta-analysis models.

Analysis of COVID-19 Guideline Quality and Change of Recommendations: A Systematic Review

Background

Hundreds of coronavirus disease 2019 (COVID-19) clinical practice guidelines (CPGs) and expert consensus statements have been developed and published since the outbreak of the epidemic. However, these CPGs are of widely variable quality. So, this review is aimed at systematically evaluating the methodological and reporting qualities of COVID-19 CPGs, exploring factors that may influence their quality, and analyzing the change of recommendations in CPGs with evidence published.

Methods

We searched five electronic databases and five websites from 1 January to 31 December 2020 to retrieve all COVID-19 CPGs. The assessment of the methodological and reporting qualities of CPGs was performed using the AGREE II instrument and RIGHT checklist. Recommendations and evidence used to make recommendations in the CPGs regarding some treatments for COVID-19 (remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir) were also systematically assessed. And the statistical inference was performed to identify factors associated with the quality of CPGs.

Results

We included a total of 92 COVID-19 CPGs developed by 19 countries. Overall, the RIGHT checklist reporting rate of COVID-19 CPGs was 33.0%, and the AGREE II domain score was 30.4%. The overall methodological and reporting qualities of COVID-19 CPGs gradually improved during the year 2020. Factors associated with high methodological and reporting qualities included the evidence-based development process, management of conflicts of interest, and use of established rating systems to assess the quality of evidence and strength of recommendations. The recommendations of only seven (7.6%) CPGs were informed by a systematic review of evidence, and these seven CPGs have relatively high methodological and reporting qualities, in which six of them fully meet the Institute of Medicine (IOM) criteria of guidelines. Besides, a rapid advice CPG developed by the World Health Organization (WHO) of the seven CPGs got the highest overall scores in methodological (72.8%) and reporting qualities (83.8%). Many CPGs covered the same clinical questions (it refers to the clinical questions on the effectiveness of treatments of remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir in COVID-19 patients) and were published by different countries or organizations. Although randomized controlled trials and systematic reviews on the effectiveness of treatments of remdesivir, glucocorticoids, hydroxychloroquine/chloroquine, interferon, and lopinavir-ritonavir for patients with COVID-19 have been published, the recommendations on those treatments still varied greatly across COVID-19 CPGs published in different countries or regions, which may suggest that the CPGs do not make sufficient use of the latest evidence.

Conclusions

Both the methodological and reporting qualities of COVID-19 CPGs increased over time, but there is still room for further improvement. The lack of effective use of available evidence and management of conflicts of interest were the main reasons for the low quality of the CPGs. The use of formal rating systems for the quality of evidence and strength of recommendations may help to improve the quality of CPGs in the context of the COVID-19 pandemic. During the pandemic, we suggest developing a living guideline of which recommendations are supported by a systematic review for it can facilitate the timely translation of the latest research findings to clinical practice. We also suggest that CPG developers should register the guidelines in a registration platform at the beginning for it can reduce duplication development of guidelines on the same clinical question, increase the transparency of the development process, and promote cooperation among guideline developers all over the world. Since the International Practice Guideline Registry Platform has been created, developers could register guidelines prospectively and internationally on this platform.

Best practices for virtual care to support youth with chronic pain and their families: a rapid systematic review to inform health care and policy during COVID-19 and beyond

Supplemental Digital Content is Available in the Text.

Evidence-informed guidance to better leverage, implement, and select virtual care platforms for pediatric chronic pain, and identified knowledge gaps are in need of additional research.

The COVID-19 pandemic has acutely challenged health systems and catalyzed the need for widescale virtual care and digital solutions across all areas of health, including pediatric chronic pain. The objective of this rapid systematic review was to identify recommendations, guidelines, and/or best practices for using virtual care to support youth with chronic pain and their families (CRD42020184498). MEDLINE, CINAHL, Embase, APA PsychINFO, and Web of Science were searched the week of May 25, 2020, for English language peer-reviewed articles published since 2010 that (1) discussed children and adolescents aged <18 years reporting any type of chronic pain (ie, pain lasting >3 months); (2) focused on any type of virtual care (eg, telephone, telehealth, telemedicine, mHealth, eHealth, online, or digital); and (3) reported on guidelines, best practices, considerations, or recommendations for virtual care. Abstract and full text screening and data extraction were performed in duplicate. Meta-ethnography was used to synthesize concepts across articles. Of 4161 unique records screened, 16 were included addressing diverse virtual care and pediatric chronic pain conditions. Four key themes were identified: (1) opportunities to better leverage virtual care, (2) direct effective implementation of virtual care, (3) selection of virtual care platforms, and (4) gaps in need of further consideration when using virtual care to support youth with chronic pain and their families. No existing guidelines for virtual care for pediatric chronic pain were identified; however, best practices for virtual care were identified and should be used by health professionals, decision makers, and policymakers in implementing virtual care.