Methods of conduct and reporting of living systematic reviews: a protocol for a living methodological survey

Methods and guidance on conducting, reporting, publishing, and appraising living systematic reviews: a scoping review

BACKGROUND AND OBJECTIVE

The living systematic review (LSR) approach is based on ongoing surveillance of the literature and continual updating. Most currently available guidance documents address the conduct, reporting, publishing, and appraisal of systematic reviews (SRs), but are not suitable for LSRs per se and miss additional LSR-specific considerations. In this scoping review, we aim to systematically collate methodological guidance literature on how to conduct, report, publish, and appraise the quality of LSRs and identify current gaps in guidance.

METHODS

A standard scoping review methodology was used. We searched MEDLINE (Ovid), EMBASE (Ovid), and The Cochrane Library on August 28, 2021. As for searching gray literature, we looked for existing guidelines and handbooks on LSRs from organizations that conduct evidence syntheses. The screening was conducted by two authors independently in Rayyan, and data extraction was done in duplicate using a pilot-tested data extraction form in Excel. Data was extracted according to four pre-defined categories for (i) conducting, (ii) reporting, (iii) publishing, and (iv) appraising LSRs. We mapped the findings by visualizing overview tables created in Microsoft Word.

RESULTS

Of the 21 included papers, methodological guidance was found in 17 papers for conducting, in six papers for reporting, in 15 papers for publishing, and in two papers for appraising LSRs. Some of the identified key items for (i) conducting LSRs were identifying the rationale, screening tools, or re-revaluating inclusion criteria. Identified items of (ii) the original PRISMA checklist included reporting the registration and protocol, title, or synthesis methods. For (iii) publishing, there was guidance available on publication type and frequency or update trigger, and for (iv) appraising, guidance on the appropriate use of bias assessment or reporting funding of included studies was found. Our search revealed major evidence gaps, particularly for guidance on certain PRISMA items such as reporting results, discussion, support and funding, and availability of data and material of a LSR.

CONCLUSION

Important evidence gaps were identified for guidance on how to report in LSRs and appraise their quality. Our findings were applied to inform and prepare a PRISMA 2020 extension for LSR.

Pragmatic and open science-based solution to a current problem in the reporting of living systematic reviews

Living systematic reviews (LSRs) are an increasingly common approach to keeping reviews up to date, in which new relevant studies are incorporated as they become available, so as to inform healthcare policy and practice in a timely manner. While journal publishers have been exploring the publication of LSRs using different updating and publishing approaches, readers cannot currently assess if the evidence underpinning a published LSR is up to date, as neither the search details, the selection process, nor the list of identified studies is made available between the publication of updates. We describe a new method to transparently report the living evidence surveillance process that occurs between published LSR versions. We use the example of the living Cochrane Review on nirmatrelvir combined with ritonavir (Paxlovid) for preventing and treating COVID-19 to illustrate how this can work in practice. We created a publicly accessible spreadsheet on the Open Science Framework platform, linking to the living Cochrane Review, that details the search and study selection process, enabling readers to track the progress of eligible ongoing or completed studies. Further automation of the evidence surveillance process should be explored.

The life and death of living systematic reviews: a methodological survey

OBJECTIVES

The objectives of this study are to describe the characteristics of living systematic reviews (LSRs) and to understand their life cycles.

STUDY DESIGN AND SETTING

We conducted a comprehensive search up to April 2021 then selected articles and abstracted data in duplicate and independently. We undertook descriptive analyses and calculated delay in version update and delay since the last published version.

RESULTS

We included 76 eligible LSRs with a total of 279 eligible versions. The majority of LSRs was from the clinical field (70%), was COVID-19 related (63%), and had a funding source specified (62%). The median number of versions per LSR was 2 (interquartile range (IQR) 1-4; range 1-19). The median and IQR for the ratio of the actual period of update to the planned period of update was 1.12 (0.81; 1.71). Out of all reviews with a 'planned period of update' and at least one update (N = 19), eight LSRs (42%) had a period since last published version greater than 3 times the planned period of update. No LSR included a 'retirement notice' in their latest published version.

CONCLUSION

While most LSR complied with the planned period of producing updates, a substantive proportion lagged since their last update.

Extension of the PRISMA 2020 statement for living systematic reviews (LSRs): protocol

Background: While the PRISMA 2020 statement is intended to guide the reporting of original systematic reviews, updated systematic reviews, and living systematic reviews (LSRs), its explanation and elaboration document notes that additional considerations for updated systematic reviews and LSRs may need to be addressed. This paper reports the protocol for developing an extension of the PRISMA 2020 statement for LSRs. Methods: We will follow the EQUATOR Network's guidance for developing health research reporting guidelines. We will review the literature to identify possible items of the PRISMA 2020 checklist that need modification, as well as new items that need to be added. Then, we will survey representatives of different stakeholder groups for their views on the proposed modifications of the PRISMA 2020 checklist. We will summarize, present, and discuss the results of the survey in an online meeting, aiming to reach consensus on the content of the LSR extension. We will then draft the checklist, explanation and elaboration for each item, and flow diagram for the PRISMA 2020 extension. Then, we will share these initial documents with stakeholder representatives for final feedback and approval. Discussion: We anticipate that the PRISMA 2020 extension for LSRs will benefit LSR authors, editors, and peer reviewers of LSRs, as well as different users of LSRs, including guideline developers, policy makers, healthcare providers, patients, and other stakeholders.

The Australian living guidelines for the clinical care of people with COVID-19: What worked, what didn't and why, a mixed methods process evaluation

Introduction

The Australian National COVID-19 Clinical Evidence Taskforce is producing living, evidence-based, national guidelines for treatment of people with COVID-19 which are updated each week. To continually improve the process and outputs of the Taskforce, and inform future living guideline development, we undertook a concurrent process evaluation examining Taskforce activities and experience of team members and stakeholders during the first 5 months of the project.

Methods

The mixed-methods process evaluation consisted of activity and progress audits, an online survey of all Taskforce participants; and semi-structured interviews with key contributors. Data were collected through five, prospective 4-weekly timepoints (beginning first week of May 2020) and three, fortnightly retrospective timepoints (March 23, April 6 and 20). We collected and analysed quantitative and qualitative data.

Results

An updated version of the guidelines was successfully published every week during the process evaluation. The Taskforce formed in March 2020, with a nominal start date of March 23. The first version of the guideline was published two weeks later and included 10 recommendations. By August 24, in the final round of the process evaluation, the team of 11 staff, working with seven guideline panels and over 200 health decision-makers, had developed 66 recommendations addressing 58 topics. The Taskforce website had received over 200,000 page views. Satisfaction with the work of the Taskforce remained very high (>90% extremely or somewhat satisfied) throughout. Several key strengths, challenges and methods questions for the work of the Taskforce were identified.

Conclusions

In just over 5 months of activity, the National COVID-19 Clinical Evidence Taskforce published 20 weekly updates to the evidence-based national treatment guidelines for COVID-19. This process evaluation identified several factors that enabled this achievement (e.g. an extant skill base in evidence review and convening), along with challenges that needed to be overcome (e.g. managing workloads, structure and governance) and methods questions (pace of updating, and thresholds for inclusion of evidence) which may be useful considerations for other living guidelines projects. An impact evaluation is also being conducted separately to examine awareness, acceptance and use of the guidelines.

Outbreaks of publications about emerging infectious diseases: the case of SARS-CoV-2 and Zika virus

BACKGROUND

Outbreaks of infectious diseases generate outbreaks of scientific evidence. In 2016 epidemics of Zika virus emerged, and in 2020, a novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) caused a pandemic of coronavirus disease 2019 (COVID-19). We compared patterns of scientific publications for the two infections to analyse the evolution of the evidence.

METHODS

We annotated publications on Zika virus and SARS-CoV-2 that we collected using living evidence databases according to study design. We used descriptive statistics to categorise and compare study designs over time.

RESULTS

We found 2286 publications about Zika virus in 2016 and 21,990 about SARS-CoV-2 up to 24 May 2020, of which we analysed a random sample of 5294 (24%). For both infections, there were more epidemiological than laboratory science studies. Amongst epidemiological studies for both infections, case reports, case series and cross-sectional studies emerged first, cohort and case-control studies were published later. Trials were the last to emerge. The number of preprints was much higher for SARS-CoV-2 than for Zika virus.

CONCLUSIONS

Similarities in the overall pattern of publications might be generalizable, whereas differences are compatible with differences in the characteristics of a disease. Understanding how evidence accumulates during disease outbreaks helps us understand which types of public health questions we can answer and when.

Weekly updates of national living evidence-based guidelines: methods for the Australian living guidelines for care of people with COVID-19

Background and Objectives

The Australian National COVID-19 Clinical Evidence Taskforce is a consortium of 31 Australian health professional organisations developing living, evidence-based guidelines for care of people with COVID-19, which are updated weekly. This article describes the methods used to develop and maintain the guidelines.

Methods

The guidelines use the GRADE methods and are designed to meet Australian NHMRC standards. Each week, new evidence is reviewed, current recommendations are revised, and new recommendations made. These are published in MAGIC and disseminated through traditional and social media. Relevant new questions to be addressed are continually sought from stakeholders and practitioners. For prioritized questions, the evidence is actively monitored and updated. Evidence surveillance combines horizon scans and targeted searches. An evidence team appraises and synthesizes evidence and prepares evidence-to-decision frameworks to inform development of recommendations. A guidelines leadership group oversees the development of recommendations by multidisciplinary guidelines panels and is advised by a consumer panel.

Results

: The Taskforce formed in March 2020, and the first recommendations were published 2 weeks later. The guidelines have been revised and republished on a weekly basis for 24 weeks, and as of October 2020, contain over 90 treatment recommendations, suggesting that living methods are feasible in this context.

Conclusions

The Australian guidelines for care of people with COVID-19 provide an example of the feasibility of living guidelines and an opportunity to test and improve living evidence methods.