Existing guidance on reporting of consensus methodology: a systematic review to inform ACCORD guideline development

Standards for conducting and reporting consensus and recommendation documents: European Society of Cardiovascular Radiology policy from the Guidelines Committee

Cardiovascular imaging is exponentially increasing in the diagnosis, risk stratification, and therapeutic management of patients with cardiovascular disease. The European Society of Cardiovascular Radiology (ESCR) is a non-profit scientific medical society dedicated to promoting and coordinating activities in cardiovascular imaging. The purpose of this paper, written by ESCR committees and Executive board members and approved by the ESCR Executive Board and Guidelines committee, is to codify a standardized approach to creating ESCR scientific documents. Indeed, consensus development methods must be adopted to ensure transparent decision-making that optimizes national and global health and reaches a certain scientific credibility. ESCR consensus documents developed based on a rigorous methodology will improve their scientific impact on the management of patients with cardiac involvement. CRITICAL RELEVANCE STATEMENT: This document aims to codify the methodology for producing consensus documents of the ESCR. These ESCR indications will broaden the scientific quality and credibility of further publications and, consequently, the impact on the diagnostic management of patients with cardiac involvement. KEY POINTS: Cardiovascular imaging is exponentially increasing for diagnosis, risk stratification, and therapeutic management. The ESCR is committed to promoting cardiovascular imaging. A rigorous methodology for ESCR consensus documents will improve their scientific impact.

Developing reporting checklist items from systematic review findings: a roadmap and lessons to be learned from ACCORD

OBJECTIVES

To demonstrate how researchers can identify and translate reporting gaps from a systematic review into checklist items for reporting guidelines.

STUDY DESIGN AND SETTING

Good quality research reporting ensures transparency, reproducibility, and utility, facilitated by reporting guidelines. Conducting a systematic review is an essential step in the development of these guidelines. The Enhancing the QUAlity and Transparency Of health Research (EQUATOR) Network's toolkit (2010) assists researchers in this process and is due for an update to address current gaps and evolving research methods. One significant gap is the translation of systematic review findings into checklist items. Reflecting on our experience developing the ACcurate Consensus Reporting Document, we illustrate this translation process aiming to empower researchers developing reporting guidelines to address potential biases and promote transparency. We highlight the challenges faced and how they were addressed.

RESULTS

The systematic review search process was iterative, involving multiple adjustments to balance precision and sensitivity. Excessively stringent exclusion criteria may lead to missed valuable insights, especially when studies offer relevant content. An information specialist was invaluable in developing the search strategy. Key lessons learned include the necessity of maintaining flexibility and openness during data extraction, continuous adaptation based on panelist feedback, and promoting clear communication through understandable language. These principles can guide the development of future reporting guidelines and the updating of the EQUATOR toolkit, promoting transparency and robustness in research reporting.

CONCLUSION

Maintaining flexibility, capturing evolving insights, clear communication, and accommodating changes in research and technologies are key to translating systematic review findings into effective reporting checklists.

Designing Prefaculty Competencies for Diverse Learners Through a Modified Delphi Process

Importance

Faculty diversity in academic medicine may better prepare the next generation of equity-minded health care practitioners and leaders. Prefaculty development is an emerging concept to support trainees in achieving key knowledge, skills, and experiences to become successful faculty.

Objective

To outline competencies, with corresponding milestones, to support the academic career development of learners, inclusive of racial, ethnic, sexual, and gender identities minoritized in medicine.

Design, Setting, and Participants

Using a modified Delphi process, a national working group consisting of 13 members was established. The group used the published literature and listening sessions with diverse stakeholders to draft a set of competencies and milestones in July 2022. Diverse expert panelists reviewed the draft set over 2 rounds between September 2022 and January 2023. The group considered qualitative data to further refine the draft set between rounds. Consensus was reached when competencies and milestones were rated as agree or strongly agree on importance or appropriateness by 75% or greater of expert panelists after the second round. A final set of competencies and milestones was generated in February 2023. Data from round 1 were analyzed in October 2022 and data from round 2 were analyzed in January 2023.

Main Outcomes and Measures

The development of prefaculty competencies with corresponding milestones by expert panel rankings and comments.

Results

The national working group consisted of 13 members who represented diversity across racial, ethnic, and gender identities and academic and career tracks. The working group developed an initial set of 36 competencies and corresponding milestones across 12 domains. After 2 rounds, consensus among 46 expert panelists generated a final list of 32 competencies with corresponding milestones across 11 domains. A total of 26 panelists (56.5%) were women, 11 (23.9%) were Black or African American, 17 (37.0%) were Latina/o/x/e, Hispanic, or of Spanish origin, and 10 (21.7%) were White. Competency domains were divided into 2 groups: foundational (academic career choice and professional identity, mentorship, networking, financial skills, diversity and inclusion, personal effectiveness and self-efficacy, and leadership) and focused (education, community engagement, research, and clinical medicine). Consensus for inclusion or elimination of items was greater than 90% between the 2 rounds.

Conclusions and Relevance

There was consensus among the working group and expert panelists regarding the importance and appropriateness of the competencies and milestones for diverse trainees to successfully obtain faculty positions. Institutions and national organizations can use these competencies as a framework to develop curricula that support diverse learners' career development toward academia.

Consensus methods in patellofemoral pain: how rigorous are they? A scoping review

OBJECTIVE

Clinicians treating patients with patellofemoral pain (PFP) rely on consensus statements to make the best practice recommendations in the absence of definitive evidence on how to manage PFP. However, the methods used to generate and assess agreement for these recommendations have not been examined. Our objective was to map the methods used to generate consensus-based recommendations for PFP and apply four novel questions to assess the rigour of consensus development.

DESIGN

Scoping review.

DATA SOURCES

We searched Medline, SPORTDiscus, CINAHL and Embase from inception to May 2022 to identify consensus-derived statements or practice guidelines on PFP. The Joanna Briggs Institute Manual for Evidence Synthesis was followed to map the existing evidence. We measured the consensus methods based on four sets of questions addressing the panel composition, application of the consensus method chosen, agreement process and the use of evidence mapping.

ELIGIBILITY CRITERIA

All consensus statements or clinical guidelines on PFP were considered.

RESULTS

Twenty-two PFP consensus statements were identified. Panel composition: 3 of the 22 (14%) consensus groups reported the panellists' experience, 2 (9%) defined a desired level of expertise, 10 (45%) reported panellist sex and only 2 (9%) included a patient. Consensus method: 7 of 22 (32%) reported using an established method of consensus measurement/development. Agreement process: 10 of 22 (45%) reported their consensus threshold and 2 (9%) acknowledged dissenting opinions among the panel. Evidence mapping: 6 of 22 (27%) reported using systematic methods to identify relevant evidence gaps.

CONCLUSIONS

PFP consensus panels have lacked diversity and excluded key partners including patients. Consensus statements on PFP frequently fail to use recognised consensus methods, rarely describe how 'agreement' was defined or measured and often neglect to use systematic methods to identify evidence gaps.

Panel stacking is a threat to consensus statement validity

Consensus statements can be very influential in medicine and public health. Some of these statements use systematic evidence synthesis but others fail on this front. Many consensus statements use panels of experts to deduce perceived consensus through Delphi processes. We argue that stacking of panel members toward one particular position or narrative is a major threat, especially in absence of systematic evidence review. Stacking may involve financial conflicts of interest, but nonfinancial conflicts of strong advocacy can also cause major bias. Given their emerging importance, we describe here how such consensus statements may be misleading, by analyzing in depth a recent high-impact Delphi consensus statement on COVID-19 recommendations as a case example. We demonstrate that many of the selected panel members and at least 35% of the core panel members had advocated toward COVID-19 elimination (Zero-COVID) during the pandemic and were leading members of aggressive advocacy groups. These advocacy conflicts were not declared in the Delphi consensus publication, with rare exceptions. Therefore, we propose that consensus statements should always require rigorous evidence synthesis and maximal transparency on potential biases toward advocacy or lobbyist groups to be valid. While advocacy can have many important functions, its biased impact on consensus panels should be carefully avoided.

ACcurate COnsensus Reporting Document (ACCORD) explanation and elaboration: Guidance and examples to support reporting consensus methods

BACKGROUND

When research evidence is limited, inconsistent, or absent, healthcare decisions and policies need to be based on consensus amongst interested stakeholders. In these processes, the knowledge, experience, and expertise of health professionals, researchers, policymakers, and the public are systematically collected and synthesised to reach agreed clinical recommendations and/or priorities. However, despite the influence of consensus exercises, the methods used to achieve agreement are often poorly reported. The ACCORD (ACcurate COnsensus Reporting Document) guideline was developed to help report any consensus methods used in biomedical research, regardless of the health field, techniques used, or application. This explanatory document facilitates the use of the ACCORD checklist.

METHODS AND FINDINGS

This paper was built collaboratively based on classic and contemporary literature on consensus methods and publications reporting their use. For each ACCORD checklist item, this explanation and elaboration document unpacks the pieces of information that should be reported and provides a rationale on why it is essential to describe them in detail. Furthermore, this document offers a glossary of terms used in consensus exercises to clarify the meaning of common terms used across consensus methods, to promote uniformity, and to support understanding for consumers who read consensus statements, position statements, or clinical practice guidelines. The items are followed by examples of reporting items from the ACCORD guideline, in text, tables and figures.

CONCLUSIONS

The ACCORD materials - including the reporting guideline and this explanation and elaboration document - can be used by anyone reporting a consensus exercise used in the context of health research. As a reporting guideline, ACCORD helps researchers to be transparent about the materials, resources (both human and financial), and procedures used in their investigations so readers can judge the trustworthiness and applicability of their results/recommendations.

ACCORD (ACcurate COnsensus Reporting Document): A reporting guideline for consensus methods in biomedicine developed via a modified Delphi

BACKGROUND

In biomedical research, it is often desirable to seek consensus among individuals who have differing perspectives and experience. This is important when evidence is emerging, inconsistent, limited, or absent. Even when research evidence is abundant, clinical recommendations, policy decisions, and priority-setting may still require agreement from multiple, sometimes ideologically opposed parties. Despite their prominence and influence on key decisions, consensus methods are often poorly reported. Our aim was to develop the first reporting guideline dedicated to and applicable to all consensus methods used in biomedical research regardless of the objective of the consensus process, called ACCORD (ACcurate COnsensus Reporting Document).

METHODS AND FINDINGS

We followed methodology recommended by the EQUATOR Network for the development of reporting guidelines: a systematic review was followed by a Delphi process and meetings to finalize the ACCORD checklist. The preliminary checklist was drawn from the systematic review of existing literature on the quality of reporting of consensus methods and suggestions from the Steering Committee. A Delphi panel (n = 72) was recruited with representation from 6 continents and a broad range of experience, including clinical, research, policy, and patient perspectives. The 3 rounds of the Delphi process were completed by 58, 54, and 51 panelists. The preliminary checklist of 56 items was refined to a final checklist of 35 items relating to the article title (n = 1), introduction (n = 3), methods (n = 21), results (n = 5), discussion (n = 2), and other information (n = 3).

CONCLUSIONS

The ACCORD checklist is the first reporting guideline applicable to all consensus-based studies. It will support authors in writing accurate, detailed manuscripts, thereby improving the completeness and transparency of reporting and providing readers with clarity regarding the methods used to reach agreement. Furthermore, the checklist will make the rigor of the consensus methods used to guide the recommendations clear for readers. Reporting consensus studies with greater clarity and transparency may enhance trust in the recommendations made by consensus panels.

Perioperative hypothermia prevention: development of simple principles and practice recommendations using a multidisciplinary consensus-based approach

OBJECTIVES

To develop a consensus on evidence-based principles and recommendations for perioperative hypothermia prevention in the Australian context.

DESIGN

This study was informed by CAN-IMPLEMENT using the ADAPTE process: (1) formation of a multidisciplinary development team; (2) systematic search process identifying existing guidance for perioperative hypothermia prevention; (3) appraisal using the AGREE II Rigor of Development domain; (4) extraction of recommendations from guidelines meeting a quality threshold using the AGREE-REX tool; (5) review of draft principles and recommendations by multidisciplinary clinicians nationally and (6) subsequent round of discussion, drafting, reflection and revision by the original panel member team.

SETTING

Australian perioperative departments.

PARTICIPANTS

Registered nurses, anaesthetists, surgeons and anaesthetic allied health practitioners.

RESULTS

A total of 23 papers (12 guidelines, 6 evidence summaries, 3 standards, 1 best practice sheet and 1 evidence-based bundle) formed the evidence base. After evidence synthesis and development of draft recommendations, 219 perioperative clinicians provided feedback. Following refinement, three simple principles for perioperative hypothermia prevention were developed with supporting practice recommendations: (1) actively monitor core temperature for all patients at all times; (2) warm actively to keep body temperature above 36°C and patients comfortable and (3) minimise exposure to cold at all stages of perioperative care.

CONCLUSION

This consensus process has generated principles and practice recommendations for hypothermia prevention that are ready for implementation with local adaptation. Further evaluation will be undertaken in a large-scale implementation trial across Australian hospitals.

International standardization and optimization group for intersphincteric resection (ISOG-ISR): modified Delphi consensus on anatomy, definition, indication, surgical technique, specimen description and functional outcome

AIM

Intersphincteric resection (ISR) is an oncologically complex operation for very low-lying rectal cancers. Yet, definition, anatomical description, operative indications and operative approaches to ISR are not standardized. The aim of this study was to standardize the definition of ISR by reaching international consensus from the experts in the field. This standardization will allow meaningful comparison in the literature in the future.

METHOD

A modified Delphi approach with three rounds of questionnaire was adopted. A total of 29 international experts from 11 countries were recruited for this study. Six domains with a total of 37 statements were examined, including anatomical definition; definition of intersphincteric dissection, intersphincteric resection (ISR) and ultra-low anterior resection (uLAR); indication for ISR; surgical technique of ISR; specimen description of ISR; and functional outcome assessment protocol.

RESULTS

Three rounds of questionnaire were performed (response rate 100%, 89.6%, 89.6%). Agreement (≥80%) reached standardization on 36 statements.

CONCLUSION

This study provides an international expert consensus-based definition and standardization of ISR. This is the first study standardizing terminology and definition of deep pelvis/anal canal anatomy from a surgical point of view. Intersphincteric dissection, ISR and uLAR were specifically defined for precise surgical description. Indication for ISR was determined by the rectal tumour's maximal radial infiltration (T stage) below the levator ani. A new surgical definition of T3isp was reached by consensus to define T3 low rectal tumours infiltrating the intersphincteric plane. A practical flowchart for surgical indication for uLAR/ISR/abdominoperineal resection was developed. A standardized ISR surgical technique and functional outcome assessment protocol was defined.

An international consensus definition for contextual factors: findings from a nominal group technique

Objective

Emerging literature suggests contextual factors are important components of therapeutic encounters and may substantially influence clinical outcomes of a treatment intervention. At present, a single consensus definition of contextual factors, which is universal across all health-related conditions is lacking. The objective of this study was to create a consensus definition of contextual factors to better refine this concept for clinicians and researchers.

Design

The study used a multi-stage virtual Nominal Group Technique (vNGT) to create and rank contextual factor definitions. Nominal group techniques are a form of consensus-based research, and are beneficial for identifying problems, exploring solutions and establishing priorities.

Setting

International.

Main outcome measures

The initial stages of the vNGT resulted in the creation of 14 independent contextual factor definitions. After a prolonged discussion period, the initial definitions were heavily modified, and 12 final definitions were rank ordered by the vNGT participants from first to last.

Participants

The 10 international vNGT participants had a variety of clinical backgrounds and research specializations and were all specialists in contextual factors research.

Results

A sixth round was used to identify a final consensus, which reflected the complexity of contextual factors and included three primary domains: (1) an overall definition; (2) qualifiers that serve as examples of the key areas of the definition; and (3) how contextual factors may influence clinical outcomes.

Conclusion

Our consensus definition of contextual factors seeks to improve the understanding and communication between clinicians and researchers. These are especially important in recognizing their potential role in moderating and/or mediating clinical outcomes.