Protocol for a Delphi consensus exercise to identify a core set of criteria for selecting health related outcome measures (HROM) to be used in primary health care

Development of quality assessment tool for systematic reviews and meta-analyses of real-world studies: a Delphi consensus survey

The increasing adoption of real-world studies in healthcare for decision making and planning has further necessitated the need for a specific quality assessment tool for evidence synthesis. This study aimed to develop a quality assessment tool for systematic reviews (SR) and meta-analysis (MA) involving real-world studies (QATSM-RWS) using a formal consensus method. Based on scoping review, the authors identified a list of items for possible inclusion in the quality assessment tool. A Delphi survey was formulated based on the identified items. A total of 89 experts, purposively recruited, with research experience in real-world data were invited to participate in the first round of Delphi survey. The participants who responded in the first Delphi round were invited to participate (n = 15) in the phrasing of the items. Strong level of agreement was found on the proposed list of items after the first round of Delphi. A rate of agreement ≥ 0.70 was used to define which items to keep in the tool. A list of 14 items emerged as suitable for QATSM-RWS. The items were structured under five domains: introduction, methods, results, discussions, and others. All participants agreed with the proposed phrasing of the items. This is the first study that has developed a specific tool that can be used to appraise the quality of SR and MA involving real-world studies. QATSM-RWS may be used by policymakers, clinicians, and practitioners when evaluating and generating real-world evidence. This tool is now undergoing validation process.

Bridging the gap from medical to psychological safety assessment: consensus study in a digital mental health context

BACKGROUND

Digital Mental Health Interventions (DMHIs) that meet the definition of a medical device are regulated by the Medicines and Healthcare products Regulatory Agency (MHRA) in the UK. The MHRA uses procedures that were originally developed for pharmaceuticals to assess the safety of DMHIs. There is recognition that this may not be ideal, as is evident by an ongoing consultation for reform led by the MHRA and the National Institute for Health and Care Excellence.

AIMS

The aim of this study was to generate an experts' consensus on how the medical regulatory method used for assessing safety could best be adapted for DMHIs.

METHOD

An online Delphi study containing three rounds was conducted with an international panel of 20 experts with experience/knowledge in the field of UK digital mental health.

RESULTS

Sixty-four items were generated, of which 41 achieved consensus (64%). Consensus emerged around ten recommendations, falling into five main themes: Enhancing the quality of adverse events data in DMHIs; Re-defining serious adverse events for DMHIs; Reassessing short-term symptom deterioration in psychological interventions as a therapeutic risk; Maximising the benefit of the Yellow Card Scheme; and Developing a harmonised approach for assessing the safety of psychological interventions in general.

CONCLUSION

The implementation of the recommendations provided by this consensus could improve the assessment of safety of DMHIs, making them more effective in detecting and mitigating risk.

Core Descriptor Sets for Rectal Prolapse Outcomes Research Using a Modified Delphi Consensus

BACKGROUND

There is wide variation in prolapse care.

OBJECTIVE

To determine core descriptor sets for rectal prolapse to enhance outcomes research.

DESIGN

Descriptors for patients undergoing rectal prolapse surgery were generated through a systematic review and expert opinion. Stakeholders were recruited internationally via listserv and social media. Experts were encouraged to consider the minimum descriptors that could be considered during clinical care, and descriptors were grouped into core descriptor sets. Consensus was defined as greater than 70% agreement.

SETTING

A 3-round Delphi process using a 9-point Likert scale based on expert results was distributed via survey. The final interactive meeting used a polling platform.

PARTICIPANTS

The Pelvic Floor Disorders Consortium interdisciplinary group convened to advance the clinical care of pelvic floor disorders.

MAIN OUTCOME MEASURES

To achieve expert consensus for core descriptor sets for rectal prolapse using a modified Delphi method.

RESULTS

A total of 206 providers participated, with survey response rates of 82% and 88%, respectively. Responders were from North America (56%), Europe (29%), and Latin America, Asia, Australia, New Zealand, and Africa (15%). Ninety-one percent of participants identified as colorectal surgeons and 80% reported >5 years of experience (35% reported >15 years). Fifty-seven attendees participated in the final meeting and voted on core descriptor sets. Ninety-three percent of participants agreed that descriptors such as age, BMI, frailty, nutrition, and the American Society of Anesthesiology score correlated to physiologic status. One hundred percent of participants agreed to include baseline bowel function. One hundred percent of participants reported willingness to complete a synoptic operative report. Follow-up intervals 1, 3, and 5 years after surgery (76%) with a collection of recurrence and functional outcomes at those time periods reached an agreement.

LIMITATIONS

Individual bias, self-identification of experts, and paucity of knowledge related to rectal prolapse.

CONCLUSIONS

This represents the first steps toward international consensus to unify language and data collection processes for rectal prolapse. See Video Abstract .

CONJUNTOS DE DESCRIPTORES BSICOS PARA LA INVESTIGACIN DE RESULTADOS DE PROLAPSO RECTAL MEDIANTE UN CONSENSO DELPHI MODIFICADO

ANTECEDENTES:Existe una amplia variación en la atención del prolapso.OBJETIVO:Determinar conjuntos de descriptores básicos para el prolapso rectal para mejorar los resultados de la investigación.DISEÑO:Los descriptores para pacientes sometidos a cirugía de prolapso rectal se generaron a través de una revisión sistemática y la opinión de expertos. Las partes interesadas fueron reclutadas internacionalmente a través de listas de servicio y redes sociales. Se animó a los expertos a considerar los descriptores mínimos que podrían considerarse durante la atención clínica, y los descriptores se agruparon en conjuntos de descriptores básicos. El consenso se definió como > 70% de acuerdo.AJUSTE:Se distribuyó mediante encuesta un proceso Delphi de tres rondas que utiliza una escala Likert de 9 puntos basada en resultados de expertos. La reunión interactiva final utilizó una plataforma de votación.PARTICIPANTES:El grupo interdisciplinario del Consorcio de Trastornos del Suelo Pélvico se reunió para avanzar en la atención clínica de los trastornos del suelo pélvico.MEDIDAS PRINCIPALES DE RESULTADOS:Lograr el consenso de expertos para los conjuntos de descriptores básicos para el prolapso rectal utilizando un método Delphi modificado.RESULTADOS:Participaron 206 proveedores con tasas de respuesta a la encuesta del 82% y 88% respectivamente. Los encuestados procedían de América del Norte (56%), Europa (29%) y América Latina, Asia, Australia, Nueva Zelanda y África (15%). El noventa y uno por ciento se identificó como cirujanos colorrectales y el 80% reportó más de 5 años de experiencia (35% > 15 años). Cincuenta y siete asistentes participaron en la reunión final y votaron sobre conjuntos de descriptores básicos. El noventa y tres por ciento estuvo de acuerdo en que descriptores como edad, índice de masa corporal, fragilidad, nutrición y puntuación de la Sociedad Estadounidense de Anestesiología se correlacionaban con el estado fisiológico. El cien por ciento estuvo de acuerdo en incluir la función intestinal inicial. El 100% refirió disposición para realizar un informe operativo sinóptico. Los intervalos de seguimiento 1,3,5 años después de la cirugía (76%) con un conjunto de recurrencias y los resultados funcionales en esos períodos de tiempo coincidieron.LIMITACIONES:Sesgo individual, autoidentificación de los expertos y escasez de conocimientos relacionados con el prolapso rectal.CONCLUSIONES:Esto representa los primeros pasos hacia un consenso internacional para unificar el lenguaje y los procesos de recolección de datos para el prolapso rectal. (Traducción-Yesenia Rojas-Khalil ).

Building consensus on common features and interoperability use cases for community health information systems: a Delphi study

INTRODUCTION

Information systems for community health have become increasingly sophisticated and evidence-based in the last decade and they are now the most widely used health information systems in many low-income and middle-income countries. This study aimed to establish consensus regarding key features and interoperability priorities for community health information systems (CHISs).

METHODS

A Delphi study was conducted among a systematically selected panel of CHIS experts. This impressive pool of experts represented a range of leading global health institutions, with gender and regional balance as well as diversity in their areas of expertise. Through five rounds of iterative surveys and follow-up interviews, the experts established a high degree of consensus. We supplemented the Delphi study findings with a series of focus group discussions with 10 community health worker (CHW) leaders.

RESULTS

CHISs today are expected to adapt to a wide range of local contextual requirements and to support and improve care delivery. While once associated with a single role type (CHWs), these systems are now expected to engage other end users, including patients, supervisors, clinicians and data managers. Of 30 WHO-classified digital health interventions for care providers, experts identified 23 (77%) as being important for CHISs. Case management and care coordination features accounted for more than one-third (14 of 37, 38%) of the core features expected of CHISs today, a higher proportion than any other category. The highest priority use cases for interoperability include CHIS to health management information system monthly reporting and CHIS to electronic medical record referrals.

CONCLUSION

CHISs today are expected to be feature-rich, to support a range of user roles in community health systems, and to be highly adaptable to local contextual requirements. Future interoperability efforts, such as CHISs in general, are expected not only to move data efficiently but to strengthen community health systems in ways that measurably improve care.

Third national surgical consensus conference of the Italian Association of Breast Surgeons (ANISC) on management after neoadjuvant chemotherapy: The difficulty in reaching a consensus

INTRODUCTION

Neoadjuvant chemotherapy (NAC) has a profound impact on surgical management of breast cancer. For this reason, the Italian Association of Breast Surgeons (ANISC) promoted the third national Consensus Conference on this subject, open to multidisciplinary specialists.

MATERIALS AND METHODS

The Consensus Conference was held on-line in November 2022, and after an introductory session with five core-team experts, participants were asked to vote on eleven controversial issues, while results were collected in real-time with a polling system.

RESULTS

A total of 164 dedicated specialists from 74 Breast Centers participated. Consensus was reached for only three of the eleven issues, including: 1) the indication to assess the response with Magnetic Resonance Imaging (79 %); 2) the need to re-assess the biological factors of the residual tumor if present (96 %); 3) the possibility of omitting a formal axillary node dissection for cN1 patients if a pathologic Complete Response (pCR) was confirmed with analysis of one or more sentinel lymph nodes (82 %). The majority voted in favor of mapping both the breast and nodal lesions pre-NAC (59 %), and against the omission of sentinel lymph node biopsy in cN0 patients in the case of pathologic or clinical Complete Response (69 %). In cases of cT3/cN1+ tumors with pCR, only 8 % of participants considered appropriate the omission of Post-Mastectomy Radiation Therapy.

CONCLUSION

There is still a wide variability in surgical approaches after NAC in the "real world". As NAC is increasingly used, multidisciplinary teams should be attuned to conforming their procedures to the rapid advances in this field.

Gaining Consensus Around Patient Risk Groups and Prognostic Profiles to Guide CMV Management Among Patients With Allogeneic Hematopoietic Stem Cell Transplant: Insights From a Delphi Panel With Hematopoietic Stem Cell Transplant Experts

INTRODUCTION

This study aimed to characterize patient risk groups and prognostic profiles to optimize clinical decision-making and guide appropriate medical cytomegalovirus (CMV) management among patients with allogeneic hematopoietic stem cell transplant (HSCT).

METHODS

Between 8/2021 and 2/2022, a 3-round modified Delphi study was conducted to generate consensus among 10 international experts in HSCT and infectious diseases. Experts were asked about treatment and prognoses for patients in 7 distinct clinical scenarios. Furthermore, experts were asked to risk-stratify patients by pre-/post-transplant characteristics. Consensus around opting for/against a treatment was observed if ≥75% or <25% of experts reported ≥50% likelihood to recommend or if treatments were ranked inside/outside the top 2 options and ≥75% of experts were within 1 SD of mean ranks.

RESULTS

Experts agreed on several unmet needs in CMV disease management post-HSCT, particularly avoidance of treatment-limiting toxicities with conventional CMV therapy and the emergence of both refractory and drug-resistant treatment failures. Experts considered CMV viral load, resistance profile, and route of administration as critical to treatment selection. For newer CMV therapeutic options, experts listed a lack of long-term use data, concerns over potential resistance, high cost, and limited availability as challenges restricting adoption and successful patient management.

CONCLUSIONS

Experts achieved consensus around patient risk stratifications and factors influencing therapeutic options. Recommendations emerging from this Delphi study may support practicing physicians when confronted with challenging CMV scenarios in patients with HSCT.

Development of entrustable professional activities for emergency medicine medical education fellowships: A modified Delphi study

Background

It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows.

Methods

From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics.

Results

Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration.

Conclusions

We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd.

The definition of treatment resistance in anxiety disorders: a Delphi method-based consensus guideline

Anxiety disorders are very prevalent and often persistent mental disorders, with a considerable rate of treatment resistance which requires regulatory clinical trials of innovative therapeutic interventions. However, an explicit definition of treatment-resistant anxiety disorders (TR-AD) informing such trials is currently lacking. We used a Delphi method-based consensus approach to provide internationally agreed, consistent and clinically useful operational criteria for TR-AD in adults. Following a summary of the current state of knowledge based on international guidelines and an available systematic review, a survey of free-text responses to a 29-item questionnaire on relevant aspects of TR-AD, and an online consensus meeting, a panel of 36 multidisciplinary international experts and stakeholders voted anonymously on written statements in three survey rounds. Consensus was defined as ≥75% of the panel agreeing with a statement. The panel agreed on a set of 14 recommendations for the definition of TR-AD, providing detailed operational criteria for resistance to pharmacological and/or psychotherapeutic treatment, as well as a potential staging model. The panel also evaluated further aspects regarding epidemiological subgroups, comorbidities and biographical factors, the terminology of TR-AD vs. "difficult-to-treat" anxiety disorders, preferences and attitudes of persons with these disorders, and future research directions. This Delphi method-based consensus on operational criteria for TR-AD is expected to serve as a systematic, consistent and practical clinical guideline to aid in designing future mechanistic studies and facilitate clinical trials for regulatory purposes. This effort could ultimately lead to the development of more effective evidence-based stepped-care treatment algorithms for patients with anxiety disorders.

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Neonatal sepsis is a serious public health problem; however, there is substantial heterogeneity in the outcomes measured and reported in research evaluating the effectiveness of the treatments. Therefore, we aim to develop a Core Outcome Set (COS) for studies evaluating the effectiveness of treatments for neonatal sepsis. Since a systematic review of key outcomes from randomised trials of therapeutic interventions in neonatal sepsis was published recently, we will complement this with a qualitative systematic review of the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers. We will interpret the outcomes of both studies using a previously established framework. Stakeholders across three different groups i.e., (1) researchers, (2) healthcare providers, and (3) patients' parents/family members and parent representatives will rate the importance of the outcomes in an online Real-Time Delphi Survey. Afterwards, consensus meetings will be held to agree on the final COS through online discussions with key stakeholders. This COS is expected to minimize outcome heterogeneity in measurements and publications, improve comparability and synthesis, and decrease research waste.

Study on a job competence evaluation system for resident physicians (including integrated postgraduates) receiving standardized training

BACKGROUND

Standardized training for resident physicians is the primary form of postgraduate medical education, and it plays a pivotal role in healthcare safety and industry stability. Currently, it has garnered significant attention from healthcare institutions.

METHODS

By conducting a comprehensive literature review and a Delphi consultation in June 2022 for which 40 experts in clinical medicine, public health, and other related fields in China were invited. The indicators were adjusted according to the results of the consultation, and the final indicator weights were determined through an analytic hierarchy process.

RESULTS

The response rate was 100%, and the expert authority coefficient was 0.879. The consistency among the experts on the tertiary indicators, as measured by Kendall's W, was 0.675 (χ2 = 42.516, p < 0.001). Based on the results of the expert consultation, a job competence evaluation system for resident physicians (including integrated postgraduates) receiving standardized training was established, which included 6 primary indicators, 18 secondary indicators, and 116 tertiary indicators. The weights for the primary indicators, namely professional quality, skills and knowledge, patient care, communication and collaboration, teaching skills, and lifelong learning, were 0.313, 0.248, 0.181, 0.083, 0.066, and 0.110, respectively. The top three secondary indicators in terms of combined weights were clinical skills (0.122), professional ethics (0.120), and professional dedication (0.109). The three tertiary indicators with the highest scores were "maintains collaboration with peers and colleagues in patient treatment," "has clinical thinking skills, makes diagnosis and treatment decisions based on analysis of evidence, and has the ability to administer suitable treatments," and "abides by laws and discipline and refuses to seek personal gains in medical practice"; their combined weights were 0.035, 0.028, and 0.027, respectively.

CONCLUSION

This study has established a concrete, objective, and quantifiable competency assessment index system for standardized training of clinical resident physicians (including integrated postgraduates). This system provides a foundation for the quantitative evaluation of the competency of clinical resident physicians (including integrated postgraduates) undergoing standardized training.

Career Needs Assessment for Early Career Academic Surgeons Using a Modified Accelerated Delphi Process

INTRODUCTION

Over the past 2 decades, physicians' wellbeing has become a topic of interest. It is currently unclear what the current needs are of early career academic surgeons (ECAS).

METHODS

Consensus statements on academic needs were developed during a Delphi process, including all presenters from the previous European Surgical Association (ESA) meetings (2018-2022). The Delphi involved (1) a literature review, (2) Delphi form generation, and (3) an accelerated Delphi process. The Delphi form was generated by a steering group that discussed findings identified within the literature. The modified accelerated e-consensus approach included 3 rounds over a 4-week period. Consensus was defined as >80% agreement in any round.

RESULTS

Forty respondents completed all 3 rounds of the Delphi. Median age was 37 years (interquartile range 5), and 53% were female. Majority were consultant/attending (52.5%), followed by PhD (22.5%), fellowship (15%), and residency (10%). ECAS was defined as a surgeon in 'development' years of clinical and academic practice relative to their career goals (87.9% agreement). Access to split academic and clinical contracts is desirable (87.5%). Consensus on the factors contributing to ECAS underperformance included: burnout (94.6%), lack of funding (80%), lack of mentorship (80%), and excessive clinical commitments (80%). Desirable factors to support ECAS development included: access to e-learning (90.9%), face-to-face networking opportunities (95%), support for research team development (100%), and specific formal mentorship (93.9%).

CONCLUSION

The evolving role and responsibilities of ECAS require increasing strategic support, mentorship, and guidance on structured career planning. This will facilitate workforce sustainability in academic surgery in the future.

Toward Shared Decision-Making in Degenerative Cervical Myelopathy: Protocol for a Mixed Methods Study

BACKGROUND

Health care decisions are a critical determinant in the evolution of chronic illness. In shared decision-making (SDM), patients and clinicians work collaboratively to reach evidence-based health decisions that align with individual circumstances, values, and preferences. This personalized approach to clinical care likely has substantial benefits in the oversight of degenerative cervical myelopathy (DCM), a type of nontraumatic spinal cord injury. Its chronicity, heterogeneous clinical presentation, complex management, and variable disease course engenders an imperative for a patient-centric approach that accounts for each patient's unique needs and priorities. Inadequate patient knowledge about the condition and an incomplete understanding of the critical decision points that arise during the course of care currently hinder the fruitful participation of health care providers and patients in SDM. This study protocol presents the rationale for deploying SDM for DCM and delineates the groundwork required to achieve this.

OBJECTIVE

The study's primary outcome is the development of a comprehensive checklist to be implemented upon diagnosis that provides patients with essential information necessary to support their informed decision-making. This is known as a core information set (CIS). The secondary outcome is the creation of a detailed process map that provides a diagrammatic representation of the global care workflows and cognitive processes involved in DCM care. Characterizing the critical decision points along a patient's journey will allow for an effective exploration of SDM tools for routine clinical practice to enhance patient-centered care and improve clinical outcomes.

METHODS

Both CISs and process maps are coproduced iteratively through a collaborative process involving the input and consensus of key stakeholders. This will be facilitated by Myelopathy.org, a global DCM charity, through its Research Objectives and Common Data Elements for Degenerative Cervical Myelopathy community. To develop the CIS, a 3-round, web-based Delphi process will be used, starting with a baseline list of information items derived from a recent scoping review of educational materials in DCM, patient interviews, and a qualitative survey of professionals. A priori criteria for achieving consensus are specified. The process map will be developed iteratively using semistructured interviews with patients and professionals and validated by key stakeholders.

RESULTS

Recruitment for the Delphi consensus study began in April 2023. The pilot-testing of process map interview participants started simultaneously, with the formulation of an initial baseline map underway.

CONCLUSIONS

This protocol marks the first attempt to provide a starting point for investigating SDM in DCM. The primary work centers on developing an educational tool for use in diagnosis to enable enhanced onward decision-making. The wider objective is to aid stakeholders in developing SDM tools by identifying critical decision junctures in DCM care. Through these approaches, we aim to provide an exhaustive launchpad for formulating SDM tools in the wider DCM community.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/46809.

Developing a Core Outcome Set for Prognostic Research in Palliative Cancer Care: Protocol for a Mixed Methods Study

BACKGROUND

Studies exploring the impact of receiving end-of-life prognoses in patients with advanced cancer use a variety of different measures to evaluate the outcomes, and thus report often conflicting findings. The standardization of outcomes reported in studies of prognostication in palliative cancer care could enable uniform assessment and reporting, as well as intertrial comparisons. A core outcome set promotes consistency in outcome selection and reporting among studies within a particular population. We aim to develop a set of core outcomes to be used to measure the impact of end-of-life prognostication in palliative cancer care.

OBJECTIVE

This protocol outlines the proposed methodology to develop a core outcome set for measuring the impact of end-of-life prognostication in palliative cancer care.

METHODS

We will adopt a mixed methods approach consisting of 3 phases using methodology recommended by the Core Outcome Measure in Effectiveness Trials (COMET) initiative. In phase I, we will conduct a systematic review to identify existing outcomes that prognostic studies have previously used, so as to inform the development of items and domains for the proposed core outcome set. Phase II will consist of semistructured interviews with patients with advanced cancer who are receiving palliative care, informal caregivers, and clinicians, to explore their perceptions and experiences of end-of-life prognostication. Outcomes identified in the interviews will be combined with those found in existing literature and taken forward to phase III, a Delphi survey, in which we will ask patients, informal caregivers, clinicians, and relevant researchers to rate these outcomes until consensus is achieved as to which are considered to be the most important for inclusion in the core outcome set. The resulting, prioritized outcomes will be discussed in a consensus meeting to agree and endorse the final core outcome set.

RESULTS

Ethical approval was received for this study in September 2022. As of July 2023, we have completed and published the systematic review (phase I) and have started recruitment for phase II. Data analysis for phase II has not yet started. We expect to complete the study by October 2024.

CONCLUSIONS

This protocol presents the stepwise approach that will be taken to develop a core outcome set for measuring the impact of end-of-life prognostication in palliative cancer care. The final core outcome set has the potential for translation into clinical practice, allowing for consistent evaluation of emerging prognostic algorithms and improving communication of end-of-life prognostication. This study will also potentially facilitate the design of future clinical trials of the impact of end-of-life prognostication in palliative care that are acceptable to key stakeholders.

TRIAL REGISTRATION

Core Outcome Measures in Effectiveness Trials 2136; https://www.comet-initiative.org/Studies/Details/2136.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/49774.

Organizational Impact of Immunotherapies in Advanced Cancers in France

PURPOSE

In 2020, the French National Authority for Health (Haute Autorité de Santé) published a methodologic guide called organizational impact (OI) cartography to define and structure assessment of the OI of health technologies. As immunotherapies are associated with extended survival and improved quality of life in advanced cancer, we aimed to identify OIs that immunotherapies had on health care systems and professionals. To our knowledge, we suggest the first implementation for OI assessment on the basis of the cartography.

METHODS

A literature review was conducted, and interviews with health care professionals (HCPs) were performed to identify OIs of immunotherapies. They were asked if immunotherapies had OIs classified into three macrocriteria, namely, impact on the care process (six criteria), impact on capacities and skills required (six criteria), and impact on society (four criteria). If an OI was mentioned for a criterion, information on its impact (minor/moderate/major) and its timing was collected. We considered that an OI existed when 75% of HCPs mentioned an impact for a given criterion.

RESULTS

Overall, 27 HCPs were interviewed. For 12 of 16 criteria, most HCPs mentioned an impact, whereas the literature identified impacts for 11 criteria. Four criteria (skills and transfer between HCPs, scheduling capabilities, and social relationship) had consensus among HCPs and a high impact; two criteria (rhythm or care duration, working/living conditions) showed consensus but a moderate impact; two criteria (funding and scheduling capabilities cross-structure) had a high impact but no consensus. For eight criteria (as environment or inequity), there was no consensus and moderate impact.

CONCLUSION

The introduction of immunotherapies for advanced cancer has had an important OI in France, regarding capacities and skills. Further research using qualitative analysis of interviews will provide more information regarding OI.

Delphi-Based Expert Consensus Statements for the Management of Percutaneous Radiofrequency Neurotomy in the Treatment of Lumbar Facet Joint Syndrome

INTRODUCTION

A modified Delphi strategy was implemented for obtaining recommendations that could be useful in the management of percutaneous radiofrequency treatment of lumbar facet joint syndrome, as the literature on the argument was poor in quality.

METHODS

An Italian research team conducted a comprehensive literature search, defined the investigation topics (diagnosis, treatment, and outcome evaluation), and developed an explorative semi-structured questionnaire. They also selected the members of the panel. After an online meeting with the participants, the board developed a structured questionnaire of 15 closed statements (round 1). A five-point Likert scale was used and the cut-off for consensus was established at a minimum of 70% of the number of respondents (level of agreement ≥ 4, agree or strongly agree). The statements without consensus were rephrased (round 2).

RESULTS

Forty-one clinicians were included in the panel and responded in both rounds. After the first round, consensus (≥ 70%) was obtained in 9 out of 15 statements. In the second round, only one out of six statements reached the threshold. The lack of consensus was observed for statements concerning the use of imaging for a diagnosis [54%, median 4, interquartile range (IQR) 3-5], number of diagnostic blocks (37%, median 4, IQR 2-4), bilateral denervation (59%, median 4, IQR 2-4), technique and number of lesions (66%, median 4, IQR 3-5), and strategy after denervation failure (68%, median 4, IQR 3-4).

CONCLUSION

Results of the Delphi investigations suggest that there is a need to define standardized protocols to address this clinical problem. This step is essential for designing high-quality studies and filling current gaps in scientific evidence.

Development of a telemedicine group educational program for patients with heart failure: A delphi study

Aims

Evidence regarding the most effective and feasible videoconferencing group educational program for patients with heart failure is still equivocal. This study aimed to reach consensus about the structure, acceptability, and feasibility of videoconferencing for people with heart failure in Jordan that improves access to healthcare and clinical outcomes.

Methods

There were two Delphi survey studies of three rounds each. Delphi one survey involved 32 healthcare staff, experienced in heart failure clinical practice and telehealth, to obtain a consensus of opinion on a proposed group videoconferencing program for patients with heart failure. Delphi two involved seven staff of the information technology center, experienced in videoconferencing and using supporting applications, to obtain their consensus on the current capabilities of the healthcare system and patients about information technology. Descriptive statistics were used for each item to determine whether consensus was achieved or not. Items that received 80% disagreement or 80% agreement of participants were not presented for re-rating in the third round, while the items that scored varying degrees of agreement were presented for experts for re-rating.

Results

In Delphi one a group of items reached consensus regarding structure, factors influencing, and effectiveness of the videoconferencing program. In Delphi two, the findings indicated that videoconferencing modality is applicable and feasible in Jordan.

Conclusion

This is the first study that addresses the equivocal evidence for the design and implementation of heart failure videoconferencing programs. The framework of the current proposed program can be utilized as a guideline to test or develop a future videoconference program.

New International Guidelines and Consensus on the Use of Lung Ultrasound

Following the innovations and new discoveries of the last 10 years in the field of lung ultrasound (LUS), a multidisciplinary panel of international LUS experts from six countries and from different fields (clinical and technical) reviewed and updated the original international consensus for point-of-care LUS, dated 2012. As a result, a total of 20 statements have been produced. Each statement is complemented by guidelines and future developments proposals. The statements are furthermore classified based on their nature as technical (5), clinical (11), educational (3), and safety (1) statements.

Accessible Guide for People With Intellectual Disabilities in a Fitness Environment: A Delphi Study

The purpose of this study was to draw consensus among an expert panel regarding essential elements of an accessible fitness center guide for people with intellectual disabilities that will enable them to engage in physical activity fully and effectively. The study was situated in the socioecological model of disability. Researchers drew expert consensus regarding the essential features of accessible guides in fitness environments. A three-round Delphi procedure was used, involving repeated circulation of the questionnaire to an expert panel (N = 33). The panel was asked to rate the importance and adequacy of 66 items regarding the accessible fitness guide. A consensus was reached regarding 43 items after three rounds. The items include 7 body-weight exercises, 2 machine exercises, 12 environment-related items, 15 exercise preparations, 4 social etiquettes, and 3 emergencies.

Inpatient Geriatric Rehabilitation: Definitions and Appropriate Admission Criteria, as Established by Maltese National Experts

(1) Background: The importance of having an appropriate admissions system for geriatric rehabilitation is on the increase. However, the process of admitting patients to inpatient rehabilitation is a complex process. This is yet to be standardised across the European Union, as the approach to geriatric rehabilitation tends to vary from one Member State to another. (2) Objective: To discuss evidence-based practice with clinical experts, in order to define geriatric rehabilitation and admission criteria based on the Maltese population. (3) Method: The study entailed conducting four panel sessions using a purposive sample of thirteen local clinicians with extensive knowledge in clinical rehabilitation and healthcare management. A total of 48 items, based on the literature and clinical experience, were presented to the panel. Data analysis was done quantitatively and qualitatively, using IBM SPSS Statistics Version 24 and thematic analysis. (4) Results: The panel formulated a definition of rehabilitation, which shared common elements with the definition provided by the World Health Organization (WHO) and other sources/literature. The panel agreed on a list of eight criteria for appropriate inpatient geriatric rehabilitation admission in Malta. Consensus was also reached on: the need for a consultant-led multidisciplinary approach to assessment; the adoption of a standardised assessment processes for an equitable chance for all older adults assessed; the benefit of digital health in assessments; and the consideration that most patients would have some form of rehabilitation potential, depending on availability of resources. (5) Conclusion: Inpatient geriatric rehabilitation hospitals should have a unified strategy for rehabilitation services. The conclusions reached by the panel, could be useful in supporting the clinical evidence and establishing future rehabilitation guidelines and standards for inpatient rehabilitation.

Defining and measuring multimorbidity in primary care in Singapore: Results of an online Delphi study

Multimorbidity, common in the primary care setting, has diverse implications for both the patient and the healthcare system. However, there is no consensus on the definition of multimorbidity globally. Thus, we aimed to conduct a Delphi study to gain consensus on the definition of multimorbidity, the list and number of chronic conditions used for defining multimorbidity in the Singapore primary care setting. Our Delphi study comprised three rounds of online voting from purposively sampled family physicians in public and private settings. Delphi round 1 included open-ended questions for idea generation. The subsequent two rounds used questions with pre-selected options. Consensus was achieved based on a pre-defined criteria following an iterative process. The response rates for the three rounds were 61.7% (37/60), 86.5% (32/37) and 93.8% (30/32), respectively. Among 40 panellists who responded, 46.0% were 31-40 years old, 64.9% were male and 73.0% were from the public primary healthcare setting. Based on the findings of rounds 1, 2 and 3, consensus on the definition of a chronic condition, multimorbidity and finalised list of chronic conditions were achieved. For a condition to be chronic, it should last for six months or more, be recurrent or persistent, impact patients across multiple domains and require long-term management. The consensus-derived definition of multimorbidity is the presence of three or more chronic conditions from a finalised list of 23 chronic conditions. We anticipate that our findings will inform multimorbidity conceptualisation at the national level, standardise multimorbidity measurement in primary care and facilitate resource allocation for patients with multimorbidity.

Development of a Clinical Data Model Addressing the Nursing Focus "Anxiety": A Consensus Development Study

Anxiety disorders are among the most prevalent mental disorders worldwide. However, anxiety is not exclusive to anxiety disorders. In fact, the nursing discipline approaches anxiety as a human response to health conditions/life processes. Health information systems should primarily contribute to improving the quality of care, patient safety, and the effectiveness of care delivery. Nevertheless, nursing information systems still fail to incorporate evidence-based clinical data models addressing the nursing focus "anxiety." Thus, this study aimed to obtain consensus on the data to be included in a clinical data model addressing the nursing focus "anxiety," its organization, and its interrelationships by using a brainstorming session and a modified e-Delphi technique with a panel of nurse experts from across Portugal. Eight experts participated in the brainstorming session. A total of 59 and 54 participants completed the survey in e-Delphi rounds 1 and 2, respectively. Consensus was achieved to all data presented to the participants, and these data were later included in the clinical data model. This evidence-based clinical data model, grounded on a nursing theory and with standardized nursing language, will substantially contribute to nursing documentation and, consequently, to nursing care targeted at patients with anxiety.

Validation of a Script to Facilitate Social Determinant of Health Conversations with Adolescent Patients

Current social determinants of health (SDOH) tools exist to assess patient exposure; however, healthcare providers for the adolescent population are unsure of how to integrate SDOH knowledge into clinical practice. The purpose of this study was to validate a focused history script designed to facilitate SDOH conversations between clinicians and adolescents through the use of the Delphi method. Six individuals (1 clinician, 5 educators/researchers) participated as expert panelists. Panelists provided critical feedback on the script for rounds 1 and 2. For rounds 3-7, panelists received an electronic questionnaire asking them to indicate agreement on a 6-point Likert scale (1 = strongly disagree, 6 = strongly agree). We defined consensus as mean item agreement ≥ 5.0 and percent agreement ≥ 80%. In round 7, panelists rated overall script level of agreement. After seven rounds of feedback, the focused history script achieved content validity with 100% of panelists agreeing on the final 40-item script. A focused history script for the SDOH was content validated to aid conversations between healthcare providers and adolescent patients on factors that affect their life, school, and play. Addressing social determinants of health with adolescent patients will improve cultural proficiency and family-centered care delivered by school healthcare professionals.

Protocol for CAMUS Delphi Study: A Consensus on Comprehensive Reporting and Grading of Complications After Urological Surgery

BACKGROUND

Reproducible assessment of postoperative complications is essential for reliable evaluation of quality of care to enable comparison between healthcare centres and ensure transparent patient counselling. Currently, significant discrepancies exist in complication reporting and grading due to heterogeneous definitions and methodologies.

OBJECTIVE

To develop a standardised and reproducible assessment of perioperative complications and overall associated morbidity, to allow for the construction of a uniform language for complication reporting and grading.

DESIGN, SETTING, AND PARTICIPANTS

The 12-part REDCap-based Delphi survey was developed in conjunction with methodologist review and experienced urologist opinion. International urologists, anaesthetists, and intensive care unit specialists will be included. A minimum sample size of 750 participants (500 urologists and 250 critical care specialities) is targeted.

OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS

The survey assesses participant demographics, opinion on complication reporting and the proposed Complications After Major & Minor Urological Surgery (CAMUS) reporting recommendations, grading of intervention events using the existing Clavien-Dindo classification and the proposed CAMUS classification, and rating of various clinical scenarios. Consensus will be defined as ≥75% majority agreement. If consensus is not reached, then subsequent Delphi rounds will be performed under steering committee guidance.

RESULTS AND LIMITATIONS

Twenty-one participants completed the draft survey. The median survey completion time was 128 min (interquartile range 88-135). The survey revealed that 90% of participants believe that the current complication classification systems are useful but inaccurate, while 100% of participants believe that there is a universal demand for reporting consensus. Several amendments were made following feedback. Limitations include complexity of the proposed supplemental grades and time to completion of the survey.

CONCLUSIONS

To ensure comprehensive and comparable complication reporting and grading across centres worldwide, a conclusive uniform language for complication reporting must be created. We intend to address shortcomings of the current complication reporting and classification systems with a new CAMUS classification system developed through multidisciplinary expert consensus obtained through a Delphi survey. Ultimately, standardisation of urological complication reporting and grading may improve patient counselling and quality of care.

PATIENT SUMMARY

The reporting and grading of operative complications that occur during or after an operation and associated costs provide a means to stratify quality of patient care. Current complication reporting and classification systems are not standardised and somewhat inaccurate, and thus significantly underestimate patient morbidity and surgical risk. This Delphi survey will provide the basis for the creation of a uniform complication reporting and grading system. Our new system may allow improved reporting and grading between centres, and ultimately improve patient counselling and care.

The development of the Religious Health Interventions in Behavioural Science (RHIBS) Taxonomy: a scientific classification of religious practices in health

The development and delivery of religiously integrated health interventions is increasing, however lack of nomenclature to specify the religious components presents barriers to replication, implementation, and evidence synthesis. We describe the development of the "Religious Health Interventions in Behavioural Sciences (RHIBS)" Taxonomy, the first scientific classification of religious intervention components to be used globally by chaplains, healthcare providers, and researchers interested in the scientific study of religion, spirituality, and health. We developed a taxonomy of empirically used religious intervention components in health, sought international cross-disciplinary consensus for definitions and tested its usability. Study 1: systematic review of intervention studies to identify religious components tested within healthcare; development of taxonomy nomenclature, definitions, and categories. Study 2: Delphi exercise with 19 international, cross-disciplinary experts from a variety of religions. Study 3: "think aloud" study and usability testing with 10 end-users. Study 1: 12,337 papers identified from search, 167 intervention studies included, plus an additional 74 from hand-searching 14 systematic reviews. A taxonomy of 191 religious components, grouped into 27 categories resulted. Study 2: two Delphi rounds resulted in international and cross-disciplinary consensus of a revised taxonomy of 81 religious components grouped into 23 categories. Study 3: usability testing by participants (range of disciplines, geography, and religions) led to a final taxonomy comprising 82 religious components grouped into 22 categories and supported by online training. The "RHIBS Taxonomy," is the first multidisciplinary, global shared language within religion, spirituality, and health, ushering in a new era for religious interventions to be precisely defined, developed, and tested; shaping the evidence-base for future healthcare research/practice.

Artificial intelligence in medical education curriculum: An e-Delphi study for competencies

Background

Artificial intelligence (AI) has affected our day-to-day in a great extent. Healthcare industry is one of the mainstream fields among those and produced a noticeable change in treatment and education. Medical students must comprehend well why AI technologies mediate and frame their decisions on medical issues. Formalizing of instruction on AI concepts can facilitate learners to grasp AI outcomes in association with their sensory perceptions and thinking in the dynamic and ambiguous reality of daily medical practice. The purpose of this study is to provide consensus on the competencies required by medical graduates to be ready for artificial intelligence technologies and possible applications in medicine and reporting the results.

Materials and methods

A three-round e-Delphi survey was conducted between February 2020 and November 2020. The Delphi panel accorporated experts from different backgrounds; (i) healthcare professionals/ academicians; (ii) computer and data science professionals/ academics; (iii) law and ethics professionals/ academics; and (iv) medical students. Round 1 in the Delphi survey began with exploratory open-ended questions. Responses received in the first round evaluated and refined to a 27-item questionnaire which then sent to the experts to be rated using a 7-point Likert type scale (1: Strongly Disagree—7: Strongly Agree). Similar to the second round, the participants repeated their assessments in the third round by using the second-round analysis. The agreement level and strength of the consensus was decided based on third phase results. Median scores was used to calculate the agreement level and the interquartile range (IQR) was used for determining the strength of the consensus.

Results

Among 128 invitees, a total of 94 agreed to become members of the expert panel. Of them 75 (79.8%) completed the Round 1 questionnaire, 69/75 (92.0%) completed the Round 2 and 60/69 (87.0%) responded to the Round 3. There was a strong agreement on the 23 items and weak agreement on the 4 items.

Conclusions

This study has provided a consensus list of the competencies required by the medical graduates to be ready for AI implications that would bring new perspectives to medical education curricula. The unique feature of the current research is providing a guiding role in integrating AI into curriculum processes, syllabus content and training of medical students.

Active involvement of nursing staff in reporting and grading complication‐intervention events—Protocol and results of the CAMUS Pilot Nurse Delphi Study

Objectives

The aim of this study is to gain experienced nursing perspective on current and future complication reporting and grading in Urology, establish the CAMUS CCI and quality control the use of the Clavien‐Dindo Classification (CDC) in nursing staff.

Subjects and Methods

The 12‐part REDCap‐based Delphi survey was developed in conjunction with expert nurse, urologist and methodologist input. Certified local and international inpatient and outpatient nurses specialised in urology, perioperative nurses and urology‐specific advanced practice nurses/nurse practitioners will be included. A minimum sample size of 250 participants is targeted. The survey assesses participant demographics, nursing experience and opinion on complication reporting and the proposed CAMUS reporting recommendations; grading of intervention events using the existing CDC and the proposed CAMUS Classification; and rating various clinical scenarios. Consensus will be defined as ≥75% agreement. If consensus is not reached, subsequent Delphi rounds will be performed under Steering Committee guidance.

Results

Twenty participants completed the pilot survey. Median survey completion time was 58 min (IQR 40–67). The survey revealed that 85% of nursing participants believe nurses should be involved in future complication reporting and grading but currently have poor confidence and inadequate relevant background education. Overall, 100% of participants recognise the universal demand for reporting consensus and 75% hold a preference towards the CAMUS System. Limitations include variability in nursing experience, complexity of supplemental grades and survey duration.

Conclusion

The integration of experienced nursing opinion and participation in complication reporting and grading systems in a modern and evolving hospital infrastructure may facilitate the assimilation of otherwise overlooked safety data. Incorporation of focused teaching into routine nursing education will be essential to ensure quality control and stimulate awareness of complication‐related burden. This, in turn, has the potential to improve patient counselling and quality of care.

A modified Delphi consensus on generic indicators for a low- and middle-income country's quality nursing care measurement

Aim

This study aimed to develop generic quality nursing care indicators for a low‐ and middle‐income country’s quality nursing care measurement through a modified Delphi consensus.

Design

A three‐round modified Delphi process guided the items' development and acceptance consensus.

Methods

Twenty‐four academic and clinical nursing experts from different parts of Nigeria participated in the Delphi rounds. In the first round, 96 items (structure – 31, process – 38, and outcome – 27) were distributed to the panellists through e‐mail. The same method guided round 2. In round 3, the panellists reached a consensus on the number and structure of the items.

Results

In round 1, the initial items were reduced to 75 that met the Delphi consensus. In round 2, the items were modified further and reduced to 74. In round 3, the experts reached a consensus on 70 items (structure – 28, process – 32, and outcome – 10).

What medical students think about measurement of their well-being: cross-sectional survey and qualitative interviews

OBJECTIVES

To find out how medical students think well-being should be measured.

DESIGN

A mixed-methods study comprising a cross-sectional online survey (November 2020-March 2021) and semi-structured online interviews. Views on the frequency of availability for measurement, the format, type and purpose of measurement, and with whom well-being should be discussed were measured. When an outcome was scored 7-9 on a 9-point Likert scale of agreement by ≥75% of participants it was considered critical. Inductive thematic analysis was undertaken on the interview transcripts.

SETTING

All medicine programmes at University of Southampton.

PARTICIPANTS

Medical students from all years took part in the survey (n=118) and interviews (n=16).

RESULTS

Most participants (94%) felt able to give 5 min to measure their well-being at least once per month. Research, governance and individual feedback were all considered critically important. Only subjective assessments undertaken by the individual in real-time were rated critically important (78.1%) measurement tools. Students selected that they would discuss their well-being with other medical students (n=87) nearly as often as they selected a member of the faculty (n=104). Five interview themes further explained these findings: (1) well-being is mental well-being; (2) exercise and support from friends and family are most important; (3) isolation and the design of the medicine programme are detrimental to well-being; (4) there are advantages to surveys, and conversations; (5) personal academic tutors and medical students in later years are the best to discuss well-being with.

CONCLUSIONS

Medical students thought that measurement of their well-being was critically important for governance showing their support for quality assurance of well-being and peer support. They wanted to be able to choose surveys, or conversations, to measure their well-being, as well as the person they discussed well-being with. Four recommendations are discussed in light of these findings.

Screening, patient identification, evaluation, and treatment in patients with Gaucher disease: Results from a Delphi consensus

Several guidelines are available for identification and management of patients with Gaucher disease, but the most recent guideline was published in 2013. Since then, there have been significant advances in newborn screening, phenotypic characterization, identification of biomarkers and their integration into clinical practice, and the development and approval of new treatment options. Accordingly, the goal of this Delphi consensus exercise was to extend prior initiatives of this type by addressing issues related to newborn screening, diagnostic evaluations, and treatment (both disease directed and adjunctive). The iterative Delphi process involved creation of an initial slate of statements, review by a steering committee, and three rounds of consensus development by an independent panel. A preliminary set of statements was developed by the supporting agency based on literature searches covering the period from 1965 to 2020. The Delphi process reduced an initial set of 185 statements to 65 for which there was unanimous support from the panel. The statements supported may ultimately provide a framework for more detailed treatment guidelines. In addition, the statements for which unanimous support could not be achieved help to identify evidence gaps that are targets for future research.

Barriers and Facilitators to Automated Self-Scheduling: Consensus from a Delphi Panel of Key Stakeholders

INTRODUCTION

Automated self-scheduling may benefit healthcare organizations, yet uptake has been slow. The aim of this study was to develop a consensus statement regarding the organizational-level determinants of implementation success based on the collective knowledge of experts. A three-stage modified Delphi method was used to reach consensus on the top determinants of implementation of self-scheduling solutions by healthcare organizations. A panel of 53 experts representing 41 academic health systems identified barriers and facilitators involving the organization's inner and outing settings, as well as the characteristics of the intervention and the individuals engaged in the solution. Offering convenience for patients is the leading enabler for organizations to implement the technology. The consensus may aid healthcare organizations and suppliers engaged in adopting and developing self-scheduling technology to improve implementation success. Further research is recommended to diagnose and examine each barrier and facilitator and how these factors interact.

OBJECTIVE

The aim of this study was to develop a consensus statement regarding the determinants of implementation success based on the collective knowledge of experts working in the field.

METHODS

A Delphi panel was constructed based on selected participants employed by academic health systems and experienced with self-scheduling implementation. Panelists were recruited based on participation in an educational event that featured the topic. Purposive and snowball sampling were used. Panelists participated in surveys collected over three rounds. An 80 percent agreement among panelists and interquartile range (IQR) <1 determined the barriers and facilitators. The top-10 determinants were presented in rank order.

RESULTS

Between January 6, 2021, and May 26, 2021, 53 panelists representing 41 academic health systems participated in three rounds of surveys to reach consensus on the barriers and facilitators to implementation of self-scheduling by healthcare organizations in the United States. In round one, panelists documented 530 determinants. In round two, the determinants were grouped into 72 barriers and 85 facilitators, each of which participants rated on a five-point Likert scale. Fifteen determinants met the 80 percent threshold and 1.0 IQR. The final round concluded with a top-10, rank-ordered listing of determinants (seven facilitators and three barriers) that also incorporated a median rating score using five-point Likert scale.

CONCLUSION

A three-stage modified Delphi method was used to reach consensus on the top determinants of implementation of self-scheduling solutions by academic health systems. The consensus may aid healthcare organizations and suppliers engaged in adopting and developing self-scheduling technology to improve implementation success. Further research is recommended to diagnose and examine each barrier and facilitator and how these factors interact.

A Delphi Consensus Approach for Difficult-to-Treat Patients with Severe Hemophilia A without Inhibitors

Introduction

Over the past decade, there has been an increase in novel therapeutic options to treat hemophilia A. It is still unclear how these novel treatments are used in the management of patients with hemophilia A, particularly those with challenging clinical scenarios who are typically excluded in clinical trials.

Purpose

This study aimed to understand the areas of consensus and disagreement among hematologists regarding the preferences toward therapeutic approaches for difficult-to-treat patients with severe hemophilia A without inhibitors.

Patients and Methods

During February-June 2020, a three-round modified Delphi study was conducted to generate consensus among 13 US experts in the field of hemophilia. Experts were asked about their preferences toward therapeutic options for patients with challenging clinical situations, including age-related morbidities (eg, myocardial infarction, joint arthropathy), increasing demand for high-impact physical activities, early onset osteoporosis, and newborns with hemophilia A. Consensus was defined as ≥75% agreement between the panelists.

Results

Consensus was reached on many, but not all cases, leaving uncertainty about appropriateness of therapeutic approaches for some patients where clinical evidence is not available or driven by physicians' or patients' preferences toward therapeutic options. A majority of panelists preferred FVIII replacement therapy rather than emicizumab prophylaxis for the challenging cases presented due to established evidence on safety, efficacy, and level of bleed protection for FVIII treatment.

Conclusion

Recommendations emerging from this study may help guide practicing hematologists in the management of challenging hemophilia A cases. Future studies are needed to address treatment options in the clinical cases where no consensus was reached.

How to use the Delphi method to aid in decision making and build consensus in pharmacy education

OUR SITUATION

As pharmacy educators, we often encounter situations such as designing new curricula or establishing shared values for an organization that may be ambiguous or controversial. To generate effective solutions, it is often necessary to build group consensus with key stakeholders. The purpose of this paper is to describe and provide recommendations for using the Delphi method, a process for facilitating discussions and aiding in decision making.

METHODOLOGICAL LITERATURE REVIEW

An overview of the Delphi method, including its multiple variations, is presented. Steps necessary to complete a Delphi study (building a protocol, developing a research question, defining panelists and panel size, piloting the protocol, round one item creation and analysis, round two and beyond, consensus, increasing clarity, and reporting) is described.

OUR RECOMMENDATIONS AND THEIR APPLICATION(S)

Practical recommendations are provided to support use the Delphi method to build consensus in research. These recommendations include: (1) clarify the purpose of the Delphi, (2) ensure the research questions are grounded in the literature and are relevant, (3) carefully consider panelist processes, (4) determine any definitions that should be given to or developed by panelists, (5) determine methods for enhancing clarity, and (6) employ methods to reduce attrition.

POTENTIAL IMPACT

The Delphi method provides a systematic approach to generating consensus in pharmacy education for commonly encountered situations such as committee meetings, research studies, faculty retreats, classroom activities, and lab meetings.

A Practical Guide to Applying the Delphi Technique in Mental Health Treatment Adaptation: The Example of Enhanced Problem-Solving Training (E-PST)

Expert consensus methods, such as the Delphi procedure, are commonly employed in consumer, education, and health services research. However, the utility of this methodology has not widely been described in relation to mental health treatment adaptation efforts. This gap is noteworthy given that evidence-based treatments are often modified in terms of core intervention content, method of delivery, and target populations. Expert consensus methods such as the Delphi procedure offer multiple practical benefits (e.g., flexibility, resource-efficiency) for psychologists who need to adapt existing treatments to meet new research and clinical practice needs. The purpose of this paper is to provide a brief overview of the Delphi procedure, and to offer a practical guide to using this method for treatment adaptation. An example is offered using our team's application of a three-round Delphi procedure to render content and context modifications to an existing problem-solving intervention to optimize its use with a new treatment population. Data were collected from Department of Veterans Affairs clinical subject matter experts. Round 1 utilized semi-structured interviews to determine necessary protocol features and modifications. Rounds 2-3 utilized a forced-choice survey and feedback loop to evaluate expert consensus. More than 91% of rated items reached consensus following Round 2, with the remainder following Round 3. Recommended modifications included minor structural and content edits, and re-balancing time allotments. We conclude that consensus methods may facilitate treatment adaptation efforts, enhance treatment feasibility, and promote content and ecological validity. Considerations for future Delphi-based treatment adaptations are offered.

Delphi methodology in healthcare research: How to decide its appropriateness

The Delphi technique is a systematic process of forecasting using the collective opinion of panel members. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine. The Delphi methods assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically/logistically difficult or evidence is conflicting. However, the attempts to assess the quality standard of Delphi studies have reported significant variance, and details of the process followed are usually unclear. We recommend systematic quality tools for evaluation of Delphi methodology; identification of problem area of research, selection of panel, anonymity of panelists, controlled feedback, iterative Delphi rounds, consensus criteria, analysis of consensus, closing criteria, and stability of the results. Based on these nine qualitative evaluation points, we assessed the quality of Delphi studies in the medical field related to coronavirus disease 2019. There was inconsistency in reporting vital elements of Delphi methods such as identification of panel members, defining consensus, closing criteria for rounds, and presenting the results. We propose our evaluation points for researchers, medical journal editorial boards, and reviewers to evaluate the quality of the Delphi methods in healthcare research.

Delphi methodology in healthcare research: How to decide its appropriateness

The Delphi technique is a systematic process of forecasting using the collective opinion of panel members. The structured method of developing consensus among panel members using Delphi methodology has gained acceptance in diverse fields of medicine. The Delphi methods assumed a pivotal role in the last few decades to develop best practice guidance using collective intelligence where research is limited, ethically/logistically difficult or evidence is conflicting. However, the attempts to assess the quality standard of Delphi studies have reported significant variance, and details of the process followed are usually unclear. We recommend systematic quality tools for evaluation of Delphi methodology; identification of problem area of research, selection of panel, anonymity of panelists, controlled feedback, iterative Delphi rounds, consensus criteria, analysis of consensus, closing criteria, and stability of the results. Based on these nine qualitative evaluation points, we assessed the quality of Delphi studies in the medical field related to coronavirus disease 2019. There was inconsistency in reporting vital elements of Delphi methods such as identification of panel members, defining consensus, closing criteria for rounds, and presenting the results. We propose our evaluation points for researchers, medical journal editorial boards, and reviewers to evaluate the quality of the Delphi methods in healthcare research.

First Surgical National Consensus Conference of the Italian Breast Surgeons association (ANISC) on breast cancer management in neoadjuvant setting: Results and summary

RATIONALE

On October 15th, 2020, the first Surgical National Consensus Conference on neoadjuvant chemotherapy (NACT) was promoted by the Italian Association of Breast Surgeons (ANISC).

METHOD

The Consensus Conference was entirely held online due to anti-Covid-19 restrictions and after an introductory four lectures held by national and international experts in the field, a total of nine questions were presented and a digital "real-time" voting system was obtained. A consensus was reached if 75% or more of all panelists agreed on a given question.

RESULTS

A total of 202 physicians, from 76 different Italian Breast Centers homogeneously distributed throughout the Italian country, participated to the Conference. Most participants were surgeons (75%). Consensus was reached for seven out of the nine considered topics, including management of margins and lymph nodes at surgery, and there was good correspondence between the 32 "Expert Panelists" and the "Participants" to the Conference. Consensus was not achieved regarding the indications to NACT for high-grade luminal-like breast tumors, and the need to perform an axillary lymph node dissection in case of micrometastases in the sentinel lymph node after NACT.

CONCLUSIONS

NACT is a topic of major interest among surgeons, and there is need to develop shared guidelines. While a Consensus was obtained for most issues presented at this Conference, controversies still exist regarding indications to NACT in luminal B-like tumors and management of lymph node micrometastases. There is need for clinical studies and analysis of large databases to improve our knowledge on this subject.

Study protocol for autism specific transition resources (T-Res Study): developing a flexible resource package for dealing with the loosening and/or lifting of COVID-19 related restrictions

Autism specific transition resources (T-Res) aims to develop a flexible resource package to support children and young people with a diagnosis of autism spectrum disorder (ASD), as well as their families and educators, during the loosening and/or lifting of coronavirus disease 2019 (COVID-19) related restrictions on movement. A secondary aim is to determine the current and long-term impacts of the COVID-19 related restrictions on the wellbeing of individuals with autism spectrum disorders and their parents/caregivers. Measuring and addressing the psychosocial impact of the COVID-19 pandemic and related restrictions in movement is of prime importance at this time.  The impacts of this crisis will be far reaching and many may not be realised for many years. The proposed research will focus on children and young people with a diagnosis of ASD, their families and educators.  The ASD population alone is sizable with 14,000 (or 1.55%) of students in schools holding a diagnosis. When parents, teachers, tutors and special needs assistants (SNAs) are also considered this is a considerable group. The proposed research has the potential to have impacts that are social, psychological, educational and economic. This will be achieved through development of an online transition package to guide parents and educators in preparing children and young people for the resumption of regular daily routines following the lifting of COVID-19 restrictions.  This resource will be developed based on the needs of families and young people, as measured through surveys, as well as expert consensus on the targets and means of intervention.  This ambitious project can be commenced quickly and is designed to produce outputs quickly, which will in turn be disseminated to key stakeholders.

Defining core competencies in transfusion medicine for resident physicians: A multi-specialty Delphi consensus study

BACKGROUND

Although resident physicians across disciplines are responsible for ordering blood products and managing sequelae of blood product transfusion, no designated set of competencies in transfusion medicine has been established for postgraduate trainees. The primary goal of this study was to determine core transfusion-related competencies that such residents should possess.

STUDY DESIGN AND METHODS

A modified Delphi method was used to achieve consensus among a panel of clinical faculty and program leadership in six medical specialties to establish core transfusion-related competencies for resident physicians. Review of transfusion education literature, relevant clinical responsibilities, and specialty licensing requirements facilitated generation of an initial transfusion medicine topic list and additional topics were considered if suggested by experts. In two Delphi rounds, experts rated the clinical significance of initial topics on a 5-point Likert scale. Select topics were deemed core competencies if identified as Extremely Important or Moderately Important by at least 75% of panelists to meet a minimum content validity index (CVI) of 0.75 and if topics achieved a minimum content validity ratio (CVR) of 0.5.

RESULTS

Nineteen invited clinical experts completed both Delphi rounds with 100% completion across the two rounds. Twenty transfusion medicine topics achieved minimum CVI 0.75 and minimum CVR 0.5. Highest-ranked topics by level of importance include Red Blood Cell (RBC) Transfusion Indications, Platelet Transfusion Indications, and Pulmonary Reactions.

CONCLUSIONS

Multispecialty panelists across six medical specialties reached consensus in identification of core transfusion-related competencies for resident physicians. Such consensus-driven core competencies may inform the development of transfusion medicine curricula and assessments to improve transfusion safety.

An International Delphi Survey on Exercise Priorities in CKD

Introduction

Defining the role of exercise in chronic kidney disease (CKD) is a top research priority for people with CKD. We aimed to achieve consensus on specific research priorities in exercise and CKD among an international panel of stakeholders.

Methods

Using the Delphi method, patients/caregivers, researchers, clinicians, and policymakers submitted their top research priorities in round 1 and ranked their importance in rounds 2 and 3 using a 9-point Likert scale. The mean, median, and proportion of scores ranked 7 to 9 were calculated. Consensus was defined as priorities that scored above the overall mean and median score within each stakeholder panel. Qualitative description was used to understand participants’ rankings.

Results

Seventy participants (78% response) completed round 1: 15 (21.4%) clinicians, 33 (47.1%) researchers, 13 (18.6%) policymakers, and 9 (12.9%) patients; (85.7%) completed round 3. The top research priorities were defining exercise-related outcomes meaningful to patients, identifying patients’ motivation and perspective towards exercise, understanding the effect of exercise on the risk of institutionalization, mortality, and mobility, and understanding the effect of pre- and post-transplant exercise on postoperative recovery. Themes from the qualitative analysis were individualization, personal experience, and holistic approach to exercise (patients), the need to address common clinical problems (clinicians), developing targeted interventions (researchers), and the importance of evidence-based development versus implementation (policymakers).

Conclusions

Preventing physical disability was a common priority. Policymakers emphasized that more efficacy studies were needed. Other panels expressed the need for holistic and targeted exercise interventions and for outcomes that address common clinical problems.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and an expert survey

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in health-related systematic literature searching. Our study will be guided by the following research questions: What are the benefits of citation tracking for health-related systematic literature searching? Which perspectives and experiences do experts in the field of literature retrieval methods have with regard to citation tracking in health-related systematic literature searching? Methods: Our study will have two parts: a scoping review and an expert survey. The scoping review aims at identifying methodological studies on benefits or problems of citation tracking in health-related systematic literature searching with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE, The Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science Core Collection, two information science databases, and free web searching. Two reviewers will independently assess full texts of selected abstracts. We will conduct direct backward and forward citation tracking on included articles. The results of the scoping review will inform our expert survey through which we aim to learn about experts΄ perspectives and experiences. We will narratively synthesize the results and derive recommendations for performing health-related systematic reviews.

Using citation tracking for systematic literature searching - study protocol for a scoping review of methodological studies and a Delphi study

Background: Up-to-date guidance on comprehensive study identification for systematic reviews is crucial. According to current recommendations, systematic searching should combine electronic database searching with supplementary search methods. One such supplementary search method is citation tracking. It aims at collecting directly and/or indirectly cited and citing references from "seed references". Tailored and evidence-guided recommendations concerning the use of citation tracking are strongly needed. Objective: We intend to develop recommendations for the use of citation tracking in systematic literature searching for health-related topics. Our study will be guided by the following research questions: What is the benefit of citation tracking for systematic literature searching for health-related topics? Which methods, citation indexes, and other tools are used for citation tracking? What terminology is used for citation tracking methods? Methods: Our study will have two parts: a scoping review and a Delphi study. The scoping review aims at identifying methodological studies on the benefit and use of citation tracking in systematic literature searching for health-related topics with no restrictions on study design, language, and publication date. We will perform database searching in MEDLINE (Ovid), CINAHL (EBSCOhost), Web of Science Core Collection, two information science databases, web searching, and contact experts in the field. Two reviewers will independently perform study selection. We will conduct direct backward and forward citation tracking on included articles. Data from included studies will be extracted using a prespecified extraction sheet and presented in both tabular and narrative form. The results of the scoping review will inform the subsequent Delphi study through which we aim to derive consensus recommendations for the future practice and research of citation tracking.

Delphi consensus on core criteria set selecting among health-related outcome measures (HROM) in primary health care

OBJECTIVES

There has been a lack of consensus on which criteria are important when selecting health-related outcome measures (HROM), especially in primary health care A Delphi consensus was undertaken to identify core criteria for selecting among different HROM and contextual factors affecting decision-making.

STUDY DESIGN AND SETTING

A Delphi survey using standard methodology (consensus set as 75% agreement) was sent to primary care practitioners, administrators, policymakers, researchers and experts in HROM development. Criteria (n = 35) were selected from a targeted literature search; questions probed contextual factors that may influence selection of core criteria including the following: 1) weighting, 2) clinical contexts, and 3) core areas.

RESULTS

Participants (n = 45) achieved consensus on 17 items in round 1 (plus four new criteria), with one more item in round 2. Unequal weighting of criteria was favored, and the type of clinical scenario did not influence selection of HROM.

CONCLUSION

Eighteen core criteria from 39 were identified for inclusion in a core set, reflected predominately aspects of feasibility and acceptability. This is the first study providing evidence that contextual factors affect selection of HROM in a core criteria set. Care in primary health care is complex, and therefore, the items not reaching consensus may reflect this heterogeneity.

A Binational Need Assessment to Define the Level of Endovascular Expertise Required by Vascular Surgical Trainees

BACKGROUND

There has been a shift toward competency-based surgical education programs to improve trainee performance and achieve better patient outcomes. Endovascular procedures comprise a significant volume of vascular surgery, but the current methods for assessing the endovascular competence of vascular trainees in Australia and New Zealand are suboptimal. The objective of this study was to perform a need assessment to define the scope of endovascular expertise required by vascular surgical trainees to later aid in the development of novel surgical training assessment tools.

METHODS

A modified Delphi method was used to achieve expert consensus. Fifty-three key stakeholders in vascular surgical education and training (SET) in Australia and New Zealand were invited to take part in the 2-stage survey. Experts were asked which procedures they considered to be requisite for vascular surgery trainees and at which SET level competence should be achieved. The results were reiterated to the expert panel in the second stage, and consensus considered achieved if over 75% of experts were in agreement.

RESULTS

In the first stage 25 experts reached consensus that competence in 18 of the 26 procedures should be requisite for SET trainees. Twenty-two experts responded to the second stage and consensus was achieved for 12 out of 14 of the procedural items with mean percentage of experts in agreement being 90%.

CONCLUSIONS

A need assessment using a modified Delphi method has achieved consensus among experts in vascular surgery regarding the endovascular procedures considered to be requisite for vascular surgery trainees in Australia and New Zealand.

Combining volunteers and primary care teamwork to support health goals and needs of older adults: a pragmatic randomized controlled trial

BACKGROUND

The Health TAPESTRY (Health Teams Advancing Patient Experience: STRengthening QualitY) intervention was designed to improve primary care teamwork and promote optimal aging. We evaluated the effectiveness of Health TAPESTRY in attaining goals of older adults (e.g., physical activity, productivity, social connection, medical status) and other outcomes.

METHODS

We conducted a pragmatic randomized controlled trial between January and October 2015 in a primary care practice in Hamilton, Ontario. Older adults were randomized (1:1) to Health TAPESTRY (n = 158) or control (n = 154). Trained community volunteers gathered information on people's goals, needs and risks in their homes, using electronic forms. Interprofessional primary care teams reviewed summaries and addressed issues. Participants reported goal attainment (primary outcome), self-efficacy, quality of life, optimal aging, social support, empowerment, physical activity, falls, and access to and comprehensiveness of the health system. We determined use of health care resources through chart audit.

RESULTS

There were no differences between groups in goal attainment or many other patient-reported outcome and experience assessments at 6 months. More primary care visits took place in the intervention versus control group over 6 months (mean ± standard deviation [SD] 4.93 ± 3.86 v. 3.50 ± 3.53; difference of 1.52 [95% confidence interval (CI) 0.84 to 2.19]). The odds of having 1 or more hospital admission were lower for the intervention group (odds ratio [OR] 0.44 [95% CI 0.20 to 0.95]).

INTERPRETATION

Health TAPESTRY did not improve the primary outcome of goal attainment but showed signals of shifting care from reactive to active preventive care. Further evaluation will help in understanding effective components, costs and consequences of the intervention. Trial registration: ClinicalTrials.gov, no. NCT02283723.