Delphi methodology in healthcare research: How to decide its appropriateness

Glossary of healthcare pathways: a methodological approach involving a transdisciplinary team in public health

Introduction

The healthcare pathway is at the heart of public health organization concerns, but communication between the various players can be an obstacle. This work, produced by a French transdisciplinary team, offers a methodological approach based on formalized consensus to elaborate a glossary of healthcare pathways. A two-steps procedure was elaborated, including a double rounded Delphi method to formalize expert consensus, and two groups of experts: a workgroup and a review group.

Methods

The workgroup provided a list of words or expressions that, in their opinion, described, evaluated or compared the healthcare pathways for patients, caregivers or regulators. The review group checked this list and added or deleted words or expressions. Then, definitions were added by the workgroup based into account three dimensions: official, academic and from the field. The review group validated the definitions and provided complementary proposals if needed.

Results

After pooling the list of words proposed by each of the six members of the working group, 417 words/expressions were ranked. After the two rounds of evaluation, 294 words/expressions were rated "appropriate" and were analyzed by the review group. This group, after two rounds of evaluation, agreed on 263 words/expressions that were transmitted to the working group who defined them. These definitions were rated by the review group. The first round of evaluation established 195 definitions as being appropriated whereas 68 definitions were amended by the review group.

Conclusion

This glossary supports transdisciplinary communication, reduces the extent of variations in practice and optimizes decision-making. International debate on all aspects might be strengthened by an improved understanding of the concept of health pathway.

British Oncology Pharmacy Association Delphi consensus guidelines: Co-infusion of trometamol-containing calcium folinate (Leucovorin) with systemic anti-cancer treatments

Drug stability and compatibility are critical factors influencing the cost and logistics of treatment delivery, therapeutic effectiveness, and patient safety. This is particularly significant in the realm of cancer chemotherapeutics, where stability and compatibility studies play a vital role in ensuring rational and safe medicine administration. Oxaliplatin, fluorouracil, and irinotecan, commonly used in various combinations for gastrointestinal cancers, are complemented by co-administration of folinic acid in certain protocols. Notably, some folinic acid preparations include trometamol as an excipient, potentially impacting the stability of the chemotherapeutic agents if infused concomitantly. This study seeks to establish guidelines for oncology multidisciplinary teams, addressing potential risks associated with the combination of trometamol-containing folinic acid and chemotherapeutics. To achieve this, a quantitative questionnaire was distributed to members of the British Oncology Pharmacy Association (BOPA) and non-BOPA members through an online survey. Nineteen healthcare professionals with oncology experience, comprising 18 pharmacists and one nurse, completed the questionnaires. Each participant rated the validity and clarity of statements on a 5-point scale. The Delphi process concluded after the fourth round, consolidating the findings and recommendations from the multidisciplinary team. Twelve recommendations for safe practice have been made.

[Strengthening leadership for educational management in the Americas: an action research strategyFortalecimento da liderança para a gestão educacional na Região das Américas: uma estratégia de pesquisa-ação]

The training of human resources for health (HRH) is a recurring concern. The Virtual Campus for Public Health (VCPH) - the educational platform of the Pan American Health Organization (PAHO) - seeks to improve access to public health education, emphasizing leadership training for teams that plan, implement, and evaluate policies and educational initiatives.The objective of this work is to present the results of a consultation with experts on the virtual course "Leadership for Educational Management in Health Organizations", as part of a PAHO/VCPH strategy to strengthen leadership capacities for educational management in the Americas.A qualitative action-research study was carried out using the Delphi method with two iterations of consultations (one virtual and one in person) with experts in educational management in health organizations.The results show the importance of eight dimensions of analysis: recipient profiles, competencies, approach and contents, activities, evaluation of and for learning, adaptability, implementation, and monitoring during and after the course.The participation of actors from different geopolitical spheres in the design and implementation of a regional educational initiative fosters adaptations in the context of its implementation and improves the likelihood it will be adopted. This course can play a strategic role as a catalyst in the formation and consolidation of an integrated network of organizations that strengthen educational leadership in the Americas. This study also highlights the value of the methodological strategy used to improve the quality of HRH training.

Exploring the concept of unmet need within sexual and reproductive health in England: A qualitative Delphi exercise

BACKGROUND

Unmet need within sexual and reproductive health (SRH) is a concept that is difficult to define and measure. This qualitative Delphi exercise was used to ascertain the opinions of SRH professionals on the conceptualisation and measurement of unmet need within SRH.

METHODS

This exercise was carried out in two rounds. In the first round, respondents responded narratively to three prompts, which were then used to create a series of statements. In the second round, participants responded narratively to the statements created in the first round. Responses from both rounds were then coded and analysed thematically.

RESULTS

Participants felt that an understanding of unmet need is an important part of SRH service design and provision, and believed that certain populations are often underrepresented within the datasets that are used to assess unmet need. Many respondents felt that a full understanding of unmet need within SRH would only come from involvement of relevant stakeholders in the process of investigating unmet need, and that qualitative methods may also have a role to play in gaining a more holistic understanding of unmet need within SRH.

CONCLUSIONS

Respondents within this study felt that unmet need is complex concept that has a significant impact on service delivery and the outcomes and experiences of the most vulnerable populations. We need to improve our understanding of unmet need and prioritise stakeholder voices if we want to create interventions that address unmet need within SRH.

Perioperative Patients With Hemodynamic Instability: Consensus Recommendations of the Anesthesia Patient Safety Foundation

In November of 2022, the Anesthesia Patient Safety Foundation held a Consensus Conference on Hemodynamic Instability with invited experts. The objective was to review the science and use expert consensus to produce best practice recommendations to address the issue of perioperative hemodynamic instability. After expert presentations, a modified Delphi process using discussions, voting, and feedback resulted in 17 recommendations regarding advancing the perioperative care of the patient at risk of, or with, hemodynamic instability. There were 17 high-level recommendations. These recommendations related to the following 7 domains: Current Knowledge (5 statements); Preventing Hemodynamic Instability-Related Harm During All Phases of Care (4 statements); Data-Driven Quality Improvement (3 statements); Informing Patients (2 statements); The Importance of Technology (1 statement); Launch a National Campaign (1 statement); and Advancing the Science (1 statement). A summary of the recommendations is presented in Table 1 .

Pilot Test of A Gender-conscious Sexual Health Intake Questionnaire: Increasing Inclusivity and Mitigating Bias in Sexual History Taking

Background

Sexual health is critical to overall health, yet sexual history taking is challenging. LGBTQ+ patients face additional barriers due to cis/heteronormativity from the medical system. We aimed to develop and pilot test a novel sexual history questionnaire called the Sexual Health Intake (SHI) form for patients of diverse genders and sexualities.

Methods

The SHI comprises four pictogram-based questions about sexual contact at the mouth, anus, vaginal canal, and penis. We enrolled 100 sexually active, English-speaking adults from a gender-affirming surgery clinic and urology clinic from November 2022 to April 2023. All surveys were completed in the office. Patients also answered five feedback questions and 15 questions from the Patient-Reported Outcomes Measurement Information System Sexual Function and Satisfaction (PROMIS-SexFS) survey as a validated comparator.

Results

One hundred patients aged 19-86 years representing an array of racial/ethnic groups, gender identities, and sexuality completed the study. Forms of sexual contact varied widely and included all possible combinations asked by the SHI. Feedback questions were answered favorably in domains of clinical utility, inclusiveness of identity and anatomy, and comprehensiveness of forms of sexual behavior. The SHI captured more positive responses than PROMIS-SexFS in corresponding questions about specific types of sexual activity. The SHI also asks about forms of sexual contact that are not addressed by PROMIS-SexFS, such as penis-to-clitoris.

Conclusions

SHI is an inclusive, patient-directed tool to aid sexual history taking without cisnormative or heteronormative biases. The form was well received by a diverse group of participants and can be considered for use in the clinical setting.

International Consensus Recommendations for Safe Use of LAMS for On- and Off-Label Indications Using a Modified Delphi Process

INTRODUCTION

The study aimed to develop international consensus recommendations on the safe use of lumen-apposing metal stents (LAMSs) for on- and off-label indications.

METHODS

Based on the available literature, statements were formulated and grouped into the following categories: general safety measures, peripancreatic fluid collections, endoscopic ultrasound (EUS)-biliary drainage, EUS-gallbladder drainage, EUS-gastroenterostomy, and gastric access temporary for endoscopy. The evidence level of each statement was determined using the Grading of Recommendations Assessment, Development, and Evaluation methodology.International LAMS experts were invited to participate in a modified Delphi process. When no 80% consensus was reached, the statement was modified based on expert feedback. Statements were rejected if no consensus was reached after the third Delphi round.

RESULTS

Fifty-six (93.3%) of 60 formulated statements were accepted, of which 35 (58.3%) in the first round. Consensus was reached on the optimal learning path, preprocedural imaging, the need for airway protection and essential safety measures during the procedure, such as the use of Doppler, and measurement of the distance between the gastrointestinal lumen and the target structure. Specific consensus recommendations were generated for the different LAMS indications, covering, among others, careful patient selection, the preferred size of the LAMS, the need for antibiotics, the preferred anatomic location of the LAMS, the need for coaxial pigtail placement, and the appropriate management of LAMS-related adverse events.

DISCUSSION

Through a modified international Delphi process, we developed general and indication-specific experience- and evidence-based recommendations on the safe use of LAMS.

The PICOTS-ComTeC Framework for Defining Digital Health Interventions: An ISPOR Special Interest Group Report

OBJECTIVES

Digital health definitions are abundant, but often lack clarity and precision. We aimed to develop a minimum information framework to define patient-facing digital health interventions (DHIs) for outcomes research.

METHODS

Definitions of digital-health-related terms (DHTs) were systematically reviewed, followed by a content analysis using frameworks, including PICOTS (population, intervention, comparator, outcome, timing, and setting), Shannon-Weaver Model of Communication, Agency for Healthcare Research and Quality Measures, and the World Health Organization's Classification of Digital Health Interventions. Subsequently, we conducted an online Delphi study to establish a minimum information framework, which was pilot tested by 5 experts using hypothetical examples.

RESULTS

After screening 2610 records and 545 full-text articles, we identified 101 unique definitions of 67 secondary DHTs in 76 articles, resulting in 95 different patterns of concepts among the definitions. World Health Organization system (84.5%), message (75.7%), intervention (58.3%), and technology (52.4%) were the most frequently covered concepts. For the Delphi survey, we invited 47 members of the ISPOR Digital Health Special Interest Group, 18 of whom became the Delphi panel. The first, second, and third survey rounds were completed by 18, 11, and 10 respondents, respectively. After consolidating results, the PICOTS-ComTeC acronym emerged, involving 9 domains (population, intervention, comparator, outcome, timing, setting, communication, technology, and context) and 32 optional subcategories.

CONCLUSIONS

Patient-facing DHIs can be specified using PICOTS-ComTeC that facilitates identification of appropriate interventions and comparators for a given decision. PICOTS-ComTeC is a flexible and versatile tool, intended to assist authors in designing and reporting primary studies and evidence syntheses, yielding actionable results for clinicians and other decision makers.

An International Competency Framework for High-Quality Workforce Development in Integrated Care (IC): A Modified Delphi Study Among Global Participants

Introduction

There have been increasing calls in the literature recommending training in integrated care (IC) for health and social care professionals. Although studies have focused on different stakeholders' perceptions of education and training, there is no consistent definition of the key competencies or approach to implementing these competencies among health and social care providers. This study used a modified Delphi consensus-building method with global panellists with experience in delivering and designing training in IC to ascertain which competencies are important in an international framework guiding workforce development in IC.

Methods

A four-step methodological process was used. First, a scoping review identified a potential list of competencies and features of education and training in IC. Second, predefined criteria were used to identify global panellists with IC education experience. Third, two anonymous iterative Delphi rounds were conducted to (1) reach a consensus on the level of importance of the competencies and key themes to be included and (2) identify existing models of training in IC. This was followed by the analysis of the Delphi study and presentation of the results.

Results

A list of eight domains and 40 competencies was generated. Twenty-one panellists reviewed the competencies in the first and second round. The highest importance rankings were allocated to person-centred care, interprofessional teamwork and care coordination. The lower-ranking domains focused on professional workforce attributes.

Discussion and conclusion

The study provides a global consensus on the competencies required for workforce training and development in IC and offers recommendations on how these competencies can be implemented in higher education and vocational institutions and workplace settings. The results will be useful for developing policy and curriculum by health and education providers and accreditation bodies.

A consensus process to identify research priorities in paediatric urology in the United Kingdom

INTRODUCTION

Paediatric urologists manage a spectrum of conditions, much of the evidence for relevant treatment pathways is of low quality. For many conditions treatment varies according to location and surgeon; children with the same condition might have surgery in one unit but watchful waiting in another. Underlying this variation are differences in opinion, and insufficient high-level evidence with few prospective randomized studies. Such studies may be challenging to design, fund and recruit into, and are more likely to succeed if there is a collaborative approach. Research prioritization is a tool to identify the research of most value. Delphi methodology is an interpretive technique aiming to gain the consensus view of interested parties. The British Association of Paediatric Urologists (BAPU) set out to ascertain consensus on what paediatric urologists, working in the UK, consider to be areas of priority for research. This paper describes the process used, and the resulting list of research questions.

METHODS

A scoping survey of paediatric urologists in the UK was undertaken to identify an initial set of research questions. These were refined by the BAPU research committee (BAPU RC), then prioritized using a modified Delphi process. During Stage 1a multiple new research questions were submitted leading to Stage 1b, an interim process. All UK paediatric urologists were invited to take part in Stage 2 of the prioritization process.

RESULTS

Sixty-five questions were submitted to the scoping survey by 24 paediatric urologists. The BAPU RC refined these to 60 questions, which were submitted to Stage 1a of the modified Delphi process. Sixty-seven people completed Stage 1a, at the same time submitting 224 additional research questions. The BAPU RC revised the entire question set, ensuring the key subject of the original question was not altered and novel questions were retained. The BAPU RC undertook interim scoring of the resultant 79 questions, the top scoring 25 questions plus 5 lower scoring 'wild card' questions (to ensure the breadth of the specialty was represented) were put forward to Stage 2. A total of 65 people completed Stage 2, including a lay representative. A list of 30 priority research questions was generated; the top 10 includes management of neuropathic bladder, posterior urethral valves, antibiotic prophylaxis, DSD & CAH, continence, male external genitalia, VUR and transition care (Table).

CONCLUSION

This process has provided BAPU, paediatric urologists in the UK, and funders with areas of research considered a priority in the specialty.

Consensus on surgical technique for sentinel lymph node dissection in cervical cancer

OBJECTIVE

The purpose of this study was to establish a consensus on the surgical technique for sentinel lymph node (SLN) dissection in cervical cancer.

METHODS

A 26 question survey was emailed to international expert gynecological oncology surgeons. A two-step modified Delphi method was used to establish consensus. After a first round of online survey, the questions were amended and a second round, along with semistructured interviews was performed. Consensus was defined using a 70% cut-off for agreement.

RESULTS

Twenty-five of 38 (65.8%) experts responded to the first and second rounds of the online survey. Agreement ≥70% was reached for 13 (50.0%) questions in the first round and for 15 (57.7%) in the final round. Consensus agreement identified 15 recommended, three optional, and five not recommended steps. Experts agreed on the following recommended procedures: use of indocyanine green as a tracer; superficial (with or without deep) injection at 3 and 9 o'clock; injection at the margins of uninvolved mucosa avoiding vaginal fornices; grasping the cervix with forceps only in part of the cervix is free of tumor; use of a minimally invasive approach for SLN biopsy in the case of simple trachelectomy/conization; identification of the ureter, obliterated umbilical artery, and external iliac vessels before SLN excision; commencing the dissection at the level of the uterine artery and continuing laterally; and completing dissection in one hemi-pelvis before proceeding to the contralateral side. Consensus was also reached in recommending against injection at 6 and 12 o'clock, and injection directly into the tumor in cases of the tumor completely replacing the cervix; against removal of nodes through port without protective maneuvers; absence of an ultrastaging protocol; and against modifying tracer concentration at the time of re-injection after mapping failure.

CONCLUSION

Recommended, optional, and not recommended steps of SLN dissection in cervical cancer have been identified based on consensus among international experts. These represent a surgical guide that may be used by surgeons in clinical trials and for quality assurance in routine practice.

Identifying critical quality metrics in Mohs Surgery: A national expert consensus process

BACKGROUND

Amid a movement toward value-based healthcare, increasing emphasis has been placed on outcomes and cost of medical services. To define and demonstrate the quality of services provided by Mohs surgeons, it is important to identify and understand the key aspects of Mohs micrographic surgery (MMS) that contribute to excellence in patient care.

OBJECTIVE

The purpose of this study is to develop and identify a comprehensive list of metrics in an initial effort to define excellence in MMS.

METHODS

Mohs surgeons participated in a modified Delphi process to reach a consensus on a list of metrics. Patients were administered surveys to gather patient perspectives.

RESULTS

Twenty-four of the original 66 metrics met final inclusion criteria. Broad support for the initiative was obtained through physician feedback.

LIMITATIONS

Limitations of this study include attrition bias across survey rounds and participation at the consensus meeting. Furthermore, the list of metrics is based on expert consensus instead of quality evidence-based outcomes.

CONCLUSION

With the goal of identifying metrics that demonstrate excellence in performance of MMS, this initial effort has shown that Mohs surgeons and patients have unique perspectives and can be engaged in a data-driven approach to help define excellence in the field of MMS.

Defining competencies in robotic thyroidectomy: development of a model assessing an expert operator's intraoperative performance skills and cognitive strategies

Background

The changing medical education environment emphasizes the need for time efficiency, increasing the demand for competency-based medical education to improve trainees' learning strategies. This study was performed to determine the competencies required for successful performance of robotic thyroidectomy (RT) and to determine the consensus of experts for performance of RT.

Methods

Data were collected through 12 semi-structured interviews with RT experts and 11 field observations. Cognitive task analysis was performed to determine the competencies required for experts to perform RT. A modified Delphi methodology was used to determine how 20 experts rated the importance of each item of RT performance on a Likert 7-point scale. The criteria for the Delphi consensus were set at a Cronbach's α≥0.80 with two survey rounds.

Results

After 11 field observations and 12 semi-structured interviews, 89 items were identified within six modules. These items were grouped into sub-modules according to their theme. The modified Delphi survey, involving 21 experts, reached the consensus standard during the second round (Cronbach's α=0.954), enabling the identification of the 64 most important items within six modules related to RT performance: midline incision to isthmectomy (MID module; n=8), lateral dissection (LAT module; n=7), preservation of inferior parathyroid glands (INF module; n=16), preservation of recurrent laryngeal nerve and dissection of the ligament of Berry (BER module; n=21), dissection of the thyroid upper pole (SUP module; n=10), and specimen removal and closure (END module; n=2).

Conclusions

This mixed-method study combining qualitative and quantitative methodology identified modules of core competencies required to perform RT. These modules can be used as a standard and objective guide to train surgeons to perform RT and evaluate outcomes.

Epidemiology, Characteristics of Disease, and Unmet Needs of Patients with Generalized Pustular Psoriasis: A Large Italian Delphi Consensus

INTRODUCTION

Generalized pustular psoriasis (GPP) is a rare and chronic, debilitating skin condition characterized, in its acute flare phase, by clinically severe and potentially life-threatening systemic manifestations. Data on GPP are still scanty, particularly in Europe and at a national level. The aim of this study was to provide expert indications on several disease-related and patient-related aspects of GPP, with specific focus to the Italian context.

METHODS

We conducted an iterative eDelphi study following the recommended criteria for reporting methods and results. After a thorough bibliographic review aimed to identify unknown or controversial issues in GPP, the following areas were investigated through a few specific questions/statements for each area: (1) disease epidemiology; (2) disease characteristics, with specific interest toward GPP flares; (3) diagnosis and diagnostic delay; (4) GPP treatment; (5) GPP patient journey and use of healthcare resources in Italy; (6) unmet needs and quality of life. An Executive Board of 9 principal investigators revised and approved the topics to be examined and overviewed the whole project. A total of 35 experts from different Italian areas, including 34 board-certified Italian dermatologists and 1 representative of patients' associations, took part in the study.

RESULTS

A high agreement in responses from Italian experts emerged during two eDelphi iterations on - among several other aspects - GPP prevalence and incidence in Italy, use of European Rare and Severe Psoriasis Expert Network diagnostic criteria, flare frequency and duration, best diagnostic and care pathway, and main unmet needs of Italian patients. On the other hand, a broad spectrum of treatments (of different drug classes) was reported both in the acute and chronic phases of GPP, and no consensus on the issue was thus achieved.

CONCLUSIONS

Consensus findings from this Delphi study of GPP experts may be useful to fill gaps of knowledge and improve awareness of this rare disease, as well as to help clinical and public health management of GPP in Italy.

Development of the national consensus statement on ear health and hearing check recommendations for Aboriginal and Torres Strait Islander children aged under 6 years attending primary care: systematic scoping review and e-Delphi

BACKGROUND

Early detection of long-term, often asymptomatic, middle ear infection in young Aboriginal and Torres Strait Islander children is more likely to be achieved when ear health and hearing checks are routinely undertaken in primary healthcare. Evidence consistently demonstrates the adverse impacts of this condition on the development and wellbeing of children and their families. We aimed to develop feasible, evidence- and consensus-based primary healthcare recommendations addressing the components and timing of ear health and hearing checks for Aboriginal and Torres Strait Islander children aged under 6 years, not already known to have, nor being actively managed for, ear and hearing problems.

METHODS

A 22-person working group comprising Aboriginal and Torres Strait Islander and non-Indigenous members from the primary healthcare, ear, hearing, and research sectors provided guidance of the project. A systematic scoping review addressed research questions relating to primary health ear health and hearing checks for Aboriginal and Torres Strait Islander and other populations at increased risk of persistent ear health problems. Twelve primary studies and eleven guidelines published between 1998 and 2020 were identified and reviewed. Quality and certainty of evidence and risk of bias ratings were completed for studies and guidelines. In the absence of certain and direct evidence, findings and draft recommendations were presented for consensus input to a 79-member expert panel using a modified e-Delphi process. Recommendations were finalised in consultation with working group members and presented to expert panel members for input on considerations relating to implementation.

RESULTS

Overall, the quality, certainty, and directness of evidence in the studies and guidelines reviewed was low. However, the findings provided a basis and structure for the draft recommendations presented during the consensus-building process. After two e-Delphi rounds, seven goals and eight recommendations on the components and timing of Ear Health and Hearing Checks in primary healthcare for young Aboriginal and Torres Strait Islander children were developed.

CONCLUSIONS

The systematic scoping review and consensus-building process provided a pragmatic approach for producing strong recommendations within a reasonably short timeframe, despite the low quality and certainty of evidence, and paucity of studies pertaining to primary healthcare settings.

'Towards a conceptualization of nurses' support of hospitalised patients' self-management-A modified Delphi study'

AIM

To determine patients', nurses' and researchers' opinions on the appropriateness and completeness of the proposed conceptualization of nurses' support of hospitalised patients' self-management.

DESIGN

A modified Delphi study.

METHODS

We conducted a two-round Delphi survey. The panel group consisted of patients, nurses and researchers. The conceptualization of nurses' support of hospitalised patients' self-management presented in the first Delphi round was based on previous research, including a scoping review of the literature. Data was analysed between both rounds and after the second round. Results are reported in accordance with the guidance on Conducting and Reporting Delphi Studies (CREDES).

RESULTS

In the first round all activities of the proposed conceptualization were considered appropriate to support the patients' self-management. Panel members' comments led to the textual adjustment of 19 activities, the development of 15 new activities, and three general questions related to self-management support during hospitalisation. In the second round the modified and the newly added activities were also deemed appropriate. The clarification statements raised in the first Delphi round were accepted, although questions remained about the wording of the activities and about what is and what is not self-management support.

CONCLUSION

After textual adjustments and the addition of some activities, the proposed conceptualization of nurses' support in patients' self-management while hospitalised have been considered appropriate and complete. Nevertheless, questions about the scope of this concept still remains. The results provide a starting point for further discussion and the development of self-management programs aimed at the hospitalised patient.

IMPLICATION FOR THE PROFESSION AND/OR PATIENT CARE

The results can be considered as a starting point for practice to discuss the concept of nurses' support for hospitalised patients' self-management and develop, implement and research self-management programs specific for their patient population.

REPORTING METHOD

Results are reported in accordance with the guidance on Conducting and Reporting Delphi Studies (CREDES).

PATIENT OR PUBLIC CONTRIBUTION

Patients were involved as expert panellist in this Delphi study. Impact statement What problem did the study address? Self-management support during hospitalisation is understudied, which undermines the development of evidence-based interventions. What were the main findings? A panel, consisting of patients, nurses and researchers, agreed on the appropriateness of a conceptualization of nurses' support of inpatients' self-management, and identified some points for discussion, mainly related to the boundaries of the concept self-management. Where and on whom will the research have an impact? This study is crucial for generating conceptual understanding of how nurses support patients' self-management during hospitalisation. This is necessary for policy, clinical practice, education, and research on this topic.

Peri-articular bone formation involving the temporomandibular joint: a narrative summary and Delphi consensus of a new classification system

Heterotopic ossification (HO) is defined as 'bone where it does not belong'. Given the historical variety of nomenclature and eponyms, there is significant confusion about the etiology, pathogenesis, classification, and treatment of HO related to the temporomandibular joint (TMJ). The existing classifications for TMJ HO have shortcomings: they relate to specific populations, use differing imaging studies and demographic data, do not universally include alloplastic/autologous replacements, are based variously on radiological and/or clinical presentations that cannot always be combined, and were largely developed to assist oral and maxillofacial surgeons in surgical management. These deficiencies make it problematic to compare studies, draw valid conclusions, and pursue research. The aim of this study was to develop a new, more inclusive classification for TMJ HO. Currently available classifications were evaluated and a Delphi-type system used to build consensus from clinicians and researchers to develop a new system. Fourteen unique classifications for TMJ ankylosis/HO were identified. In light of the biological specifics related to heterotopic calcification of extracellular matrix versus heterotopic formation of actual bone, the group recommends a more unambiguous term - peri-articular bone formation - and proposes a new classification. This will help clinicians and researchers to study, describe, and manage various types of ectopic bone associated with the TMJ.

Recognition of climate-related risks for prehospital emergency medical service and emergency department in Finland - A Delphi study

BACKGROUND

Emergency departments (ED) and prehospital emergency medical services (EMS) will experience new or increasing challenges due to the changing climate. The aims of this study was to add knowledge about these challenges in Finland and to help EMS and ED operators to prepare for the effects of climate change.

METHODS

A two-round Delphi study was conducted. Ten participants expressed their views of climate change-related challenges currently and in the future, and how to prepare for challenges ahead. First-round questions based on the literature search about the climate-related impacts on EMS and ED. The stage one data was analysed by thematic analysis, which generated the second-round survey where the probability of the statements was estimated.

RESULTS

Various climate change-related challenges were recognized such as negative health impacts, the increased workload, difficulties with the EMS operations and problems with the functions of society. Preparation of action plans was considered important in case for incidents and emergencies.

CONCLUSION

The study indicated that climate change may cause various challenges for EMS and ED in Finland. To meet the future challenges, it is important to identify potential future risks and create plans to manage them. Further studies are needed to create climate resilient healthcare systems.

Percutaneous Left Ventricular Unloading During High-Risk Coronary Intervention: Rationale and Design of the CHIP-BCIS3 Randomized Controlled Trial

INTRODUCTION

Percutaneous coronary intervention for complex coronary disease is associated with a high risk of cardiogenic shock. This can cause harm and limit the quality of revascularization achieved, especially when left ventricular function is impaired at the outset. Elective percutaneous left ventricular unloading is increasingly used to mitigate adverse events in patients undergoing high-risk percutaneous coronary intervention, but this strategy has fiscal and clinical costs and is not supported by robust evidence.

METHODS

CHIP-BCIS3 (Controlled Trial of High-Risk Coronary Intervention With Percutaneous Left Ventricular Unloading) is a prospective, multicenter, open-label randomized controlled trial that aims to determine whether a strategy of elective percutaneous left ventricular unloading is superior to standard care (no planned mechanical circulatory support) in patients undergoing nonemergent high-risk percutaneous coronary intervention. Patients are eligible for recruitment if they have severe left ventricular systolic dysfunction, extensive coronary artery disease, and are due to undergo complex percutaneous coronary intervention (to the left main stem with calcium modification or to a chronic total occlusion with a retrograde approach). Cardiogenic shock and acute ST-segment-elevation myocardial infarction are exclusions. The primary outcome is a hierarchical composite of all-cause death, stroke, spontaneous myocardial infarction, cardiovascular hospitalization, and periprocedural myocardial infarction, analyzed using the win ratio. Secondary outcomes include completeness of revascularization, major bleeding, vascular complications, health economic analyses, and health-related quality of life. A sample size of 250 patients will have in excess of 80% power to detect a hazard ratio of 0.62 at a minimum of 12 months, assuming 150 patients experience an event across all follow-up.

CONCLUSIONS

To date, 169 patients have been recruited from 21 National Health Service hospitals in the United Kingdom, with recruitment expected to complete in 2024.

REGISTRATION

URL: https://www.clinicaltrials.gov; Unique identifier: NCT05003817.

The development and implementation of the Northern Health lung cancer digital care pathway: a case study in service change

This case study details the approach utilised to establish an easy to use, accessible and sustainable method for routine collection of Patient Reported Outcome Measures for patients newly diagnosed with lung cancer. We sought to enhance communication with patients and their families, particularly around shared decision making, their quality of life and symptoms, as well as the impacts of their care or treatment. We detail the co-design methodology utilised with consumers and healthcare providers to develop and implement a multi-lingual, fully automated digital care pathway which has been proven to be highly impactful and powerful for both healthcare providers working within the service and consumers enrolled within the digital pathway. This innovative initiative has changed the practice of the lung cancer service across a health service. Furthermore, its success has evolved the organisational strategy, to embed 'Outcomes for Impact' across the health service.

Indian Society of Critical Care Medicine and Indian Association of Palliative Care Expert Consensus and Position Statements for End-of-life and Palliative Care in the Intensive Care Unit

End-of-life care (EOLC) exemplifies the joint mission of intensive and palliative care (PC) in their human-centeredness. The explosion of technological advances in medicine must be balanced with the culture of holistic care. Inevitably, it brings together the science and the art of medicine in their full expression. High-quality EOLC in the ICU is grounded in evidence, ethical principles, and professionalism within the framework of the Law. Expert professional statements over the last two decades in India were developed while the law was evolving. Recent landmark Supreme Court judgments have necessitated a review of the clinical pathway for EOLC outlined in the previous statements. Much empirical and interventional evidence has accumulated since the position statement in 2014. This iteration of the joint Indian Society of Critical Care Medicine-Indian Association of Palliative Care (ISCCM-IAPC) Position Statement for EOLC combines contemporary evidence, ethics, and law for decision support by the bedside in Indian ICUs.

How to cite this article

Mani RK, Bhatnagar S, Butola S, Gursahani R, Mehta D, Simha S, et al. Indian Society of Critical Care Medicine and Indian Association of Palliative Care Expert Consensus and Position Statements for End-of-life and Palliative Care in the Intensive Care Unit. Indian J Crit Care Med 2024;28(3):200-250.

Consensus methodology to investigate the crucial referral criteria to pharmacist-led counseling clinics in Makkah City

Aim

Identifying and prioritizing criteria for referring patients to a counseling clinic managed by hospital pharmacists in the tertiary care setting in Saudi Arabia (SA).

Method

A two-phase consensus Delphi methodological approach was adopted in this study. Data was collected from physicians and pharmacists from different specialties working in different hospitals in Makkah City. In Phase 1, semi-structured interviews were conducted with physicians and pharmacists to discuss and develop the initial list of potential referral criteria for post-discharge counseling. Phase 2 consisted of two rounds of online surveys where participants were asked to independently rank the referral criteria using a 5-point Likert Scale.

Results

In Phase 1, four participants undertook the interviews (two physicians and two pharmacists). Overall, no major comments were given on the suggested criteria. In Phase 2, most suggested referral criteria to the counseling clinic reached participants' consensus agreement of >70 % in both rounds for all three domains. Among all criteria that achieved consensus agreement, two demographic criteria were top-ranked by the participants; the elderly patients (100 %) and those who needed help with their devices (96 %). These were followed by five medication-related criteria, which are medication-related problems, polypharmacy, medication that needs monitoring, high-risk medication, and medication with special formulations. All had a consensus agreement of 96 %.

Conclusion

This study suggests that a counseling clinic led by pharmacists is particularly advisable for the elderly, individuals requiring assistance with their devices, and those encountering medication issues. It is essential to prioritize specific patient demographics when contemplating the extensive establishment and integration of such clinics across various hospitals in SA.

Consensus for a primary care clinical decision-making tool for assessing, diagnosing, and managing low back pain in Alberta, Canada

BACKGROUND

Low back pain (LBP) is a common condition causing disability and high healthcare costs. Alberta faces challenges with unnecessary referrals to specialists and long wait times. A province-wide standardized clinical care pathway based on evidence-based best practices can improve efficiency, reduce wait times, and enhance patient outcomes. Implementing such pathways has shown success in other areas of healthcare in Alberta. This study developed a clinical decision-making pathway to standardize care and minimize uncertainty in assessment, diagnosis, and management.

METHODS

A systematic rapid review identified existing tools and evidence that could support a comprehensive LBP clinical decision-making tool. Forty-seven healthcare professionals participated in four rounds of a modified Delphi approach to reach consensus on the assessment, diagnosis, and management of patients presenting to primary care with LBP in Alberta, Canada. This project was a collaborative effort between Alberta Health Services' Bone and Joint Health Strategic Clinical Network (BJHSCN) and the Alberta Bone and Joint Health Institute (ABJHI).

RESULTS

A province-wide expert panel consisting of professionals from different health disciplines and regions collaborated to develop an LBP clinical decision-making tool. This tool presents clinical care pathways for acute, subacute, and chronic LBP. It also provides guidance for history-taking, physical examination, patient education, and management.

CONCLUSIONS

This clinical decision-making tool will help to standardize care, provide guidance on the diagnosis and management of LBP, and assist in clinical decision-making for primary care providers in both public and private sectors.

Social support systems involved in suicide prevention and intervention among adolescents: A Delphi study in Shanghai

The rotes of attempted and completed suicide among adolescents are increasing globally. Social support can help decrease the risk of adolescent suicide, but this aspect has been rarely studied in China. The present study aimed to use the Delphi methodology to establish a set of guidelines for the development of social support resources, with the goal of collectively reducing suicide risks among adolescents in Shanghai. We commenced the study in April 2021, established a research team, searched the keywords using Web of Science from 2016 to 2021 and ultimately designed a pre-evaluation index. Next, following Donabedian's Structure-Process-Outcome model, the research developed a questionnaire comprising 3 level-1, 12 level-2, and 73 level-3 indicators. Ten experts were enlisted to conduct three rounds of e-mail inquiries in order to finalize the indicator system, resulting in 2 level-1, 11 level-2, and 52 level-3 indicators, as well as expert consensus. Our findings indicate that the social support systems should include parents, schools, psychiatric hospitals, social organizations, and government departments, with the government sector being the most important ((M = 9.4). Furthermore, our study revealed that school counselors and psychologists play similar roles to psychiatrists within the interdisciplinary team (M ± S = 9.2 ± 1.1). As per the expert consensus, social support systems should strengthen government-led and interdisciplinary collaboration, prioritize suicide prevention in schools and encourage greater involvement from social organizations.

Developing an Emergency Medicine assessment tool for medical interns within the South African context - A Modified Delphi Study

Introduction

Emergency Medicine is a relatively new specialty in South Africa. Limited data is available regarding junior doctors' competence in managing emergencies, however previous surveys have identified limited teaching and supervision of junior doctors in Emergency Medicine. Currently there is no formal standardised assessment tool to assess an intern's Emergency Medicine competence. The aim of the study was to, through expert opinion and consensus, develop an Emergency Medicine assessment tool to identify a level of appropriate Emergency Medicine knowledge at the end of internship.

Methods

The Modified Delphi Methodology was used to create an assessment tool via interaction with a panel of experts and took place over 4 rounds via an online survey platform. The initial round identified the high-priority topics within each intern domain. A questionnaire was created based on these topics and was presented to the panel for consensus during the following round/s. Rounds continued until each question met consensus of 75 %.

Results

A total of 35 panellists consented to participate, representing 6 provinces. The majority were Emergency Medicine specialists. High-priority topics included acute respiratory distress, polytrauma, dehydration and shock in children, airway management, and the agitated patient. A 40-question, multiple choice questionnaire was created with all questions reaching consensus.

Conclusion

This study highlighted the core high-priority Emergency Medicine topics that interns should be exposed to during their internship and created a questionnaire aimed at evaluating them. The study findings provide a novel contribution to identifying gaps in Emergency Medicine knowledge during intern training, allowing for potential interventions to be implemented to improve intern EM training. The addition of a clinical skills component and increasing the question database is suggested to further develop this tool. Larger iterative studies involving the HPCSA, and health education experts provide avenues for future research.

Family-Centered Early Intervention Deaf/Hard of Hearing (FCEI-DHH): Methods

This is the fourth article in a series of eight that comprise a special issue on family-centered early intervention (FCEI) for children who are deaf or hard of hearing (DHH) and their families, FCEI-DHH. This article describes the co-production team and the consensus review method used to direct the creation of the 10 Principles described in this special issue. Co-production is increasingly being used to produce evidence that is useful, usable, and used. A draft set of 10 Principles for FCEI-DHH and associated Tables of recommended behaviors were developed using the knowledge creation process. Principles were refined through two rounds of eDelphi review. Results for each round were analyzed using measures of overall group agreement and measures that indicated the extent to which the group members agreed with each other. After Round 2, with strong agreement and low to moderate variation in extent of agreement, consensus was obtained for the 10 Principles for FCEI-DHH presented in this special issue. This work can be used to enhance evolution of FCEI-DHH program/services and systems world-wide and adds to knowledge in improvement science.

Adapting the Donabedian model in undergraduate nursing education: a modified Delphi study

BACKGROUND

Donabedian conceptual and multidimensional framework has been used in several studies in an educational context. In this study, we aimed to adapt the Donabedian three-component model (structure, process, and outcome) in undergraduate nursing education. This conceptual framework provides a comprehensive image of nursing education institutions and can help to evaluate institutions by measuring different aspects of their performance. A comprehensive understanding of the various elements of an educational institution helps to develop a complete, appropriate relevant set of performance indicators.

METHODS

This was a modified Delphi study. It had three rounds. The expert panel consisted of nursing faculty members and nursing Ph.D. students. In the first round, a questionnaire was designed based on interviews, focus groups, and a literature review. Experts rated their agreement with each element on a 5-point Likert scale in rounds two and three. The consensus level was set as 75%. The stability between rounds was also determined by calculating kappa coefficients. One Sample T-Test was also calculated for new items in round three.

RESULTS

All 55 items of the questionnaire were confirmed in the second round based on the consensus percentage of 75. Five new items were added to the third round based on comments in round two. Eventually, all elements except one were confirmed according to the consensus level, kappa values, means, and One-Sample T-Test in round three. The structure's key elements include staff (academic and non-academic); equipment; guidelines; resources and facilities; and students' demographics and characteristics. Process key elements include communication; education; evaluation; cooperation; and consultation. Outcome key elements include knowledge development; nursing image; alumni's outcome; students' outcome; related medical centers' performance; accreditation and evaluation results; and satisfaction.

CONCLUSIONS

Different elements of a nursing education institution at the bachelor's level were determined. The results of this study can help related bodies to develop and implement a comprehensive and systematic evaluation. These results can also be a basis for making this model useful in other nursing courses or education in other fields.

Understanding metric-related pitfalls in image analysis validation

Validation metrics are key for the reliable tracking of scientific progress and for bridging the current chasm between artificial intelligence (AI) research and its translation into practice. However, increasing evidence shows that particularly in image analysis, metrics are often chosen inadequately in relation to the underlying research problem. This could be attributed to a lack of accessibility of metric-related knowledge: While taking into account the individual strengths, weaknesses, and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multi-stage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides the first reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Focusing on biomedical image analysis but with the potential of transfer to other fields, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. To facilitate comprehension, illustrations and specific examples accompany each pitfall. As a structured body of information accessible to researchers of all levels of expertise, this work enhances global comprehension of a key topic in image analysis validation.

The role of ataluren in the treatment of ambulatory and non-ambulatory children with nonsense mutation duchenne muscular dystrophy - a consensus derived using a modified Delphi methodology in Eastern Europe, Greece, Israel and Sweden

BACKGROUND

This paper details the results of an evaluation of the level of consensus amongst clinicians on the use of ataluren in both ambulatory and non-ambulatory patients with nonsense mutation Duchenne muscular dystrophy (nmDMD). The consensus was derived using a modified Delphi methodology that involved an exploration phase and then an evaluation phase.

METHODS

The exploration phase involved 90-minute virtual 1:1 interviews of 12 paediatric neurologists who cared for 30-120 DMD patients each and had patient contact every one or two weeks. The respondents managed one to ten nmDMD patients taking ataluren. The Discussion Guide for the interviews can be viewed as Appendix A. Following the exploration phase interviews, the interview transcripts were analysed by an independent party to identify common themes, views and opinions and developed 43 draft statements that the Steering Group (authors) reviewed, refined and endorsed a final list of 42 statements. Details of the recruitment of participants for the exploration and evaluation phases can be found under the Methods section.

RESULTS

A consensus was agreed (> 66% of respondents agreeing) for 41 of the 42 statements using results from a consensus survey of healthcare professionals (n = 20) experienced in the treatment of nmDMD.

CONCLUSIONS

The statements with a high consensus suggest that treatment with ataluren should be initiated as soon as possible to delay disease progression and allow patients to remain ambulatory for as long as possible. Ataluren is indicated for the treatment of Duchenne muscular dystrophy that results from a nonsense mutation in the dystrophin gene, in ambulatory patients aged 2 years and older (see Summary of Product Characteristics for each country).

Quality indicators for collaborative care networks in persistent somatic symptoms and functional disorders: a modified delphi study

BACKGROUND

Care for persistent somatic symptoms and functional disorders (PSS/FD) is often fragmented. Collaborative care networks (CCNs) may improve care quality for PSS/FD. Effectiveness likely depends on their functioning, but we lack a straightforward quality evaluation system. We therefore aimed to develop quality indicators to evaluate CCNs for PSS/FD.

METHOD

Using an online three-round modified Delphi process, an expert panel provided, selected and ranked quality indicators for CCNs in PSS/FD. Recruited experts were diverse healthcare professionals with relevant experience in PSS/FD care in the Netherlands.

RESULTS

The expert panel consisted of 86 professionals representing 15 disciplines, most commonly physiotherapists, psychologists and medical specialists. 58% had more than 10 years experience in PSS/FD care. Round one resulted in 994 quotations, which resulted in 46 unique quality indicators. These were prioritised in round two and ranked in round three by the panel, resulting in a final top ten. The top three indicators were: "shared vision of care for PSS/FD", "pathways tailored to the individual patient", and "sufficiently-experienced caregivers for PSS/FD".

CONCLUSIONS

The identified quality indicators to evaluate CCNs in the field of PSS/FD can be implemented in clinical practice and may be useful in improving services and when assessing effectiveness.

Finnish experts' perceptions of digital healthcare forms in 2035 and the anticipated healthcare workforce impacts: a Delphi study

PURPOSE

To explore Finnish experts' perceptions of the forms of digital healthcare that are anticipated to be the most utilised in healthcare in the medium-term future (year 2035) and anticipated healthcare workforce impacts those forms will have.

DESIGN/METHODOLOGY/APPROACH

A total of 17 experts representing relevant interest groups participated in a biphasic online Delphi study. The results for each round were analysed using descriptive statistical methods and inductive content analysis.

FINDINGS

The forms of digital healthcare that the experts perceived as most likely to be utilised were those enabling patient participation, efficient organisation of services and automated data collection and analysis. The main impacts on the healthcare workforce were seen as being the redirection of workforce needs within the healthcare sector and need for new skills and new professions. The decrease in the need for a healthcare workforce was seen as less likely. The impacts were perceived as being constructed through three means: impacts within healthcare organisations, impacts on healthcare professions and impacts via patients.

RESEARCH LIMITATIONS/IMPLICATIONS

The results are not necessarily transferable to other contexts because the experts anticipated local futures. Patients' views were also excluded from the study.

ORIGINALITY/VALUE

Healthcare organisations function in complex systems where drivers, such as regional demographics, legislation and financial constraints, dictate how digital healthcare is utilised. Anticipating the workforce effects of digital healthcare utilisation has received limited attention; the study adds to this discussion.

Professional Perspectives on Children's Health Assets: A Delphi Study

This study aims to describe a local community expert's perspective on the identification of and access to children's health assets and to gather proposals to promote children's health and well-being within their community. The health asset approach is essential for health promotion, and there is evidence of its benefits to individuals' or communities' health when this approach is observed. Children's health assets are gaining increasing interest, but the literature that captures the perception of professionals working with children is scarce. Qualitative research designed with Delphi methodology was carried out with the participation of 25 professionals working in a neighbourhood with children and families. The participants stated that this neighbourhood was a good environment for the healthy and happy growth of children but pointed out that there were inequities. They emphasised the importance of economic and physical security and feeling loved. The absolute best aspects of the neighbourhood according to these experts were its support networks, mutual help, educational and health services, and green spaces, and the most deficient aspects were the possibility of a hopeful future and emotional support within the family unit. Poverty and/or the scarcity of economic resources were identified as the main barriers to accessing health assets. Special difficulties in access to health for migrant and Roma children were also identified. The panel of experts made concrete action proposals. It was recommended to support resources and services that already exist in their community. The experts prioritised work with families, education, working in conjunction with vulnerable groups, community participation, and networking.

Development of a novel dynamic nosocomial infection risk management method for COVID-19 in outpatient settings

BACKGROUND

Application of accumulated experience and management measures in the prevention and control of coronavirus disease 2019 (COVID-19) has generally depended on the subjective judgment of epidemic intensity, with the quality of prevention and control management being uneven. The present study was designed to develop a novel risk management system for COVID-19 infection in outpatients, with the ability to provide accurate and hierarchical control based on estimated risk of infection.

METHODS

Infection risk was estimated using an auto regressive integrated moving average model (ARIMA). Weekly surveillance data on influenza-like-illness (ILI) among outpatients at Xuanwu Hospital Capital Medical University and Baidu search data downloaded from the Baidu Index in 2021 and 22 were used to fit the ARIMA model. The ability of this model to estimate infection risk was evaluated by determining the mean absolute percentage error (MAPE), with a Delphi process used to build consensus on hierarchical infection control measures. COVID-19 control measures were selected by reviewing published regulations, papers and guidelines. Recommendations for surface sterilization and personal protection were determined for low and high risk periods, with these recommendations implemented based on predicted results.

RESULTS

The ARIMA model produced exact estimates for both the ILI and search engine data. The MAPEs of 20-week rolling forecasts for these datasets were 13.65% and 8.04%, respectively. Based on these two risk levels, the hierarchical infection prevention methods provided guidelines for personal protection and disinfection. Criteria were also established for upgrading or downgrading infection prevention strategies based on ARIMA results.

CONCLUSION

These innovative methods, along with the ARIMA model, showed efficient infection protection for healthcare workers in close contact with COVID-19 infected patients, saving nearly 41% of the cost of maintaining high-level infection prevention measures and enhancing control of respiratory infections.

Drawing a parallel between the trend of confirmed COVID-19 deaths in the winters of 2022/2023 and 2023/2024 in Italy, with a prediction

We studied the weekly number and the growth/decline rates of COVID-19 deaths of the period from October 31, 2022, to February 9, 2023, in Italy. We found that the COVID-19 winter wave reached its peak during the three holiday weeks from December 16, 2022, to January 5, 2023, and it was definitely trending downward, returning to the same number of deaths as the end of October 2022, in the first week February 2023. During this period of 15 weeks, that wave caused a number of deaths as large as 8,526. Its average growth rate was +7.89% deaths per week (10 weeks), while the average weekly decline rate was -15.85% (5 weeks). At the time of writing of this paper, Italy has been experiencing a new COVID-19 wave, with the latest 7 weekly bulletins (October 26, 2023 - December 13, 2023) showing that deaths have climbed from 148 to 322. The weekly growth rate had risen by +14.08% deaths, on average. Hypothesizing that this 2023/2024 wave will have a total duration similar to that of 2022/2023, with comparable extensions of both the growth period and the decline period and similar growth/decline rates, we predict that the number of COVID-19 deaths of the period from the end of October 2023 to the beginning of February 2024 should be less than 4100. A preliminary assessment of this forecast, based on 11 of the 15 weeks of the period, has already confirmed the accuracy of this approach.

Development of clinically meaningful quality indicators for contemporary lung cancer care, and piloting and evaluation in a retrospective cohort; experiences of the Embedding Research (and Evidence) in Cancer Healthcare (EnRICH) Program

OBJECTIVES

Lung cancer continues to be the most common cause of cancer-related death and the leading cause of morbidity and burden of disease across Australia. There is an ongoing need to identify and reduce unwarranted clinical variation that may contribute to these poor outcomes for patients with lung cancer. An Australian national strategy acknowledges clinical quality outcome data as a critical component of a continuously improving healthcare system but there is a need to ensure clinical quality indicators adequately measure evidence-based contemporary care, including novel and emerging treatments. This study aimed to develop a suite of lung cancer-specific, evidence-based, clinically acceptable quality indicators to measure quality of care and outcomes, and an associated comparative feedback dashboard to provide performance data to clinicians and hospital administrators.

DESIGN

A multistage modified Delphi process was undertaken with a Clinical Advisory Group of multidisciplinary lung cancer specialists, with patient representation, to update and prioritise potential indicators of lung cancer care derived from a targeted review of published literature and reports from national and international lung cancer quality registries. Quality indicators were piloted and evaluated with multidisciplinary teams in a retrospective observational cohort study using clinical audit data from the Embedding Research (and Evidence) in Cancer Healthcare Program, a prospective clinical cohort of over 2000 patients with lung cancer diagnosed from May 2016 to October 2021.

SETTING AND PARTICIPANTS

Six tertiary specialist cancer centres in metropolitan and regional New South Wales, Australia.

RESULTS

From an initial 37 potential quality indicators, a final set of 10 indicators spanning diagnostic, treatment, quality of life and survival domains was agreed.

CONCLUSIONS

These indicators build on and update previously available measures of lung cancer care and outcomes in use by national and international lung cancer clinical quality registries which, to our knowledge, have not been recently updated to reflect the changing lung cancer treatment paradigm.

Construction and validation of an instrument for evaluating Lean Healthcare in healthcare institutions

OBJECTIVE

To build and validate an instrument to evaluate Lean Healthcare in healthcare institutions.

METHOD

Methodological study conducted in three stages: 1) Instrument construction; 2) Content validity using the Delphi technique with 14 experts; and 3) Construct validation using Structural Equation Modeling with sample consisted of 113 professionals with experience in Lean Healthcare. Data collection carried out from October/2020 to January/2021 using a digital form. Data analysis performed with the SmartPLS2.0/M3 software.

RESULTS

Items were developed after an integrative review and divided into the dimensions Structure, Process and Outcome, according to Donabedian's theoretical framework. Content validation in two rounds of the Delphi technique. Final instrument, after model adjustment, containing 16 items with Cronbach's alpha of 0.77 in Structure, 0.71 in Process and 0.83 in Outcome.

CONCLUSION

The instrument presented evidence of validity and reliability, enabling its use in healthcare institutions to evaluate Lean Healthcare.

Treatment for central centrifugal cicatricial alopecia-Delphi consensus recommendations

BACKGROUND

There is no established standard of care for treating central centrifugal cicatricial alopecia (CCCA), and treatment approaches vary widely.

OBJECTIVE

To develop consensus statements regarding the use of various pharmacological therapies in treating adults with CCCA.

METHODS

We invited 27 dermatologists with expertise in hair and scalp disorders to participate in a 3-round modified Delphi study between January and March 2023. Statements met strong consensus if 75% of respondents agreed or disagreed. Statements met moderate consensus if 55% or more but less than 75% agreed or disagreed.

RESULTS

In round 1, 5 of 33 (15.2%) statements met strong consensus, followed by 9 of 28 (32.1%) in round 2. After the final round 3 meeting, strong consensus was reached for 20 of 70 (28.6%) overall statements. Two statements achieved moderate consensus.

LIMITATIONS

This study included only English-speaking, US-based dermatologists and did not consider nonpharmacological therapies.

CONCLUSION

Despite varying opinions among dermatologists, consensus was reached for several statements to help clinicians manage CCCA. We also highlight areas that lack expert consensus with the goal of advancing research and therapeutic options for CCCA.

PRIME-nurse practitioner competency model validation and criterion based OSCE rubric interrater reliability

The PRIME-NP (Professional-Reporter-Interpreter-Manager-Educator/Evaluation-Nurse Practitioner) Model is adapted from the RIME (Reporter-Interpreter-Manager-Educator) model used in medical education to guide medical student and resident education. The Delphi technique was used to validate the PRIME-NP Model. After two rounds of review by a group of experts in NP curriculum, the model was determined to be valid based on expert consensus. Agreement percent increase from the first round to the second round in all categories. Interrater reliability (IRR) was assessed using interclass correlation after instrument validation was completed for each of the five levels of the PRIME-NP model. Overall, the IRR of the instrument was found to be acceptable with some notable exceptions. No variance was noted in professional behaviors at any level. Variance was increased in management and educator/evaluator behaviors in higher/later course levels. The PRIME-NP Model and PRIME-NP OSCE Rubric is a valid and reliable instrument to assess NP student progression in objective structured clinical examinations. This instrument has the potential for adaptation for use in other types of health sciences education and settings.

Are military fitness tests safe for members with a total hip arthroplasty?

INTRODUCTION

Canadian Armed Forces (CAF) members must complete an annual fitness evaluation. Members with a total hip arthroplasty (THA) may be at risk of injury during these strenuous tests. To inform CAF policy, we sought expert consensus on the safety of fitness testing for members with a THA.

METHODS

We conducted a three-round Delphi study with a panel of hip arthroplasty experts to determine the safety of CAF operational fitness evaluations for members with a THA. The experts evaluated videos of the 10 individual tasks included in the evaluations.

RESULTS

All individual tasks were judged to be safe by consensus. One task, which involves digging with a shovel, was considered safe provided that participants avoid deep hip flexion. The nine other tasks were judged to be safe without modifications or interventions. The experts also supported a policy recommendation that would allow members to perform military fitness evaluations if they (1) have a primary THA, (2) had no episodes of instability, (3) are at least 12 months postoperatively and (4) have been cleared by an orthopaedic surgeon and a physiatrist/physiotherapist.

CONCLUSION

A panel of arthroplasty experts concluded, based on video analysis, that CAF fitness evaluations are generally safe for members with a THA.

The Aging Care 5Ms Competencies: A Modified Delphi Study to Revise Medical Student Competencies for the Care of Older Adults

PURPOSE

To revise the 2009 Canadian Geriatrics Society (CGS) Core Competencies in the Care of Older Persons for Canadian Medical Students by applying current frameworks and using a modified Delphi process.

METHOD

The working group chose the Geriatric 5Ms model and CanMEDS framework to develop and structure the competencies. National (i.e., Canadian) Delphi participants were recruited, and 3 Delphi survey rounds were conducted from 2019 to 2021. Each survey round collected quantitative data using a 7-point Likert scale (LS) and qualitative data using free-text comments. The purpose of the first round was to establish the importance of the components of the proposed competencies (categorized into 13 subsections) and identify additional themes. The second round assessed agreement with 31 proposed competencies organized into 7 themes: aging, caring for older adults, mind, mobility, medications, multicomplexity, and matters the most. The third survey-rated agreement levels after further revisions to the competencies were applied. The final 33 competencies were shared with survey participants for feedback and other stakeholders for external validation.

RESULTS

Mean LSs for the importance of the 13 competency component subsections on the first survey varied from 5.11 to 6.54, with an agreement level of 73%-93%. New themes emerged from the qualitative comments. Mean LSs for the 31 competencies on the second survey ranged from 5.57 to 6.81, with an agreement level of 80%-97%. Mean LSs for the revised competencies on the third survey ranged from 5.83 to 6.65, with an agreement level of 83%-95%.

CONCLUSIONS

The authors developed the 33 Aging Care 5Ms Competencies for Canadian medical students using a consensus process. The competencies fulfill an important need in medical education, and ultimately, society. The authors strongly believe that the competencies can be woven into existing undergraduate medical curricula through purposeful integration and collaboration, including with other specialties.

Development of entrustable professional activities for emergency medicine medical education fellowships: A modified Delphi study

Background

It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows.

Methods

From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics.

Results

Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration.

Conclusions

We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd.

Multinational Association of Supportive Care in Cancer (MASCC) clinical practice guidance for the prevention of breast cancer-related arm lymphoedema (BCRAL): international Delphi consensus-based recommendations

Background

Developing strategies to prevent breast cancer-related arm lymphoedema (BCRAL) is a critical unmet need because there are no effective interventions to eradicate it once it reaches a chronic state. Certain strategies such as prospective surveillance programs and prophylactic lymphatic reconstruction have been reported to be effective in clinical trials. However, a large variation exists in practice based on clinician preference, organizational standards, and local resources.

Methods

A two-round international Delphi consensus process was performed from February 27, 2023 to May 25, 2023 to compile opinions of 55 experts involved in the care and research of breast cancer and lymphoedema on such interventions.

Findings

Axillary lymph node dissection, use of post-operative radiotherapy, relative within-arm volume increase one month after surgery, greater number of lymph nodes dissected, and high body mass index were recommended as the most important risk factors to guide selection of patients for interventions to prevent BCRAL. The panel recommended that prospective surveillance programs should be implemented to screen for and reduce risks of BCRAL where feasible and resources allow. Prophylactic compression sleeves, axillary reverse mapping and prophylactic lymphatic reconstruction should be offered for patients who are at risk for developing BCRAL as options where expertise is available and resources allow. Recommendations on axillary management in clinical T1-2, node negative breast cancer patients with 1-2 positive sentinel lymph nodes were also provided by the expert panel. Routine axillary lymph node dissection should not be offered in these patients who receive breast conservation therapy. Axillary radiation instead of axillary lymph node dissection should be considered in the same group of patients undergoing mastectomy.

Interpretation

An individualised approach based on patients' preferences, risk factors for BCRAL, availability of treatment options and expertise of the healthcare team is paramount to ensure patients at risk receive preventive interventions for BCRAL, regardless of where they are receiving care.

Funding

This study was not supported by any funding. RJC received investigator grant support from the Australian National Health and Medical Research Council (APP1194051).

Examining the Relationship Between Value and Patient Satisfaction With Treatment in Total Joint Arthroplasty

Background

A shift toward performance, cost, outcomes, and patient satisfaction has occurred with healthcare reform promoting value-based programs. The purpose of this study was to evaluate the relationship between patient satisfaction and value with treatment in a cohort of patients undergoing total knee arthroplasty (TKA) and total hip arthroplasty (THA).

Methods

Value was determined by the relationship of treatment outcome with episodic cost. Measurements included both clinical outcomes and patient-reported outcomes. Participating surgeons took part in the modified Delphi method resulting in an algorithm measuring patient value. Treatment outcome, cost, and resultant value (outcome/cost) of both TKA and THA were evaluated using binomial logistic regression by adjusting for age, gender, body mass index, Charlson comorbidity index, tobacco, education, and income with patient-reported satisfaction as the outcome.

Results

This study had a total of 909 patients (TKA n = 438; THA n = 471), with an average age of 67 (TKA) and 65 (THA) years. Patient satisfaction shared a significant positive relationship with treatment outcome for TKA (odds ratio [OR] = 1.53, confidence interval [CI] = 1.35-1.73, P < .001) and THA (OR = 1.93, CI = 1.62-2.29, P < .001). Higher value was associated with a significantly higher odds of patient satisfaction for both TKA (OR = 1.39, CI = 1.25-1.54, P < .001) and THA (OR = 1.70, CI = 1.47-1.97, P < .001).

Conclusions

This study showed a positive relationship between treatment outcome but not cost with subsequent value and patient satisfaction. This method provides a promising approach to comprehensively evaluate outcomes, cost, and value of total joint arthroplasty procedures. This approach can help predict the probability of value-driven patient satisfaction.

Barriers and solutions for the management of severe hypoglycaemia in people with diabetes in Spain: A Delphi survey

BACKGROUND AND OBJECTIVE

Severe hypoglycaemia (SH) imposes a significant burden for people with diabetes (PwD), their caregivers (CGs), and the healthcare system. The study aimed to identify barriers and solutions in the management of SH in PwD in Spain, gathering consensus from physicians and nurses.

MATERIAL AND METHODS

Expert opinion from physicians and nurses who manage PwD was collected via a 2-round online Delphi method. Consensus was predefined as ≥ 70% of the panellists agreeing or disagreeing with the statement.

RESULTS

Physicians (n = 25) and nurses (n = 17) reached ≥ 90% consensus on the following barriers for the management of SH: absence of symptoms, cost to the health system, lack of implementation of glucose monitoring devices, lack of patient training to identify and manage SH, and the fear of SH in children and CGs. Main solutions, identified with ≥ 70% consensus, included training, education, and psychological support using diabetes nurse educators and the use of new glucose monitoring technologies and applications.

CONCLUSIONS

This study provides valuable insights on the barriers and solutions in the management of SH in Spain. Structured self-management training, the support of diabetes educators, and the use of insulin delivery devices and glucose monitoring technologies is required for the management of SH.

Structured request form in musculoskeletal radiology examinations (CONCERTO): results of an expert Delphi consensus-structured radiology request form for correct classification of patients to undergo radiological examinations of the Italian Society of Medical and Interventional Radiology (SIRM), the Italian Society of Rheumatology (SIR) and the Italian Society of Orthopedics and Traumatology (SIOT)

PURPOSE

To describe a Delphi consensus for the realization of a structured radiology request form for patients undergoing musculoskeletal imaging.

METHODS

A steering committee (four radiologists, a rheumatologist and an orthopedic surgeon) proposed a form to an expert panel (30 members, ten radiologists, ten rheumatologists and ten orthopedic surgeons). Through an online survey, the panelists voted on their level of agreement with the statements of the form using a 10-point Likert scale (1: no agreement; 10: total agreement) in a three-round process. A combination of two distinct criteria, a mean agreement level ≥ 8 and a percentage of at least 75% of responses with a value ≥ 8, was deemed as acceptable.

RESULTS

The form achieved high median ratings in all the assessed key features. During the first round, all items met the threshold to be advanced as unmodified in the next round. Additional proposed items were considered and introduced in the next round (six items in Section 1, five items in Section 2, ten items in Section 3, 11 items in Section 4, six items in Section 5, eight items in Section 6, ten items in Section 7 and eight items in Section 8). Of these items, in round 3, only six reached the threshold to be integrated into the final form.

CONCLUSIONS

Implementation of a structured radiology request form can improve appropriateness and collaboration between clinicians and radiologists in musculoskeletal imaging.

Understanding metric-related pitfalls in image analysis validation

Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.

Identifying priority gaps in contextual factors research and force-based manipulation. An international and interdisciplinary Delphi study

OBJECTIVE

To establish priority gaps related to contextual factors (CFs) research and force-based manipulation (FBM).

METHODS

A three-round Delphi following recommended guidelines for conducting and reporting Delphi studies (CREDES) involving international and interdisciplinary panelists with expertise in CFs and FBM. Round 1 was structured around two prompting questions created by the workgroup. Ranking of each priority gap was done by calculating composite scores for each theme generated. Consensus threshold was set with an agreement ≥75% among panelists. Median and interquartile range were calculated for each priority gap to provide the central tendency of responses. Wilcoxon rank-sum test was used to evaluate the consistency and stability of responses between rounds 2 and 3.

RESULTS

Forty-six panelists participated in all three rounds of the Delphi. Consensus was reached for 16 of 19 generated themes for priority gaps in CFs research and FBM. The ranking of each identified gap was computed and presented. Wilcoxon rank-sum test was non-significant (P > .05), demonstrating consistency and stability of results between rounds.

CONCLUSION

The result of this Delphi provides international and interdisciplinary consensus-based priority gaps in CFs research and FBM. The gaps identified can be used to generate future research inquiries involving CFs research and FBM.

Return to Sports Following Sports-Related Concussion in Collision Sports: An Expert Consensus Statement Using the Modified Delphi Technique

PURPOSE

To perform a Delphi consensus for return to sports (RTS) following sports-related concussion (SRC).

METHODS

Open-ended questions in rounds 1 and 2 were answered. The results of the first 2 rounds were used to develop a Likert-style questionnaire for round 3. If agreement at round 3 was ≤80% for an item, if panel members were outside consensus or there were >30% neither agree/disagree responses, the results were carried forward into round 4. The level of agreement and consensus was defined as 90%.

RESULTS

Individualized graduated RTS protocols should be used. A normal clinical, ocular and balance examination with no more headaches, and asymptomatic exertional test allows RTS. Earlier RTS can be considered if athletes are symptom free. The Sports Concussion Assessment Tool 5 and vestibular and ocular motor screening are recognized as useful tools to assist in decision-making. Ultimately RTS is a clinical decision. Baseline assessments should be performed at both collegiate and professional level and a combination of neurocognitive and clinical tests should be used. A specific number of recurrent concussions for season-or career-ending decisions could not be determined but will affect decision making for RTS.

CONCLUSIONS

Consensus was achieved for 10 of the 25 RTS criteria: early RTS can be considered earlier than 48 to 72 hours if athletes are completely symptom-free with no headaches, a normal clinical, ocular and balance examination. A graduated RTS should be used but should be individualized. Only 2 of the 9 assessment tools were considered to be useful: Sports Concussion Assessment Tool 5 and vestibular and ocular motor screening. RTS is mainly a clinical decision. Only 31% of the baseline assessment items achieved consensus: baseline assessments should be performed at collegiate and professional levels using a combination of neurocognitive and clinical tests. The panel disagreed on the number of recurrent concussions that should be season- or career-ending.

LEVEL OF EVIDENCE

Level V, expert Opinion.

On-Field and Pitch-Side (Sideline) Assessment of Sports Concussion in Collision Sports: An Expert Consensus Statement Using the Modified Delphi Technique

PURPOSE

To perform a Delphi consensus for on-field and pitch-side assessment of sports-related concussion (SRC).

METHODS

Open-ended questions in rounds 1 and 2 were answered. The results of the first 2 rounds were used to develop a Likert-style questionnaire for round 3. If agreement at round 3 was ≤80% for an item, if panel members were outside consensus, or there were >30% neither agree/disagree responses, the results were carried forward into round 4. The level of agreement and consensus was defined as 90%.

RESULTS

Loss of consciousness (LOC) or suspected LOC, motor incoordination/ataxia, balance disturbance, confusion/disorientation, memory disturbance/amnesia, blurred vision/light sensitivity, irritability, slurred speech, slow reaction time, lying motionless, dizziness, headaches/pressure in the head, falling to the ground with no protective action, slow to get up after a hit, dazed look, and posturing/seizures were clinical signs of SRC and indicate removal from play. Video assessment is helpful but should not replace clinical judgment. LOC/unresponsiveness, signs of cervical spine injury, suspicion of other fractures (skull/maxillo-facial), seizures, Glasgow Coma Scale score <14 and abnormal neurologic examination findings are indications for hospitalization. Return to play should only be considered when no clinical signs of SRC are present. Every suspected concussion should be referred to an experienced physician.

CONCLUSIONS

Consensus was achieved for 85% of the clinical signs indicating concussion. On-field and pitch-side assessment should include the observation of the mechanism, a clinical examination, and cervical spine assessment. Of the 19 signs and red flags requiring removal from play, consensus was reached for 74%. Normal clinical examination and HIA with no signs of concussion allow return to play. Video assessment should be mandatory for professional games but should not replace clinical decision-making. Sports Concussion Assessment Tool, Glasgow Coma Scale, vestibular/ocular motor screening, Head Injury Assessment Criteria 1, and Maddocks questions are useful tools. Guidelines are helpful for non-health professionals.

LEVEL OF EVIDENCE

Level V, expert opinion.