Delphi methodology in healthcare research: How to decide its appropriateness

Development of the WHO eye care competency framework

BACKGROUND

The eye care workforce, particularly in lower resource settings, face challenges of limited integration into the health system, limited workforce capacity, mismatch of workforce to population need and poor quality of care. In recognition of these challenges, coupled with a gap in existing tools, provides a strong rationale for the development of the Eye care competency framework (ECCF).

METHODS

A mixed methods approach was utilised to develop and validate the ECCF. Content was developed by extracting relevant components of existing frameworks used both within and outside of eye care. A diverse technical working group provided feedback and guidance on the structure, design, and content to create a preliminary draft. Competencies and activities were validated using a modified-Delphi study, and the framework was then piloted at four sites to understand how the tool can be implemented in different settings.

RESULTS

The final version of the ECCF included eight outcomes, nine guiding principles, and content of each of the key elements, including the six domains, 22 competencies, 21 activities, 193 behaviours and 234 tasks, and the knowledge and skills that underpin them. 95/112 participants from the six WHO regions completed the modified-Delphi study, yielding an average of 96% agreement across the competencies and activities in the ECCF. The pilot showcased the versatility and flexibility of the ECCF, where each of the four sites had a different experience in implementing the ECCF. All sites found that the ECCF enabled them to identify gaps within their current workforce documentation.

CONCLUSIONS

The ECCF was developed using a collaborative approach, reflecting the opinions of participants and stakeholders from all around the world. The comprehensive competencies and activities developed in the ECCF encompass the diverse roles of eye care workers, and thus encourage multi-disciplinary care and better integration into the health system. It is recommended that eye care workforce planners and developers use the ECCF, and adapt it to their context, to support workforce development and focus on the quality and scope of eye care service provision.

Construction of a Form for Users of the Child Welfare System Based on the Delphi Method

Professionals in charge of designing individualized plans for children and adolescents in the child welfare system often lack the necessary information, either because it has not been systematically collected or because there are doubts about the reliability of the data obtained. The lack of consensual and validated instruments that gather the necessary information has led to the development of a rigorous and effective form, based on the Delphi methodology, aimed at obtaining an exhaustive knowledge of the characteristics of children and adolescents under the child welfare system. Once a consensus of different specialists approved the hetero-informed form, it was completed by 41 professionals working in residential care facilities for 307 children and adolescents. It consists of 66 items grouped into six dimensions: general information, school/work situation, child welfare system history, family visitation history, biological family information, and experiences of sexual abuse. During its construction and validation, a panel of experts analyzed its format and content during the different phases. Most of the items showed good performance, and professionals highlighted their ease of use and relevance. The method used ensured the content validity of this form. This instrument has proven to be a useful and effective tool for collecting sociodemographic information on children and adolescents in the child welfare system, which may improve their conditions.

Practice of neuromonitoring in open and endovascular thoracoabdominal aortic repair-an international expert-based modified Delphi consensus study

OBJECTIVES

Spinal cord injury is detrimental for patients undergoing open or endovascular thoracoabdominal aortic aneurysm (TAAA) repair. The aim of this survey and of the modified Delphi consensus was to gather information on current practices and standards in neuroprotection in patients undergoing open and endovascular TAAA.

METHODS

The Aortic Association conducted an international online survey on neuromonitoring in open and endovascular TAAA repair. In a first round an expert panel put together a survey on different aspects of neuromonitoring. Based on the answers from the first round of the survey, 18 Delphi consensus questions were formulated.

RESULTS

A total of 56 physicians completed the survey. Of these, 45 perform open and endovascular TAAA repair, 3 do open TAAA repair and 8 do endovascular TAAA repair. At least 1 neuromonitoring or protection modality is utilized during open TAAA surgery. Cerebrospinal fluid (CSF) drainage was used in 97.9%, near infrared spectroscopy in 70.8% and motor evoked potentials or somatosensory evoked potentials in 60.4%. Three of 53 centres do not utilize any form of neuromonitoring or protection during endovascular TAAA repair: 92.5% use CSF drainage; 35.8%, cerebral or paravertebral near infrared spectroscopy; and 24.5% motor evoked potentials or somatosensory evoked potentials. The utilization of CSF drainage and neuromonitoring varies depending on the extent of the TAAA repair.

CONCLUSIONS

The results of this survey and of the Delphi consensus show that there is broad consensus on the importance of protecting the spinal cord to avoid spinal cord injury in patients undergoing open TAAA repair. Those measures are less frequently utilized in patients undergoing endovascular TAAA repair but should be considered, especially in patients who require extensive coverage of the thoracoabdominal aorta.

Delphi-Based Expert Consensus Statements for the Management of Percutaneous Radiofrequency Neurotomy in the Treatment of Lumbar Facet Joint Syndrome

INTRODUCTION

A modified Delphi strategy was implemented for obtaining recommendations that could be useful in the management of percutaneous radiofrequency treatment of lumbar facet joint syndrome, as the literature on the argument was poor in quality.

METHODS

An Italian research team conducted a comprehensive literature search, defined the investigation topics (diagnosis, treatment, and outcome evaluation), and developed an explorative semi-structured questionnaire. They also selected the members of the panel. After an online meeting with the participants, the board developed a structured questionnaire of 15 closed statements (round 1). A five-point Likert scale was used and the cut-off for consensus was established at a minimum of 70% of the number of respondents (level of agreement ≥ 4, agree or strongly agree). The statements without consensus were rephrased (round 2).

RESULTS

Forty-one clinicians were included in the panel and responded in both rounds. After the first round, consensus (≥ 70%) was obtained in 9 out of 15 statements. In the second round, only one out of six statements reached the threshold. The lack of consensus was observed for statements concerning the use of imaging for a diagnosis [54%, median 4, interquartile range (IQR) 3-5], number of diagnostic blocks (37%, median 4, IQR 2-4), bilateral denervation (59%, median 4, IQR 2-4), technique and number of lesions (66%, median 4, IQR 3-5), and strategy after denervation failure (68%, median 4, IQR 3-4).

CONCLUSION

Results of the Delphi investigations suggest that there is a need to define standardized protocols to address this clinical problem. This step is essential for designing high-quality studies and filling current gaps in scientific evidence.

Explicit potentially inappropriate medications criteria for older population in Asian countries: A systematic review

BACKGROUND

Explicit potentially inappropriate medications (PIM) criteria are commonly used to identify and deprescribe potentially inappropriate prescriptions among older patients. Most of these criteria were developed specifically for the Western population, which might not be applicable in an Asian setting. The current study summarizes the methods and drug lists to identify PIM in older Asian people.

METHODS

A systematic review of published and unpublished studies were carried out. Included studies described the development of explicit criteria for PIM use in older adults and provided a list of medications that should be considered inappropriate. PubMed, Medline, EMBASE, Cochrane CENTRAL, CINAHL, PsycINFO, and Scopus searches were conducted. The PIMs were analyzed according to the general conditions, disease-specific conditions, and drug-drug interaction classes. The qualities of the included studies were assessed using a nine-point evaluation tool. The kappa agreement index was used to evaluate the level of agreement between the identified explicit PIM tools.

RESULTS

The search yielded 1206 articles, and 15 studies were included in our analysis. Thirteen criteria were identified in East Asia and two in South Asia. Twelve out of the 15 criteria were developed using the Delphi method. We identified 283 PIMs independent of medical conditions and 465 disease-specific PIMs. Antipsychotics were included in most of the criteria (14/15), followed by tricyclic antidepressants (TCAs) (13/15), antihistamines (13/15), sulfonylureas (12/15), benzodiazepines (11/15), and nonsteroidal anti-inflammatory drug (NSAIDs) (11/15). Only one study fulfilled all the quality components. There was a low kappa agreement (k = 0.230) between the included studies.

CONCLUSION

This review included 15 explicit PIM criteria, which most listed antipsychotics, antidepressants, and antihistamines as potentially inappropriate. Healthcare professionals should exercise more caution when dealing with these medications among older patients. These results may help healthcare professionals in Asian nations to create regional standards for the discontinuation of potentially harmful drugs for elderly patients.

The Gender-Diversity and Autism Questionnaire: A Community-Developed Clinical, Research, and Self-Advocacy Tool for Autistic Transgender and Gender-Diverse Young Adults

Background

Autistic transgender people face unique risks in society, including inequities in accessing needed care and related mental health disparities. Given the need for specific and culturally responsive accommodations/supports, the characterization of key experiences, challenges, needs, and resilience factors within this population is imperative. This study developed a structured self-report tool for autistic transgender young adults to communicate their experiences and needs in a report format attuned to common autistic thinking and communication styles.

Methods

This cross-nation project developed and refined the Gender-Diversity and Autism Questionnaire through an iterative community-based approach using Delphi panel methodology. This proof-of-principle project defined "expertise" broadly, employing a multi-input expert search approach to balance academic-, community-, and lived experience-based expertise.

Results

The expert collaborators (N = 24 respondents) completed a two-round Delphi study, which developed 85 mostly closed-ended items based on 90% consensus. Final item content falls within six topic areas: the experience of identities; the impact of experienced or anticipated discrimination, bias, and violence toward autistic people and transgender people; tasks and experiences of everyday life; gender diversity- or autism-related care needs and history; the experience of others doubting an individual's gender identity and/or autism; and the experience of community and connectedness. The majority of retained items relate to tasks and experiences of everyday life or the impact of experienced or anticipated discrimination, bias, and violence.

Conclusions

This study employed a multipronged multimodal search approach to maximize equity in representation of the expert measure development team. The resulting instrument, designed for clinical, research, and self-advocacy applications, has parallel Dutch and English versions and is available for immediate use. Future cross-cultural research with this instrument could help identify contextual risk and resilience factors to better understand and address inequities faced by this large intersectional population.

The Development of Recommendations for Healthcare Providers to Support Patients Experiencing Medication Self-Management Problems

Medication self-management problems such as the inability to correctly obtain, understand, organize, administer or monitor medication can result in negative patient outcomes. However, supportive tools for healthcare providers to assist patients with medication self-management problems are lacking. This study aimed to develop recommendations for healthcare providers to support patients with polypharmacy who experience medication self-management problems. A three-phase study was conducted starting with (1) the mapping of medication self-management problems, followed by (2) a scoping review providing a list of relevant interventions and actions for each respective problem and (3) a three-round modified e-Delphi study with experts to reach consensus on the relevance and clarity of the recommended interventions and actions. The cut-off for consensus on the relevance and clarity of the recommendations was set at 80% expert agreement. Experts could propose additional recommendations based on their professional experience and expertise. The experts (n = 23) involved were healthcare professionals (i.e., nurses, pharmacists, and physicians) with specific expertise in medication management of patients with polypharmacy. Simultaneous with the second e-Delphi round, a panel of patients with polypharmacy (n = 8) evaluated the usefulness of recommendations. Results obtained from the patient panel were fed back to the panel of healthcare providers in the third e-Delphi round. Descriptive statistics were used for data analysis. Twenty medication self-management problems were identified. Based on the scoping review, a list of 66 recommendations for healthcare providers to support patients with the identified medication self-management problems was composed. At the end of the three-round e-Delphi study, the expert panel reached consensus on the relevance and clarity of 67 recommendations, clustered according to the six phases of the medication self-management model by Bailey et al. In conclusion, this study resulted in a guidance document including recommendations that can serve as a resource for healthcare providers to support patients with polypharmacy in case of medication self-management problems. Future research should focus on the evaluation of the feasibility and user-friendliness of the guide with recommendations in clinical practice.

Modernizing patient-centered manual therapy: Findings from a Delphi study on orthopaedic manual therapy application

BACKGROUND

Recent literature challenges the process by which orthopaedic manual therapy (OMT) has traditionally been applied. Progressive understanding of the complexities surrounding OMT analgesia and the decreased reliance on technique specific characteristics in determining treatment effectiveness promotes an update to training paradigms related to OMT.

OBJECTIVES

The purpose of this Delphi study was to establish consensus on what trainees should be focusing on when demonstrating OMT techniques and how candidates for OMT should be identified.

DESIGN

An international three-round Delphi study following recommended guidelines for conducting and reporting of Delphi studies (CREDES) was performed.

METHODS

One-hundred sixty-four expert manual therapy educators were identified for participation across four countries. Participants were asked to provide what concepts trainees should be focusing on when demonstrating OMT techniques and how candidates for OMT should be identified. Twenty-one themes were identified for each question.

RESULTS

Twenty-eight participants completed all three rounds of the Delphi. Consensus was reached on nineteen themes and eighteen themes respectively. Results from this Delphi stress patient-centered care within a biopsychosocial pain management model. Representation across all pillars of evidence-based practice were represented. Themes reaching consensus within this study favored the importance of neurophysiological, psychological, and biomechanical principles.

CONCLUSION

This Delphi presents consensus-based recommendations for what manual therapy trainees should focus on when demonstrating OMT techniques and on how candidates for OMT should be identified. These findings in collaboration with previous consensus recommendations on concepts to focus on within OMT education promote restructuring of OMT curriculum to evidence-based patient-centered care models.

A contextual definition of longitudinal integrated clerkships within the UK and Ireland: A bi-national modified Delphi study

Disagreement exists within the UK and Ireland regarding how Longitudinal Integrated Clerkships should be defined, and the relevance of international definitions. In this modified, online Delphi study, we presented the UK and Ireland experts in Longitudinal Integrated Clerkships with statements drawn from international definitions, published LIC literature, and the research team's experience in this area and asked them to rate their level of agreement with these statements for inclusion in a bi-national consensus definition. We undertook three rounds of the study to try and elicit consensus, making adaptations to statement wording following rounds 1 and 2 to capture participants' qualitative free text-comments, following the third and final round, nine statements were accepted by our panel, and constitute our proposed definition of Longitudinal Integrated Clerkships within the UK and Ireland. This definitional statement corresponds with some international literature but offers important distinctions, which account for the unique context of healthcare (particularly primary care) within the UK and Ireland (for example, the lack of time-based criteria within the definition). This definition should allow UK and Irish researchers to communicate more clearly with one another regarding the benefits of LICs and longitudinal learning and offers cross-national collaborative opportunities in LIC design, delivery and evaluation.

Defining ruxolitinib failure and transition to next-line therapy for patients with myelofibrosis: a modified Delphi panel consensus study

Aim: To define ruxolitinib failure and develop parameters to guide transition to next-line therapy for patients with myelofibrosis. Methods: A modified Delphi panel with 14 hematologists-oncologists. Survey concepts included defining primary refractory status, loss of response, disease progression, intolerance and transition to next-line therapy. Results: Ruxolitinib failure may be defined as no improvement in symptoms or spleen size, progressive disease or ruxolitinib intolerance, following a maximally tolerated dose for ≥3 months. Loss of spleen response 1 month after initial response may prompt discontinuation. Lack of evidence to inform transition to next-line therapy was noted; tapering ruxolitinib should be considered according to ruxolitinib dose and patient characteristics. Conclusion: Expert consensus was provided on defining ruxolitinib failure and transition to next-line therapy as summarized in this position paper, which may support considerations in the development of future clinical practice guidelines.

Measurement of Humanity Among Health Professionals: Development and Validation of the Medical Humanity Scale Using the Delphi Method

BACKGROUND

Despite the importance of humanism in providing health care, there is a lack of valid and reliable tool for assessing humanity among health professionals.

OBJECTIVE

The aim of this study was to design a new humanism scale and to assess the validity of this scale in measuring humanism among Syrian health professional students.

METHODS

The Medical Humanity Scale (MHS) was designed. It consists of 27 items categorized into 7 human values including patient-oriented care, respect, empathy, ethics, altruism, and compassion. The scale was tested for internal consistency and reliability using Cronbach α and test-retest methods. The construct validity of the scale was also tested to assess the ability of the scale in differentiating between groups of health professional students with different levels of medical humanity. A 7-point Likert scale was adopted. The study included 300 participants including 97 medical, 78 dental, 82 pharmacy, and 43 preparatory-year students from Syrian universities. The Delphi method was used and factors analysis was performed. Bartlett test of sphericity and the Kaiser-Meyer-Olkin measure of sample adequacy were used. The number of components was extracted using principal component analysis.

RESULTS

The mean score of the MHS was 158.7 (SD 11.4). The MHS mean score of female participants was significantly higher than the mean score of male participants (159.59, SD 10.21 vs 155.48, SD 14.35; P=.008). The MHS mean score was significantly lower in dental students (154.12, SD 1.45; P=.005) than the mean scores of medical students (159.77, SD 1.02), pharmacy students (161.40, SD 1.05), and preparatory-year students (159.05, SD 1.94). However, no significant relationship was found between humanism and academic year (P=.32), university type (P=.34), marital status (P=.64), or financial situation (P=.16). The Kaiser-Meyer-Olkin test (0.730) and Bartlett test of sphericity (1201.611, df=351; P=.01) were performed. Factor analysis indicated that the proportion of variables between the first and second factors was greater than 10%, confirming that the scale was a single group. The Cronbach α for the overall scale was 0.735, indicating that the scale had acceptable reliability and validity.

CONCLUSIONS

The results of this study suggest that the MHS is a reliable and valid tool for measuring humanity among health professional students and the development of patient-centered care.

A Socioecological Model of Transition to Nursing Practice

A newly licensed nurse's (NLN) transition to practice (TTP) is affected by interactions inside and outside the hospital. The aim of this Delphi study was to create a socioecological model of TTP. Included in the socioecological model are personal characteristics and interactions at the relationship, community, and societal levels that the NLNs feel affects their TTP. This socioecological model can be used to plan TTP programs that address the holistic needs of the NLNs.

Development and validation of the global assessment of the evidence implementation environment [GENIE] tool

BACKGROUND

Overcoming the various barriers to evidence implementation is critical to delivering evidence-based health care. Identifying and managing these obstacles is somewhat challenging however, due to interprofessional and interjurisdictional variations in reported barriers. An efficient, systematic, comprehensive and innovative approach to isolating the barriers to evidence implementation is therefore needed.

MATERIALS AND METHODS

Using a mixed methods design, the study aimed to develop, refine and validate a tool to assess the evidence implementation environment for complementary medicine (CM) professions. The tool was developed using a five-stage process, and refined and validated using a two-round e-Delphi technique.

RESULTS

Informed by reviews examining the barriers and enablers to evidence implementation in CM, and shaped by the Behaviour Change Wheel Framework, a preliminary 33-item tool was created (i.e. the Global Assessment of the Evidence Implementation Environment [GENIE] tool). A two-round Delphi technique was used to refine the criteria, with a panel of 23 experts agreeing to the removal of two criteria, and the addition of two items. In the end, the Delphi panel reached consensus on 33 criteria, which were sorted into nine stakeholder groups.

CONCLUSION

This study has for the first time, created an innovative tool to assess the capacity and capability of CM professions to engage in evidence-based practice at an optimal level. By assessing the evidence implementation environment of CM professions, the GENIE tool is able to determine where resources, infrastructure and personnel should be directed in order to optimise the uptake of evidence-based practices within CM professions.

Drop, Cover, and Hold On versus Fetal Position in the Triangle of Life to Survive in an Earthquake: A Delphi Study

OBJECTIVE

This study aims to determine and compare the effectiveness of Drop, Cover, and Hold On versus Fetal Position in the Triangle of Life regarding reducing casualties during earthquakes and establishing a consensus among medical search and rescue experts.

METHODS

In this study, the data collected from ten experienced medical search and rescue professionals were analyzed using a three-stage Delphi technique to compare Drop, Cover, and Hold On versus Fetal Position in the Triangle of Life.

RESULTS

At the end of the first round of Delphi, all of the experts mentioned the following factors: age, position, and surface area of the injured person. A victim's time under rubble and the experience of search and rescue teams are two prominent factors related to search and rescue. After the earthquake simulation in the second round, mannequin damage rates were examined by opening rubble pavement and tunnels. Following the second round of ratings, a third round of questionnaires was administered. As part of this questionnaire, participants were asked to give a score from one to ten based on their level of agreement. Whether they agreed or disagreed with Fetal Position in the Triangle of Life and Drop, Cover, and Hold On using a ten-point Likert scale, and the agreement rates were measured and compared. Experts completed a comparison of the two positions in the third round.

CONCLUSION

According to this expert consensus, the Fetal Position in the Triangle of Life has the following advantages over Drop, Cover, and Hold On: reduced surface area, less crush injuries, protection of a larger body part from injury, better protection from hypothermia, and better maintenance of basal metabolism.

Unmet needs on the current medical management of Cushing's syndrome: results from a Delphi panel of Italian endocrinologists

BACKGROUND

Cushing's syndrome (CS) is a rare clinical condition caused by excessive cortisol secretion from adrenal glands. CS is associated with increased mortality and morbidity; therefore, a prompt diagnosis and an effective therapeutic approach are strongly necessary to improve the patient's clinical management. The first-line treatment for CS is surgery, while medical treatment has historically played a minor role. However, thanks to the availability of novel compounds, the possibility of improving hypercortisolism control using different drug combinations emerged.

PURPOSE

No absolute recommendations are available to guide the therapeutic choice for patients with CS and, consequently, the awareness of unmet needs in CS management is growing. Although new data from clinical trials are needed to better define the most appropriate management of CS, an expert consensus approach can help define unmet needs and optimize the current CS management and treatment.

METHODS

Twenty-seven endocrinologists from 12 Italian regions, working among the main Italian referral centers for hospital endocrinology where they take care of CS patients, were involved in a consensus process and used the Delphi method to reach an agreement on 24 statements about managing CS patients.

RESULTS

In total, 18 statements reached a consensus. Some relevant unmet needs in the management of CS were reported, mainly related to the lack of a pharmacological treatment successful for the majority of patients.

CONCLUSION

While acknowledging the difficulty in achieving complete disease control, a significant change in CS management requires the availability of medical treatment with improved efficacy and safety over available therapeutic options at the time of the current study.

Component Costs of CAR-T Therapy in Addition to Treatment Acquisition Costs in Patients with Multiple Myeloma

INTRODUCTION

Ciltacabtagene autoleucel (cilta-cel), is a B-cell maturation antigen-directed, genetically modified autologous chimeric antigen receptor T-cell (CAR-T) immunotherapy. It is indicated for treatment for adult patients with relapsed or refractory multiple myeloma (RRMM) after four or more prior lines of therapy, including a proteasome inhibitor, an immunomodulatory agent, and an anti-CD38 monoclonal antibody. The objective of this study was to estimate the per-patient US commercial healthcare costs related to cilta-cel (CARVYKTI^®) CAR-T therapy (i.e., costs separate from cilta-cel therapy acquisition) for patients with RRMM.

METHODS

US prescribing information for cilta-cel, publicly available data, and published literature were used with clinician input to identify the cost components and unit costs associated with administration of cilta-cel. Cost components included apheresis, bridging therapy, conditioning therapy, administration, and postinfusion monitoring for 1 year of follow-up. Adverse event (AE) management costs for all grades of cytokine release syndrome and neurologic toxicities, and additional AEs grade ≥ 3 occurring in > 5% of patients were included in the analysis.

RESULTS

The estimated per-patient average costs of cilta-cel CAR-T therapy administered exclusively in an inpatient setting, excluding cilta-cel therapy acquisition costs, totaled US$160,933 over a 12 month period. Costs assuming different proportions of inpatient/outpatient administration (85%/15% and 70%/30%) were US$158,095 and US$155,257, respectively.

CONCLUSION

Cost estimates from this analysis, which disaggregates CAR-T therapy costs, provide a comprehensive view of the cost components of CAR-T therapy that can help healthcare decision-makers make informed choices regarding the use of cilta-cel. Real-world costs may differ with improved AE prevention and mitigation strategies.

Prerequisites, barriers and opportunities in care for Q-fever patients: a Delphi study among healthcare workers

BACKGROUND

Q-fever is a zoonotic disease that can lead to illness, disability and death. This study aimed to provide insight into the perspectives of healthcare workers (HCWs) on prerequisites, barriers and opportunities in care for Q-fever patients.

METHODS

A two-round online Delphi study was conducted among 94 Dutch HCWs involved in care for Q-fever patients. The questionnaires contained questions on prerequisites for high quality, barriers and facilitators in care, knowledge of Q-fever, and optimization of care. For multiple choice, ranking and Likert scale questions, frequencies were reported, while for rating and numerical questions, the median and interquartile range (IQR) were reported.

RESULTS

The panel rated the care for Q-fever patients at a median score of 6/10 (IQR = 2). Sufficient knowledge of Q-fever among HCWs (36%), financial compensation of care (30%) and recognition of the disease by HCWs (26%) were considered the most important prerequisites for high quality care. A lack of knowledge was identified as the most important barrier (76%) and continuing medical education as the primary method for improving HCWs' knowledge (76%). HCWs rated their own knowledge at a median score of 8/10 (IQR = 1) and the general knowledge of other HCWs at a 5/10 (IQR = 2). According to HCWs, a median of eight healthcare providers (IQR = 4) should be involved in the care for Q-fever fatigue syndrome (QFS) and a median of seven (IQR = 5) in chronic Q-fever care.

CONCLUSIONS

Ten years after the Dutch Q-fever epidemic, HCWs indicate that the long-term care for Q-fever patients leaves much room for improvement. Facilitation of reported prerequisites for high quality care, improved knowledge among HCWs, clearly defined roles and responsibilities, and guidance on how to support patients could possibly improve quality of care. These prerequisites may also improve care for patients with persisting symptoms due to other infectious diseases, such as COVID-19.

Standards for evaluating the quality of undergraduate nursing e-learning programme in low- and middle-income countries: a modified Delphi study

BACKGROUND

The lack of standards for evaluating the quality of undergraduate nursing programmes hampers the evaluation of e-learning programmes in low- and middle-income countries. Fragmented approaches to evaluation coupled with a lack of uniform criteria have been a major deterrent to the growth of e-learning. Adopting standards from high-income countries has contextual challenges in low- and middle-income countries (LMICs). Holistic approaches coupled with uniform standards provide information to stakeholders hence the quality of the programmes is measurable. The e-learning situation in low-and middle-income countries provided an impetus to develop and validate these standards.

DESIGN

A modified Delphi technique.

REVIEW METHODS

Fourteen experts with experience and expertise in e-learning and regulation of undergraduate nursing from fourteen countries from LMICs participated in three rounds of the modified Delphi process. A pre-described set of standards was shared electronically for independent and blinded ratings. An 80% threshold was set for consensus decisions. The standards were modified based on experts' comments, and two subsequent rounds were used to refine the standards and criteria.

RESULTS

At the end of round one, the expert consensus was to keep 67, modify 39 and remove three criteria. At the end of the second round, the consensus was to modify 38 and remove one criteria. In the third round, experts agreed that the standards were feasible, usable, and practical in LMICs. A total of six broad standards with 104 criteria were developed.

CONCLUSION

The Technological bloom permeating all spheres of society, including education is an essential component in the development of e-learning programmes. E-learning in nursing education requires critical evaluation to ensure quality in undergraduate nursing programmes. The intricacies of the Low and middle-income context were taken into consideration in developing the standards to offer sustainable evaluation of the quality of e-learning in LMICs, and local solutions to local problems.

Brazilian society of rheumatology methodological guide for the development of evidence-based clinical guidelines in rheumatology

Clinical practice guidelines (CPG) are developed to align standards of health care around the world, aiming to reduce the incidence of misconducts and enabling more effective use of health resources. Considering the complexity, cost, and time involved in formulating CPG, strategies should be used to facilitate and guide authors through each step of this process. The main objective of this document is to present a methodological guide prepared by the Epidemiology Committee of the Brazilian Society of Rheumatology for the elaboration of CPG in rheumatology. Through an extensive review of the literature, this study compiles the main practical recommendations regarding the following steps of CPG drafting: distribution of working groups, development of the research question, search, identification and selection of relevant studies, evidence synthesis and quality assessment of the body of evidence, the Delphi methodology for consensus achievement, presentation and dissemination of the recommendations, CPG quality assessment and updating. This methodological guide serves as an important tool for rheumatologists to develop reliable and high-quality CPG, standardizing clinical practices worldwide.

Prioritising Climate Change Mitigation Behaviours and Exploring Public Health Co-Benefits: A Delphi Study

Climate change requires urgent action; however, it can be challenging to identify individual-level behaviours that should be prioritised for maximum impact. The study aimed to prioritise climate change mitigation behaviours according to their impacts on climate change and public health, and to identify associated barriers and facilitators-exploring the impact of observed behaviour shifts associated with COVID-19 in the UK. A three-round Delphi study and expert workshop were conducted: An expert panel rated mitigation behaviours impacted by COVID-19 in relation to their importance regarding health impacts and climate change mitigation using a five-point Likert scale. Consensus on the importance of target behaviours was determined by interquartile ranges. In total, seven target behaviours were prioritised: installing double/triple glazing; installing cavity wall insulation; installing solid wall insulation; moving away from meat/emission heavy diets; reducing the number of cars per household; walking shorter journeys; and reducing day/weekend leisure car journeys. Barriers related to the costs associated with performing behaviours and a lack of complementary policy-regulated subsidies. The target behaviours are consistent with recommendations from previous research. To ensure public uptake, interventions should address behavioural facilitators and barriers, dovetail climate change mitigation with health co-benefits and account for the long-term impacts of COVID-19 on these behaviours.

Acid sphingomyelinase deficiency (ASMD): addressing knowledge gaps in unmet needs and patient journey in Italy-a Delphi consensus

Acid sphingomyelinase deficiency (ASMD) is an ultra-rare disease, and several gaps of knowledge on various issues remain, particularly at a regional/national level. Expert opinions collected through well-defined consensus methodologies are increasingly used to make available reliable information in the context of rare/ultra-rare diseases. With the aim to provide indications on infantile neurovisceral ASMD (also formerly known as Niemann-Pick disease type A), chronic neurovisceral ASMD (formerly known as Niemann-Pick disease type A/B) and chronic visceral ASMD (formerly known as Niemann-Pick disease type B) in Italy, we conducted a Delphi consensus of experts focused on five main areas: (i) patients and disease characteristics; (ii) unmet needs and quality of life; (iii) diagnostic issues; (iv) treatment-related aspects; and (v) patient journey. Pre-specified, objective criteria were used to outline the multidisciplinary panel, based on 19 Italian experts in ASMD in paediatric and adult patients from different Italian Regions, including both clinicians (n = 16) and ASMD patients' advocacy or payors with expertise in rare diseases (n = 3). During two Delphi rounds, a high ratio of agreement was found on several topics related to ASMD characteristics, diagnosis, management and disease burden. Our findings may provide valuable indications for management of ASMD at a public health level in Italy.

Argument-based QUalitative Analysis strategy (AQUA) for analyzing free-text responses in health sciences Delphi studies

Delphi methods are mostly used in the health sciences to reach agreement among experts on unclear issues. Generally, consensus is reached after several rounds of Delphi using standardized items. Additional open-ended questions offer respondents the opportunity to provide reasons for judgments. Although these free-text responses contribute substantially to the steering and result generation of the Delphi process, so far no analytical strategy has been established which takes into account the context and methodological principles of the Delphi procedure. Moreover, in already published Delphi studies the analysis of qualitative data is often not sufficiently disclosed.•We provide an overview of analytical strategies for free-text responses. We critically reflect on them with regard to their use and suitability in the context of Delphi procedures.•Following established qualitative methods of qualitative content analysis according to Mayring and thematic analysis according to Braun & Clarke, we developed the Argument-based QUalitative Analysis strategy (AQUA) for Delphi studies in the health sciences and presented it using a concrete project example.•This newly developed strategy can significantly support the rule-governed and intersubjective evaluation of free-text responses in Delphi processes, the integration of the results into the feedback design, and thereby also the quality of the results.

Risk Management Activities in a Lung Cancer Multidisciplinary Team at a Comprehensive Cancer Center: Results of a Prospective Analysis

PURPOSE

The objective of the study was to highlight sources of harm that could negatively affect the lung cancer multidisciplinary team (MDT) activities to reduce the level of risk of each factor.

METHODS

A modified Delphi approach was used by a board of multi-health care professionals of the lung cancer MDT to identify the main processes, subprocesses, and risk factors of the multidisciplinary pathway of patients with lung cancer. A semiquantitative matrix was built with a five-point scale for probability of harm (likelihood) and severity of harm (consequences) according to the international risk management standards (ISO 31000-2018). The risk level was calculated by multiplying likelihood × consequences. Mitigation strategies have been identified and applied by the MDT to reduce risks to acceptable levels.

RESULTS

Three main processes (outpatient specialist visit, MDT discussion, and MDT program implementation), eight related subprocesses, and 16 risk factors were identified. Four risk factors (25%) were related to outpatient specialist visit, seven (43.75%) to case discussion, and five (31.25%) to program implementation. Overall, two risk factors were assigned a low-risk level (12.5%), 11 a moderate-risk level (68.75%), one (6.25%) a high-risk level, and two (12.5%) a very high-risk level. After the implementation of mitigation measures, the new semiquantitative risk analysis showed a reduction in almost all hazardous situations: two risk factors (12.5%) were given a very low level, six (37.5%) a low level, seven (43.75%) a moderate level, and one (6.25%) a very high level.

CONCLUSION

An interdisciplinary risk assessment analysis is applicable to MDT activities by using an ad hoc risk matrix: if the hazard is identified and monitored, the risk could be reduced and managed in a short time.

Development of a telemedicine group educational program for patients with heart failure: A delphi study

Aims

Evidence regarding the most effective and feasible videoconferencing group educational program for patients with heart failure is still equivocal. This study aimed to reach consensus about the structure, acceptability, and feasibility of videoconferencing for people with heart failure in Jordan that improves access to healthcare and clinical outcomes.

Methods

There were two Delphi survey studies of three rounds each. Delphi one survey involved 32 healthcare staff, experienced in heart failure clinical practice and telehealth, to obtain a consensus of opinion on a proposed group videoconferencing program for patients with heart failure. Delphi two involved seven staff of the information technology center, experienced in videoconferencing and using supporting applications, to obtain their consensus on the current capabilities of the healthcare system and patients about information technology. Descriptive statistics were used for each item to determine whether consensus was achieved or not. Items that received 80% disagreement or 80% agreement of participants were not presented for re-rating in the third round, while the items that scored varying degrees of agreement were presented for experts for re-rating.

Results

In Delphi one a group of items reached consensus regarding structure, factors influencing, and effectiveness of the videoconferencing program. In Delphi two, the findings indicated that videoconferencing modality is applicable and feasible in Jordan.

Conclusion

This is the first study that addresses the equivocal evidence for the design and implementation of heart failure videoconferencing programs. The framework of the current proposed program can be utilized as a guideline to test or develop a future videoconference program.

PROCURE European consensus on breast cancer multigene signatures in early breast cancer management

Breast cancer multigene signatures (BCMS) have changed how patients with early-stage breast cancer (eBC) are managed, as they provide prognostic information and can be used to select patients who may avoid adjuvant chemotherapy. Clinical guidelines make recommendations on the use of BCMS; however, little is known on the current use of BCMS in clinical practice. We conduct a two-round Delphi survey to enquire about current use and perceived utility for specific patient profiles, and unmet needs of BCMS. Overall, 133 panellists experienced in breast cancer across 11 European countries have participated, most using BCMS either routinely (66.2%) or in selected cases (27.1%). Our results show that BCMS are mainly used to assess the risk of recurrence and to select patients for adjuvant chemotherapy; notably, no consensus has been reached on the lack of utility of BCMS for selecting the type of chemotherapy to administer. Also, there are discrepancies between the recommended and current use of BCMS in clinical practice, with use in certain patient profiles for which there is no supporting evidence. Our study suggests that physician education initiatives are needed to ensure the correct use and interpretation of BCMS to, ultimately, improve management of patients with eBC.

The First Modified Delphi Consensus-Building Exercise on Surgical Ward Rounds in the United Kingdom National Health Service

BACKGROUND

The ward round is an integral part of everyday surgical practice. It is a complex clinical activity that requires both sound clinical management and communication skills. This study reports the results of a consensus-building exercise on the common aspects of the general surgical ward rounds.

METHODS

The consensus-building committee involving a range of stakeholders from 16 United Kingdom (UK) National Health Service trusts took part in this consensus exercise. The members discussed and suggested a series of statements concerning surgical ward round. An agreement of ≥ 70% among members was regarded as a consensus.

RESULTS

Thirty-two members voted on 60 statements. There was a consensus on fifty-nine statements after the first round of voting, and one statement was modified before it reached consensus in the second round. The statements covered nine sections: a preparation phase, team allocation, multidisciplinary approach to the ward round, structure of the round, teaching considerations, confidentiality and privacy, documentation, post-round arrangements, and weekend round. There was a consensus on spending time to prepare for the round, a consultant-led round, involvement of the nursing staff, an MDT round at the beginning and end of the week, a minimum of 5 min allocated to each patient, utilisation of a round checklist, afternoon virtual round, and a clear handover and plan for the weekend.

CONCLUSION

The consensus committee achieved agreement on several aspects concerning the surgical ward rounds in the UK NHS. This should help improve the care of surgical patients in the UK.

Management of Locally Advanced Unresectable or Metastatic Urothelial Carcinoma: Expert Opinion from an Indian Panel via Delphi Consensus Method

Introduction Currently, there are no guidelines for the management of locally advanced unresectable or metastatic urothelial carcinoma (mUC) from an Indian perspective. There is a lack of consensus on the utility of treatment options in first-line (1L) and second-line (2L) settings, especially in cisplatin- and platinum-unfit mUC patient subgroups.

Objective This articles aims to develop evidence-based practical consensus recommendations for the management of mUC in Indian settings.

Methods Modified Delphi consensus methodology was considered to arrive at a consensus. An expert scientific committee of 15 medical oncologists from India constituted the panel. Twelve clinically relevant questions were grouped into five categories for presentation and discussion: (1) cisplatin and platinum ineligibility criteria; (2) programmed death ligand 1 and fibroblast growth factor receptor (FGFR) testing in mUC patients; (3) treatment options in 1L settings; (4) role of switch maintenance; and (5) treatment options in 2L. Statements that reached high (≥ 80%) and moderate (60–79%) levels of consensus in the first round (electronic survey) did not undergo the second Delphi round. The questions that received a low level of consensus (< 60%) were discussed during the virtual meeting.

Results Renal impairment (creatinine clearance [CrCl] < 60 mL/min) and New York Heart Association class 3 heart failure are important assessment criteria for determining cisplatin ineligibility. Patients are unfit for any platinum-based chemotherapy in case of Eastern Cooperative Oncology Group performance status> 3 or severe renal impairment (CrCl < 30 mL/min). Gemcitabine and platinum with cisplatin over carboplatin were preferred in 1L settings. In patients unfit for cisplatin-based regimens, carboplatin–gemcitabine chemotherapy was preferred over immunotherapy (atezolizumab or pembrolizumab). Selected patients who are platinum ineligible may be considered for immunotherapy. Post-induction chemotherapy, those who do not progress may be strongly considered for avelumab maintenance. Experts recommended erdafitinib in FGFR-positive mUC patients in 2L settings. In FGFR-negative patients, immunotherapy (pembrolizumab, nivolumab, or avelumab) may be preferred over chemotherapy (paclitaxel, docetaxel, or vinflunine). Enfortumab vedotin and sacituzumab govitecan may be considered for further lines of therapy.

Conclusion Expert panel consensus will offer expert guidance to oncologists/clinicians on the management of mUC in Indian settings.

Key Points

Textbook outcomes in DIEP flap breast reconstruction: a Delphi study to establish consensus

PURPOSE

Composite measures, like textbook outcomes, may be superior to individual metrics when assessing hospital performance and quality of care. This study utilized a Delphi process to define a textbook outcome in DIEP flap breast reconstruction.

METHODS

A two-round Delphi survey defined: (1) A textbook outcome, (2) Exclusion criteria for a study population, and (3) Respondent opinion regarding textbook outcomes. An a priori threshold of ≥ 70% agreement among respondents established consensus among the tested statements.

RESULTS

Out of 85 invitees, 48 responded in the first round and 41 in the second. A textbook outcome was defined as one that meets the following within 90 days: (1) No intraoperative complications, (2) Operative duration ≤ 12 h for bilateral and ≤ 10 h for unilateral/stacked reconstruction, (3) No post-surgical complications requiring re-operation, (4) No surgical site infection requiring IV antibiotics, (5) No readmission, (6) No mortality, (7) No systemic complications, and (8) Length of stay < 5 days. Exclusion criteria for medical and surgical characteristics (e.g., BMI > 40, HgbA1c > 7) and case-volume cut-offs for providers (≥ 21) and institutions (≥ 44) were defined. Most agreed that textbook outcomes should be defined for complex plastic surgery procedures (75%) and utilized to gauge hospital performance for microsurgical breast reconstruction (77%).

CONCLUSION

This Delphi study identified (1) Key elements of a textbook outcome for DIEP flap breast reconstruction, (2) Exclusion criteria for future studies, and (3) Characterized surgeon opinions regarding the utility of textbook outcomes in serving as quality metric for breast reconstruction care.

Priorities in updating training paradigms in orthopedic manual therapy: an international Delphi study

PURPOSE

Orthopedic manual therapy (OMT) education demonstrates significant variability between philosophies and while literature has offered a more comprehensive understanding of the contextual, patient specific, and technique factors which interact to influence outcome, most OMT training paradigms continue to emphasize the mechanical basis for OMT application. The purpose of this study was to establish consensus on modifications & adaptions to training paradigms which need to occur within OMT education to align with current evidence.

METHODS

A 3-round Delphi survey instrument designed to identify foundational knowledge to include and omit from OMT education was completed by 28 educators working within high level manual therapy education programs internationally. Round 1 consisted of open-ended questions to identify content in each area. Round 2 and Round 3 allowed participants to rank the themes identified in Round 1.

RESULTS

Consensus was reached on 25 content areas to include within OMT education, 1 content area to omit from OMT education, and 34 knowledge components which should be present in those providing OMT. Support was seen for education promoting understanding the complex psychological, neurophysiological, and biomechanical systems as they relate to both evaluation and treatment effect. While some concepts were more consistently supported there was significant variability in responses which is largely expected to be related to previous training.

CONCLUSION

The results of this study indicate manual therapy educators understanding of evidence-based practice as support for all 3 tiers of evidence were represented. The results of this study should guide OMT training program development and modification.

Implementing health system improvement: resources and strategies for interprofessional teams

Health system improvement (HSI) is focused on systematic changes to organisational processes and practices to improve the efficient delivery of safe care and quality outcomes. Guidelines that specify how interprofessional teams conduct HSI and knowledge translation are needed. We address this urgent requirement providing health professional teams with resources and strategies to investigate, analyse and implement system-level improvements. HSI encompasses similar, yet different, inter-related activities across a continuum. The continuum spans three categories of activities, such as quality improvement, health management research and translational health management research. A HSI decision making guide and checklist, comprising six-steps, is presented that can be used to select and plan projects. This resource comprises six interconnected steps including, defining the activity, project outcome, aim, use of evidence, appropriate methodology and implementation plan. Each step has been developed focusing on an objective, actions and resources. HSI activities provide a foundation for interprofessional collaboration, allowing multiple professions to create, share and disseminate knowledge for improved healthcare. When planned and executed well, HSI projects assist clinical and corporate staff to make evidence-informed decisions and directions for the benefit of the service, organisation and sector.

Agile Delphi methodology: A case study on how technology impacts burnout syndrome in the post-pandemic era

Introduction

In the post-pandemic era, many habits in different areas of our lives have changed. The exponential growth in the use of technology to perform work activities is one of them. At the same time, there has been a marked increase in burnout syndrome. Is this a coincidence? Could they be two interconnected developments? What if they were? Can we use technology to mitigate this syndrome? This article presents the agile Delphi methodology (MAD), an evolved version of the Delphi method, adapted to the needs of modern-day society.

Methods

To drive Occupational Health and Safety (OHS) experts to reach a consensus on what technological and non-technological factors could be causing the burnout syndrome experienced by workers in the post-pandemic era, MAD has been used in a specific case study. This study formally presents MAD and describes the stages enacted to run Delphi experiments agilely.

Results

MAD is more efficient than the traditional Delphi methodology, reducing the time taken to reach a consensus and increasing the quality of the resulting products.

Discussion

OHS experts identified factors that affect and cause an increase in burnout syndrome as well as mechanisms to mitigate their effects. The next step is to evaluate whether, as the experts predict, burnout syndrome decreases with the mechanisms identified in this case study.

Country score tool to assess readiness and guide evidence generation of immunization programs in aging adults in Europe

Objectives

Delaying of policies for immunization of aging adults, low vaccine uptake, and the lack of supportive evidence at the national level could diminish the value in health and economics of such programs. This study aims to develop a "country score tool" to assess readiness and to facilitate evidence generation for aging adult immunization programs in Europe, and examine the comprehensiveness, relevance, acceptability, and feasibility of the tool.

Methods

The tool was developed in two phases. First, a modified Delphi process was used to construct the tool. The process included a literature review, stakeholder consultations, and a three-round Delphi study. The Delphi panel included researchers, supra-national and national decision-makers of immunization programs recruited from five countries, using snowball sampling method. The consensus was predefined at the agreement rate of 70%. Pilot testing of the tool was conducted in the Netherlands, Germany, Serbia, and Hungary involving researchers in the field of health technology assessment. After assessing the countries' readiness, researchers evaluated four features, namely comprehensiveness, relevance, acceptability, and feasibility of the tool via an online survey that included 5-scale Likert questions. The percentages of affirmative answers including "agree" and "totally agree" choices were presented.

Results

The review identified 16 tools and frameworks that formed the first version of our tool with 14 items. Eight experts were involved in the Delphi panel. Through three Delphi rounds, four items were added, one was dropped, and all others were amended. The consensus was achieved on the tool with 17 items divided into decision-making and implementation parts. Each item has a guiding question, corresponding to explanations and rationales to inform assessment with readiness scores. Eight researchers completed the pilot testing. The tool was rated as comprehensive (75%), relevant (100%), acceptable (75%), and feasible (88%) by participants.

Conclusion

Through a thorough and transparent process, a country score tool was developed helping to identify strengths, weaknesses, and evidential requirements for decision-making and implementation of immunization programs of aging adults. The tool is relevant for different European contexts and shows good comprehensiveness, acceptability, and feasibility.

Computational pathology in 2030: a Delphi study forecasting the role of AI in pathology within the next decade

BACKGROUND

Artificial intelligence (AI) is rapidly fuelling a fundamental transformation in the practice of pathology. However, clinical integration remains challenging, with no AI algorithms to date in routine adoption within typical anatomic pathology (AP) laboratories. This survey gathered current expert perspectives and expectations regarding the role of AI in AP from those with first-hand computational pathology and AI experience.

METHODS

Perspectives were solicited using the Delphi method from 24 subject matter experts between December 2020 and February 2021 regarding the anticipated role of AI in pathology by the year 2030. The study consisted of three consecutive rounds: 1) an open-ended, free response questionnaire generating a list of survey items; 2) a Likert-scale survey scored by experts and analysed for consensus; and 3) a repeat survey of items not reaching consensus to obtain further expert consensus.

FINDINGS

Consensus opinions were reached on 141 of 180 survey items (78.3%). Experts agreed that AI would be routinely and impactfully used within AP laboratory and pathologist clinical workflows by 2030. High consensus was reached on 100 items across nine categories encompassing the impact of AI on (1) pathology key performance indicators (KPIs) and (2) the pathology workforce and specific tasks performed by (3) pathologists and (4) AP lab technicians, as well as (5) specific AI applications and their likelihood of routine use by 2030, (6) AI's role in integrated diagnostics, (7) pathology tasks likely to be fully automated using AI, and (8) regulatory/legal and (9) ethical aspects of AI integration in pathology.

INTERPRETATION

This systematic consensus study details the expected short-to-mid-term impact of AI on pathology practice. These findings provide timely and relevant information regarding future care delivery in pathology and raise key practical, ethical, and legal challenges that must be addressed prior to AI's successful clinical implementation.

FUNDING

No specific funding was provided for this study.

Eliciting and prioritising determinants of improved care in multimorbidity: A modified online Delphi study

Background

Multimorbidity is a major challenge to health and social care systems around the world. There is limited research exploring the wider contextual determinants that are important to improving care for this cohort. In this study, we aimed to elicit and prioritise determinants of improved care in people with multiple conditions.

Methods

A three-round online Delphi study was conducted in England with health and social care professionals, data scientists, researchers, people living with multimorbidity and their carers.

Results

Our findings suggest a care system which is still predominantly single condition focused. 'Person-centred and holistic care' and 'coordinated and joined up care', were highly rated determinants in relation to improved care for multimorbidity. We further identified a range of non-medical determinants that are important to providing holistic care for this cohort.

Conclusions

Further progress towards a holistic and patient-centred model is needed to ensure that care more effectively addresses the complex range of medical and non-medical needs of people living with multimorbidity. This requires a move from a single condition focused biomedical model to a person-based biopsychosocial approach, which has yet to be achieved.

Determining a common understanding of interprofessional competencies for pre-registration health professionals in Aotearoa New Zealand: A Delphi study

There is growing awareness that factors such as the growing incidence of co-morbidity and increasing complexity of patient health needs cannot be addressed by health professionals practicing in isolation. Given this, there is an increasing emphasis on preparing students in health-related programs for effective interprofessional practice. Less clear, however, are the specific skills and clinical or learning opportunities necessary for students to develop effectiveness in interprofessional practice. These factors drove a team associated with a tertiary health education provider in Hamilton, New Zealand to transform traditional clinical student experiences in the form of an interprofessional student-assisted clinic. The clinic was intended, in part, to provide students with opportunities to learn and experience interprofessionalism in practice but was hampered by limited information available regarding the specific skill requirements necessary for students in New Zealand to learn in this context. In this Delphi study, we synthesize national expert opinion on student competency indicators necessary for effective interprofessional practice. The resultant set of indicators is presented and opportunities for application and further research discussed. The paper offers guidance to others seeking to innovate health curricula, develop novel service-oriented learning experiences for students, and foster interprofessional practice competence in the future health workforce.

Standards for Scaffolding in Health Sciences Programmes: A Delphi Consensus Study

Introduction

Health sciences programmes operate in complex, unpredictable contexts, underscoring the need for comprehensive scaffolding of the learning processes. Yet, the scaffolding approaches remain fragmented, and lack a shared approach to how programmes could integrate scaffolding across the curricula. The literature argues that standards result in the comprehensive implementation of educational practices. There are no reported standards related to scaffolding practices in these programmes.

OBJECTIVES

To develop standards for scaffolding in health sciences programmes utilising a consensus approach through a modified Delphi Technique.

METHODS

Following the recommendations on Conducting and REporting of DElphi Studies (CREDES), an online modified Delphi technique was applied. Evidence on the application of scaffolding in health sciences programmes, obtained through an integrative review, was synthesised to draft standards. Using purposive and snowball sampling, an international panel from diverse geographical and professional backgrounds refined and validated the standards. Descriptive statistics was utilised to analyse demographic data and consensus agreements to include standards and criteria. Qualitative analysis of textual comments ensured the synthesis and inclusion of critical divergent views and additions.

RESULTS

A total of 22 experts from around the globe agreed to participate in the study and one did not complete Delphi surveys. Most experts (n = 18) held a PhD; and an average of 19 years of teaching in health sciences programmes. Four standards and 27 criteria were included after achieving consensus during the two Delphi surveys rounds. The included standards focused on four areas: structuring and sequencing educational activities, resources/tools for scaffolding, structuring the programme and instructional strategies to support learning.

CONCLUSION

The principle-based standards developed in this study could direct and support scaffolding practices in health sciences programmes. The standards' emphases on macro-, meso- and micro-scaffolding present numerous opportunities for designing and applying contextually sensitive scaffolding strategies at every level of curriculum implementation.

Cross-cultural adaptation and validation of the Chinese version of the short-form of the Central Sensitization Inventory (CSI-9) in patients with chronic pain: A single-center study

Chronic pain affects more than 30% of the general population. The 9-item Central Sensitization Inventory (CSI-9) is a shortened version of the CSI-25, which is a patient-reported instrument used to screen people at risk of central sensitization (CS). The aim of this study was to cross-culturally adapt and validate a Chinese version of the CSI-9. The Chinese CSI-9 was generated by translation of the original English version, back-translation, cultural adaptation, and revision using the Delphi method. The Chinese CSI-9 was administered to 235 patients with chronic pain and 55 healthy controls. Structural validity (confirmatory factor analysis), construct validity (correlations with other scales), test-retest reliability (intraclass correlation coefficient, ICC), and internal consistency (Cronbach's α) were evaluated. Confirmatory factor analysis was performed using one factor. The Chinese CSI-9 score was positively correlated with the Pain Catastrophic Scale (PCS) total score (r = 0.463), PCS subscale scores (r = 0.347-0.463), Brief Pain Inventory (BPI) mean item score (r = 0.524), BPI total score (r = 0.773), and the number of painful sites (r = 0.451). The Chinese CSI-9 had excellent test-retest reliability (ICC = 0.958) and excellent internal consistency (Cronbach's α = 0.902 in the overall sample and 0.828 in the chronic pain population). The optimal cut-off value for the Chinese CSI-9 was 18 points. The Chinese CSI-9 had excellent test-retest reliability and satisfactory structural validity and construct validity. The CSI-9 could potentially be utilized in China as a self-report questionnaire in both clinical practice and research settings.

Possible solutions to enhance evidence-based practice proposed by rehabilitation professionals in Japan: a Delphi study

[Purpose] We aimed to identify possible solutions to enhance evidence-based practice (EBP) in rehabilitation professionals in Japan. [Participants and Methods] A three-round Delphi method was undertaken among a cohort of clinical therapists (328 physical therapists, 55 occupational therapists, and 6 speech therapists). In the first round, the participants listed possible solutions for promoting EBP, other than 12 solutions presented in a previous study; subsequently, a new list was created. In the second round, a newly-created list of solutions was presented, and the participants responded on a 5-point Likert scale on how much they agreed with the solutions promoting EBP in Japanese rehabilitation professionals. In the third round, the distribution of responses obtained in the second round was presented, and participant's agreement was again assessed on a 5-point Likert scale. [Results] Across the three rounds, data were collected from 33.7% to 47.0% of all eligible participants. After the first round, 17 possible solutions were developed, and a list of 29 solutions was used in the second round. After the third round, 10 solutions reached the predetermined criteria for consensus. [Conclusion] In this study, ten possible solutions to promote EBP were proposed by the Japanese rehabilitation professionals.

Quality assessment of breast cancer studies conducted with the Delphi technique

BACKGROUND

The Delphi technique is a consensus method aiming to obtain statistical estimations from a qualitative approach, through an iterative process that leads to consensus within experts. The main characteristics of the technique include iteration, anonymity, feedback, and consensus reaching. When high-quality, quantitative evidence on a particular topic is insufficient, the Delphi technique can be used for making decisions in clinical scenarios. However, the quality of studies on breast cancer conducted with this technique, has not been assessed.

OBJECTIVE

We aim to evaluate the quality of studies on breast cancer which used the Delphi technique as their method.

METHODS

A quality assessment tool (Quali-D) was created through consensus among experts on the Delphi technique. Then, the tool was applied to studies on breast cancer which used the Delphi technique as their method.

RESULTS

Studies conducted through the Delphi technique mainly assessed for quality indicators and expressed needs in patients with breast cancer. High-quality characteristics were reported in 63.89% of the studies. 98.61% used the Delphi technique due to lack of a more adequate method to solve their research question. 98.61% summarized and presented results in a clear way. In 91.67% of the studies, at least two rounds were conducted. 86.11% described the methods for expert selection in a complete manner. Only 54.17% of the studies reported an anonymous process and 4.17% of the studies disclosed conflicts of interest thoroughly.

CONCLUSIONS

A variety of topics were assessed through the Delphi technique in cases where no other technique would have been more appropriate for assessing these issues. Significant limitations are present in terms of anonymity and full disclosure of conflicts of interest. We found that the quality of studies conducted with the Delphi technique regarding breast cancer is overall good. However, the limitations of each study must be considered when applying their results to clinical practice.

Patient centered care in primary health care teleconsultations: an exploratory study

With the current evolution of health digitalization driven by the COVID-19 pandemic, it is expected that teleconsultations - specifically, synchronous audio consultations (by telephone) or video-based (video calls) between health professionals (Doctors and Nurses) and patients - will be more used in Primary Health Care. The provision of health care through teleconsultations must be evaluated by the quality management of health organizations to ensure that the needs of patients are met. For this reason, this study was carried out under the objective of identifying indicators to create a culture of Patient-Centered Care (PCC) in teleconsultations in Primary Health Care. The methodology followed was based on the Delphi method. The research aimed to analyze the suitability of 48 indicators (organized in Donabedian's quality dimensions) to assess the implementation of PCC in Primary Health Care. Despite all indicators were viewed as very important, the disparity in responses was significant. Future research should extend this study by involving other groups of experts (like academics who study the subject and members of patient associations).

Management of B-cell lineage acute lymphoblastic leukemia: expert opinion from an Indian panel via Delphi consensus method

Introduction

Currently, there are no guidelines for the management of B-cell lineage acute lymphoblastic leukemia (B-ALL) from an Indian perspective. The diagnostic workup, monitoring, and treatment of B-ALL vary among different physicians and institutes.

Objective

To develop evidence-based practical consensus recommendations for the management of B-ALL in Indian settings.

Methods

Modified Delphi consensus methodology was considered to arrive at a consensus. An expert scientific committee of 15 experts from India constituted the panel. Clinically relevant questions belonging to three major domains were drafted for presentation and discussion: (i) diagnosis and risk assignment; (ii) frontline treatment; and (iii) choice of therapy (optimal vs. real-world practice) in relapsed/refractory (R/R) settings. The questionnaire was shared with the panel members through an online survey platform. The level of consensus was categorized into high (≥ 80%), moderate (60%-79%), and no consensus (< 60%). The process involved 2 rounds of discussion and 3 rounds of Delphi survey. The questions that received near or no consensus were discussed during virtual meetings (Delphi rounds 1 and 2). The final draft of the consensus was emailed to the panel for final review.

Results

Experts recommended morphologic assessment of peripheral blood or bone marrow, flow cytometric immunophenotyping, and conventional cytogenetic analysis in the initial diagnostic workup. Berlin-Frankfurt-Münster (BFM)-based protocol is the preferred frontline therapy in pediatric and adolescent and young adult patients with B-ALL. BFM/German Multicenter Study Group for Adult Acute Lymphoblastic Leukemia-based regimen is suggested in adult patients with B-ALL. Immunotherapy (blinatumomab or inotuzumab ozogamicin) followed by allogeneic hematopoietic cell transplantation (allo-HCT) is the optimal choice of therapy that would yield the best outcomes if offered in the first salvage in patients with R/R B-ALL. In patients with financial constraints or prior allo-HCT (real-world practice) at first relapse, standard-intensive chemotherapy followed by allo-HCT may be considered. For subsequent relapses, chimeric antigen receptor T-cell therapy or palliative care was suggested as the optimal choice of therapy.

Conclusion

This expert consensus will offer guidance to oncologists/clinicians on the management of B-ALL in Indian settings.

Therapeutic Appropriateness of Cariprazine in the Management of Schizophrenia: Experts' Opinion using a Delphi Approach

BACKGROUND

Schizophrenia is a psychiatric disorder whose therapeutic objectives are aimed at reducing symptoms and improving patient's quality of life. First- and second-generation antipsychotics present numerous side effects. Recently introduced in the treatment of schizophrenia, cariprazine has shown to improve positive and negative symptoms as well as cognitive impairment, with good tolerability.

OBJECTIVE

To assess the level of consensus among Italian psychiatrists in relation to the use of cariprazine in the treatment of schizophrenia by using the Delphi technique.

METHOD

A Delphi study was undertaken between January and July 2022. Two questionnaires were consecutively sent to a panel of 97 psychiatrists from all over Italy, of which 81 actively participated, anonymously, in at least one of the two consultations with a sufficiently high response rate (83%).

RESULTS

Broad consensus in terms of the efficacy and safety of cariprazine in the treatment of schizophrenia during all phases of the disorder. The young first-episode schizophrenia patient with or without substance abuse seems to be an excellent candidate for cariprazine therapy. In addition, the lack of side effects makes cariprazine a suitable drug for adult and elderly patients with schizophrenia. However, there is still limited experience with the use of cariprazine, along with little knowledge of the most recent real-life data.

CONCLUSION

These results could encourage wider dissemination of evidence-based practices with the final aim of optimizing the clinical use of cariprazine in patients with schizophrenia.

How to establishing an indicators framework for evaluating the performances in primary TB control institutions under the new TB control model? Based on a Delphi study conducted in Guangxi, China

BACKGROUND

In China, the new TB control model of trinity form had been implemented in all parts, and the comprehensively evaluation to the performances in primary TB control institutions were closely related to the working capacity and quality of TB service, but there was still no an unified evaluation indicators framework in practice and few relevant studies. The purpose of this study was to establish an indicators framework for comprehensively evaluating the performances in primary TB control institutions under the new TB control model of trinity form in Guangxi, China.

METHODS

The Delphi method was used to establish an indicators framework for comprehensively evaluating the performances in primary TB control institutions under the new TB control model of trinity form, and the analytic hierarchy process(AHP) was used to determine the weights of all levels of indicators, from September 2021 to December 2021 in Guangxi, China.

RESULTS

A total of 14 experts who had at least 10 years working experience and engaged in TB prevention and control and public health management from health committee, CDC, TB designated hospitals and university of Guangxi were consulted in two rounds. The average age of the experts were (43.3 ± 7.549) years old, and the effective recovery rate of the questionnaire was 100.0%. The average value of authority coefficient of experts (Cr) in the two rounds of consultation was above 0.800. The Kendall's harmony coefficient (W) of experts' opinions on the first-level indicators, the second-level indicators and the third-level indicators were 0.786, 0.201 and 0.169, respectively, which were statistically significant (P < 0.05). Finally, an indicators framework was established, which included 2 first-level indicators, 10 second-level indicators and 37 third-level indicators. The results of analytic hierarchy process (AHP) showed that the consistency test of all levels of indicators were CI < 0.10, which indicating that the weight of each indicator was acceptable.

CONCLUSION

The indicators framework established in this study was in line with the reality, had reasonable weights, and could provide a scientific evaluation tool for comprehensively evaluating the performances in primary TB control institutions under the new TB control model of trinity form in Guangxi, China.

Factors associated with high-level endurance performance: An expert consensus derived via the Delphi technique

There is little agreement on the factors influencing endurance performance. Endurance performance often is described by surrogate variables such as maximum oxygen consumption, lactate threshold, and running economy. However, other factors also determine success and progression of high-level endurance athletes. Therefore, the aim was to identify the relevant factors for endurance performance assessed by international experts by adhering to a structured communication method (i.e., Delphi technique). Three anonymous evaluation rounds were conducted initiated by a list of candidate factors (n = 120) serving as baseline input variables. The items that achieved ≥70% of agreement in round 1 were re-evaluated in a second round. Items with a level of agreement of ≥70% in round 2 reached consensus and items with a level of agreement of 40-69% in round 2 were re-rated in a third round followed by a consensus meeting. Round 1 comprised of 27 panellists (n = 24 male) and in round 2 and 3 18 (n = 15 male) of the 27 panellists remained. Thus, the final endurance expert panel comprised of 18 international experts (n = 15 male) with 20 years of experience on average. The consensus report identified the following 26 factors: endurance capacity, running economy, maximal oxygen consumption, recovery speed, carbohydrate metabolism, glycolysis capacity, lactate threshold, fat metabolism, number of erythrocytes, iron deficiency, muscle fibre type, mitochondrial biogenesis, hydrogen ion buffering, testosterone, erythropoietin, cortisol, hydration status, vitamin D deficiency, risk of non-functional overreaching and stress fracture, healing function of skeletal tissue, motivation, stress resistance, confidence, sleep quality, and fatigue. This study provides an expert-derived summary including 26 key factors for endurance performance, the "FENDLE" factors (FENDLE = Factors for ENDurance Level). This consensus report may assist to optimize sophisticated diagnostics, personalized training strategies and technology.

The Place of Immune Reconstitution Therapy in the Management of Relapsing Multiple Sclerosis in France: An Expert Consensus

The treatment strategy in relapsing multiple sclerosis (RMS) is a complex decision requiring individualization of treatment sequences to maximize clinical outcomes. Current local and international guidelines do not provide specific recommendation on the use of immune reconstitution therapy (IRT) as alternative to continuous immunosuppression in the management of RMS. The objective of the program was to provide consensus-based expert opinion on the optimal use of IRT in the management of RMS. A Delphi method was performed from May 2022 to July 2022. Nineteen clinical assertions were developed by a scientific committee and sent to 14 French clinical experts in MS alongside published literature. Two consecutive reproducible anonymous votes were conducted. Consensus on recommendations was achieved when more than 75% of the respondents agreed or disagreed with the clinical assertions. After the second round, consensus was achieved amongst 16 out of 19 propositions: 13 clinical assertions had a 100% consensus, 3 clinical assertions a consensus above 75% and 3 without consensus. Expert-agreed consensus is provided on topics related to the benefit of the early use of IRT from immunological and clinical perspectives, profiles of patients who may benefit most from the IRT strategy (e.g. patients with family planning, patient preference and lifestyle requirements). These French expert consensuses provide up-to-date relevant guidance on the use of IRT in clinical practice. The current program reflects status of knowledge in 2022 and should be updated in timely manner when further clinical data in IRT become available.

PFD-SENTINEL: Development of a screening tool for pelvic floor dysfunction in female athletes through an international Delphi consensus

To develop a screening tool for pelvic floor dysfunction (PFD) in female athletes for use by sports medicine clinicians (eg, musculoskeletal/sports physiotherapists, sports and exercise medicine physicians), which guides referral to a PFD specialist (eg, pelvic floor/women’s health physiotherapist, gynaecologist, urogynaecologist, urologist).

Between February and April 2022, an international two-round modified Delphi study was conducted to assess expert opinion on which symptoms, risk factors and clinical and sports-related characteristics (items) should be included in a screening tool. We defined consensus a priori as >67% response agreement to pass each round.

41 and 34 experts participated in rounds 1 and 2, respectively. Overall, seven general statements were endorsed as relevant by most participants highlighting the importance of screening for PFD in female athletes. Through consensus, the panel developed the Pelvic Floor Dysfunction-ScrEeNing Tool IN fEmale athLetes (PFD-SENTINEL) and agreed to a cluster of PFD symptoms (n=5) and items (risk factors, clinical and sports-related characteristics; n=28) that should prompt specialist care. A clinical algorithm was also created: a direct referral is recommended when at least one symptom or 14 items are reported. If these thresholds are not reached, continuous monitoring of the athlete’s health is indicated.

Despite increasing awareness and clinical relevance, barriers to identify PFD in female athletes are still present. The PFD-SENTINEL is a new resource for sports medicine clinicians who regularly assess female athletes and represents the first step towards early PFD identification and management. Further studies to validate the tool are needed.

Development of a Protocol Using the Delphi Method for the ad interim Supply of Hormonal Contraceptives in Swiss Pharmacies

(1) Background: Pharmacists are often challenged with situations where women are already on hormonal contraceptives (HC) but have no valid prescription. By Swiss law, pharmacists are allowed to supply prescription-only drugs in exceptional situations without a physician's prescription. Because eligibility for HC can change, women at risk for complications, such as serious side effects, need to be identified. We aimed to develop a protocol to assist pharmacists in clarifying and documenting eligibility for HC. (2) Methods: We conducted a survey using the Delphi method to identify relevant clarifications and develop a protocol for pharmacists. Proposed material was created based on the literature and existing toolkits/protocols aimed at verifying eligibility for HC. A multidisciplinary expert panel, consisting of gynecologists and pharmacists, reviewed the proposed material and provided anonymized feedback over two survey cycles. (3) Results: This Delphi survey revealed items essential to the clarification of eligibility for HC in pharmacies for women who are already using it. This resulted in a protocol that maps "best practices" regarding these ad interim supplies of HC given without a prescription in Switzerland. (4) Conclusions: This survey, made using the Delphi method, allowed us to create a protocol for pharmacists that aims to verify and document eligibility for HC in Switzerland, where HC is frequently supplied without a prescription.