Applying real-time Delphi methods: development of a pain management survey in emergency nursing

Developing knowledge, attitude and practice questionnaire in desmoid tumours (KAPID study): applying e-Delphi to rare diseases

Background

Rare diseases are associated with unique challenges encountered in diagnosis, treatment and conduct of clinical research. Desmoid tumour (DT) is one such ultra-rare malignancy about which awareness among medical professionals remains limited. We developed a questionnaire to assess knowledge, attitude and practice (KAP) among medical professionals on DT.

Methods

E-Delphi method was used for the assessment of KAP for DT amongst clinical experts (experience of >/= 3 years in DT). 22 open-ended statements were developed by the core research group using current consensus guidelines. In round 1, experts provided subjective feedback which was incorporated into a 35-item questionnaire. Round 2 entailed experts giving feedback as a 5-point Likert scale classified into agreement (median score >/=4), neutral (median score 3) and disagreement (median score <3). Feedback from Round 2 was incorporated and questions with neutral consensus were modified. Questions in Round 3 achieved consensus if >/= 75% participants agreed.

Results

11 (64.7%) of 17 contacted experts responded in Round 1 including 6 (54.4%) who gave additional inputs and 5 (45.6%) who agreed to all statements. In round 2, 8 out of 11 experts responded to the 35-item questionnaire on knowledge (n = 16), attitude (n = 8) and practice (n = 11). 32 questions obtained agreement and 3 (8.5%) had neutral consensus. These were modified for round 3, in which consensus on 2 (66.6%) was attained. The final questionnaire comprises 34 items with 15, 8 and 11 questions on in the sections of knowledge, attitude and practice (KAP), respectively.

Consensus definition and diagnostic criteria for neonatal encephalopathy-study protocol for a real-time modified delphi study

BACKGROUND

'Neonatal encephalopathy' (NE) describes a group of conditions in term infants presenting in the earliest days after birth with disturbed neurological function of cerebral origin. NE is aetiologically heterogenous; one cause is peripartum hypoxic ischaemia. Lack of uniformity in the terminology used to describe NE and its diagnostic criteria creates difficulty in the design and interpretation of research and complicates communication with families. The DEFINE study aims to use a modified Delphi approach to form a consensus definition for NE, and diagnostic criteria.

METHODS

Directed by an international steering group, we will conduct a systematic review of the literature to assess the terminology used in trials of NE, and with their guidance perform an online Real-time Delphi survey to develop a consensus diagnosis and criteria for NE. A consensus meeting will be held to agree on the final terminology and criteria, and the outcome disseminated widely.

DISCUSSION

A clear and consistent consensus-based definition of NE and criteria for its diagnosis, achieved by use of a modified Delphi technique, will enable more comparability of research results and improved communication among professionals and with families.

IMPACT

The terms Neonatal Encephalopathy and Hypoxic Ischaemic Encephalopathy tend to be used interchangeably in the literature to describe a term newborn with signs of encephalopathy at birth. This creates difficulty in communication with families and carers, and between medical professionals and researchers, as well as creating difficulty with performance of research. The DEFINE project will use a Real-time Delphi approach to create a consensus definition for the term 'Neonatal Encephalopathy'. A definition formed by this consensus approach will be accepted and utilised by the neonatal community to improve research, outcomes, and parental experience.

Priorities for enhancing nurses' and social workers' competence and confidence in helping families support dependent children through parental death. A classic-Delphi survey

BACKGROUND

Annually, approximately five per cent of dependent children - aged under eighteen years - in the United Kingdom (UK), experience parental death. Nurses and social workers caring for parents with life-limiting illnesses, including cancer, help families support their children. However, these professionals have been found to lack confidence and competence in fulfilling this role.

METHODS

We conducted three rounds of a classic-Delphi survey to identify and measure a panel of topic experts' consensus on the priorities and issues for nurses and social workers when supporting families and children through parental death. The Delphi survey was conducted with a panel of UK topic experts (n=43) including lead health and social care professionals (n=30), parents bereaved of a partner whilst parenting dependent children (n=6), academics (n=4) and bereaved young adults (n=3).

RESULTS

Ninety per cent (n=18/20) of the issues for nurses and social workers and all (7/7) of the priorities rated and ordered in the survey achieved consensus. Key priorities were 1) training in opening conversations with families about dependent children, 2) training and support for nurses and social workers to manage their own and others' emotions arising from conversations with parents about children's needs regarding parental death, and 3) increasing nurses' and social workers' knowledge of sources of information to support families before the death of a parent.

CONCLUSION

We identified priorities for UK nurses and social workers. Further research is needed to identify which of these nurses and social workers would benefit most from support, and how any resultant interventions could enhance confidence and competence in helping families to support children through parental death.

Expert consensus on the attributes and competencies required for rural and remote junior physicians to work effectively in isolated indonesian communities

Indonesian physicians working in rural and remote areas must be equipped not only with generic competencies but also with the attributes and skills necessary to provide health care services without compromising quality. This study sought to reach a consensus on the attributes and competencies that are viewed as essential and important for working effectively as an early career doctor in rural and remote practice in Indonesia. A two-round Delphi study was conducted by reference to 27 consenting physicians working in rural and remote Indonesia. Forty-three items covering 9 attributes and 34 competencies were sent to these physicians to be rated on a Likert scale ranging from 1 to 5 in terms of their importance for effective rural and remote practice. Nine attributes and 29 competencies progressed to Round 2. All nine attributes and 29 competencies were identified as essential or important for junior physicians' ability to be effective in their practice. The essential attributes included professional quality related to prioritising the rural community. The essential competencies included medical skills, professional behaviour, interprofessional skills, health promotion and connection to the rural community. The consensus thus reached on these essential and important attributes and competencies can inform curriculum development for the undergraduate and postgraduate training of junior rural and remote physicians.

Developing a pediatric nursing simulation scenario template in South Korea: applying real-time Delphi methods

PURPOSE

This study aimed to describe the process of developing a validated pediatric nursing simulation scenario template using the real-time Delphi method.

METHODS

A panel of 13 pediatric nursing experts participated in a real-time Delphi survey conducted over two rounds. Initially, 83 items were included in the questionnaire focusing on the structure and content of the simulation scenario template. Data analysis involved calculating the content validity ratio (CVR) and the coefficient of variation to assess item validity and stability.

RESULTS

Through iterative rounds of the Delphi survey, a consensus was reached among the experts, resulting in the development of a pediatric nursing simulation scenario template comprising 41 items across nine parts. The CVR values ranged from 0.85 to 1.0, indicating a high consensus among experts regarding the inclusion of all items in the template.

CONCLUSION

This study presents a novel approach for developing a pediatric nursing simulation scenario template using real-time Delphi methods. The real-time Delphi method facilitated the development of a comprehensive and scientifically grounded pediatric nursing simulation scenario template. Our template aligns with the International Nursing Association for Clinical Simulation and Learning standards, and provides valuable guidance for educators in designing effective simulation scenarios, contributing to enhanced learning outcomes and better preparation for pediatric clinical practice. However, consideration of cultural and contextual adaptations is necessary, and further research should explore alternative consensus criteria.

Improving assessment of procedural skills in health sciences education: a validation study of a rubrics system in neurophysiotherapy

BACKGROUND

The development of procedural skills is essential in health sciences education. Rubrics can be useful for learning and assessing these skills. To this end, a set of rubrics were developed in case of neurophysiotherapy maneuvers for undergraduates. Although students found the rubrics to be valid and useful in previous courses, the analysis of the practical exam results showed the need to change them in order to improve their validity and reliability, especially when used for summative purposes. After reviewing the rubrics, this paper analyzes their validity and reliability for promoting the learning of neurophysiotherapy maneuvers and assessing the acquisition of the procedural skills they involve.

METHODS

In this cross-sectional and psychometric study, six experts and 142 undergraduate students of a neurophysiotherapy subject from a Spanish university participated. The rubrics' validity (content and structural) and reliability (inter-rater and internal consistency) were analyzed. The students' scores in the subject practical exam derived from the application of the rubrics, as well as the rubrics' criteria difficulty and discrimination indices were also determined.

RESULTS

The rubrics´ content validity was found to be adequate (Content Validity Index > 0.90). These showed a unidimensional structure, and an acceptable internal consistency (α = 0.71) and inter-rater reliability (Fleiss' ƙ=0.44, ICC = 0.94). The scores of the subject practical exam practically covered the entire range of possible theoretical scores, showing all the criterion medium-low to medium difficulty indices - except for the one related to the physical therapist position-. All the criterion exhibited adequate discrimination indices (rpbis > 0.39), as did the rubric as a whole (Ferguson's δ = 0.86). Students highlighted the rubrics´ usefulness for learning the maneuvers, as well as their validity and reliability for formative and summative assessment.

CONCLUSIONS

The changed rubrics constitute a valid and reliable instrument for evaluating the execution quality of neurophysiotherapy maneuvers from a summative evaluation viewpoint. This study facilitates the development of rubrics aimed at promoting different practical skills in health-science education.

Health equity in action: using community-engaged research to update an intervention promoting a healthy home food environment to Black/African American families

PURPOSE

Describe the method for conducting community-engaged research to identify needed changes for an existing evidence-based intervention, and prepare it for implementation in a community setting within the Dan L Duncan Comprehensive Cancer Center catchment area in an effort to achieve more equitable outcomes in diet-related disease risk factors.

METHODS

The intervention, Family Eats, was developed over 10 years ago. It works directly with parents of Black/African American 9-12 year old children to create a healthy home food environment to support prevention of obesity and related cancers. Data collection with community stakeholders occurred through a series of  Community Advisory Board (CAB) meetings guided by the Delphi Technique, an iterative approach for gaining group consensus on a topic.

RESULTS

Key information on needed changes and potential implementation strategies were identified. Perceived level of engagement among CAB members was high overall and in terms of both quantity and quality.

CONCLUSION

The Delphi Technique shows promise as a method for conducting community-engaged research that promotes engagement and identifies key information needed to adapt and implement an existing intervention in a community setting.

Selection of criteria for a telemedicine framework for designing, implementing, monitoring and evaluating telemedicine interventions: Validation using a modified Delphi process

Objectives

The call to scale up telemedicine services globally as part of the digital health transformation lacks an agreed-upon set of constructs to guide the implementation process. A lack of guidance hinders the development, consolidation, sustainability and optimisation of telemedicine services. The study aims to reach consensus among telemedicine experts on a set of implementation constructs to be developed into an evidence-based support tool.

Methods

A modified Delphi study was conducted to evaluate a set of evidence-informed telemedicine implementation constructs comprising cores, domains and items. The study evaluated the constructs consisting of five cores: Assessment of the Current Situation, Development of a Telemedicine Strategy, Development of Organisational Changes, Development of a Telemedicine Service, and Monitoring, Evaluation and Optimisation of Telemedicine Implementation; seven domains: Individual Readiness, Organisational Readiness, Clinical, Economic, Technological and Infrastructure, Regulation, and Monitoring, Evaluation and Optimisation; divided into 53 items. Global telemedicine specialists (n = 247) were invited to participate and evaluate 58 questions. Consensus was set at ≥70%.

Results

Forty-five experts completed the survey. Consensus was reached on 78% of the constructs evaluated. Regarding the core constructs, Monitoring, Evaluation and Optimisation of Telemedicine Implementation was determined to be the most important one, and Development of a Telemedicine Strategy the least. As for the domains, the Clinical one had the highest level of consensus, and the Economic one had the lowest.

Conclusions

This research advances the field of telemedicine, providing expert consensus on a set of implementation constructs. The findings also highlight considerable divergence in expert opinion on the constructs of reimbursement and incentive mechanisms, resistance to change, and telemedicine champions. The lack of agreement on these constructs warrants attention and may partly explain the barriers that telemedicine services continue to face in the implementation process.

Protocol for the development of a core outcome set for neonatal sepsis (NESCOS)

Neonatal sepsis is a serious public health problem; however, there is substantial heterogeneity in the outcomes measured and reported in research evaluating the effectiveness of the treatments. Therefore, we aim to develop a Core Outcome Set (COS) for studies evaluating the effectiveness of treatments for neonatal sepsis. Since a systematic review of key outcomes from randomised trials of therapeutic interventions in neonatal sepsis was published recently, we will complement this with a qualitative systematic review of the key outcomes of neonatal sepsis identified by parents, other family members, parent representatives, healthcare providers, policymakers, and researchers. We will interpret the outcomes of both studies using a previously established framework. Stakeholders across three different groups i.e., (1) researchers, (2) healthcare providers, and (3) patients' parents/family members and parent representatives will rate the importance of the outcomes in an online Real-Time Delphi Survey. Afterwards, consensus meetings will be held to agree on the final COS through online discussions with key stakeholders. This COS is expected to minimize outcome heterogeneity in measurements and publications, improve comparability and synthesis, and decrease research waste.

Establishing the core elements of a frailty at the front door model of care using a modified real-time Delphi technique

BACKGROUND

Innovations in models of care for older adults living with frailty presenting to the emergency department (ED) have become a key priority for clinicians, researchers and policymakers due to the deleterious outcomes older adults experience due to prolonged exposure to such an environment. This study aimed to develop a set of expert consensus-based statements underpinning operational design, outcome measurement and evaluation of a Frailty at the Front Door (FFD) model of care for older adults within an Irish context.

METHODS

A modified real-time Delphi method was used. Facilitation of World Café focus groups with an expert panel of 86 members and seperate advisory groups with a Public and Patient Involvement panel of older adults and members of the Irish Association of Emergency Medicine generated a series of statements on the core elements of the FFD model of care. Statements were analysed thematically and incorporated into a real-time Delphi survey, which was emailed to members of the expert panel. Members were asked to rank 70 statements across nine domains using a 9-point Likert scale. Consensus criteria were defined a priori and guided by previous research using 9-point rating scales.

RESULTS

Fifty members responded to the survey representing an overall response rate of 58%. Following analyses of the survey responses, the research team reviewed statements for content overlap and refined a final list of statements across the following domains: aims and objectives of the FFD model of care; target population; screening and assessment; interventions; technology; integration of care; evaluation and metrics; and research.

CONCLUSION

Development of a consensus derived FFD model of care represents an important step in generating national standards, implementation of a service model as intended and enhances opportunities for scientific impact. Future research should focus on the development of a core outcome set for studies involving older adults in the ED.

Control Attitudes Scale-Revised-Brazilian Version

BACKGROUND

Heart failure demands self-care skills and behaviors that can be negatively impacted by a low level of perceived control (PC), a belief about having the necessary resources to deal with negative events. Having valid and reliable instruments to measure PC is important to support interventions that improve self-care and related outcomes. The Control Attitudes Scale-Revised (CAS-R) was developed in the United States to measure PC in cardiac conditions. In Brazil, there are no instruments available to measure this construct.

OBJECTIVE

The aims of this study were to translate and adapt the CAS-R to the Brazilian population and to assess the content validity of the adapted version.

METHODS

The CAS-R was translated, back-translated, and assessed by an expert committee for linguistic equivalences. An agreement > 80% was considered adequate. Content validity (clarity, theoretical relevance, and practical pertinence) was assessed by both an expert professional panel (n = 6-8) and a panel of patients with heart failure (n = 40). A content validity coefficient > 0.70 was considered acceptable.

RESULTS

The translations to Brazilian Portuguese were considered consistent with the original CAS-R. In the third round of linguistic equivalence assessment, all items achieved acceptable agreement, except for 2 items. After modifications to the instrument to achieve adequate equivalences, the adapted version had a final content validity coefficient of 0.93. Most patients were able to understand the instrument.

CONCLUSIONS

The CAS-R-Brazilian version is equivalent to the original CAS-R and has satisfactory evidence of content validity. Additional psychometric testing will be performed to allow for the assessment of PC in individuals with heart failure in Brazil.

Comprehensive Ontology of Fibroproliferative Diseases: Protocol for a Semantic Technology Study

BACKGROUND

Fibroproliferative or fibrotic diseases (FDs), which represent a significant proportion of age-related pathologies and account for over 40% of mortality in developed nations, are often underrepresented in focused research. Typically, these conditions are studied individually, such as chronic obstructive pulmonary disease or idiopathic pulmonary fibrosis (IPF), rather than as a collective entity, thereby limiting the holistic understanding and development of effective treatments. To address this, we propose creating and publicizing a comprehensive fibroproliferative disease ontology (FDO) to unify the understanding of FDs.

OBJECTIVE

This paper aims to delineate the study protocol for the creation of the FDO, foster transparency and high quality standards during its development, and subsequently promote its use once it becomes publicly available.

METHODS

We aim to establish an ontology encapsulating the broad spectrum of FDs, constructed in the Web Ontology Language format using the Protégé ontology editor, adhering to ontology development life cycle principles. The modeling process will leverage Protégé in accordance with a methodologically defined process, involving targeted scoping reviews of MEDLINE and PubMed information, expert knowledge, and an ontology development process. A hybrid top-down and bottom-up strategy will guide the identification of core concepts and relations, conducted by a team of domain experts based on systematic iterations of scientific literature reviews.

RESULTS

The result will be an exhaustive FDO accommodating a wide variety of crucial biomedical concepts, augmented with synonyms, definitions, and references. The FDO aims to encapsulate diverse perspectives on the FD domain, including those of clinicians, health informaticians, medical researchers, and public health experts.

CONCLUSIONS

The FDO is expected to stimulate broader and more in-depth FD research by enabling reasoning, inference, and the identification of relationships between concepts for application in multiple contexts, such as developing specialized software, fostering research communities, and enhancing domain comprehension. A common vocabulary and understanding of relationships among medical professionals could potentially expedite scientific progress and the discovery of innovative solutions. The publicly available FDO will form the foundation for future research, technological advancements, and public health initiatives.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/48645.

Neuropathy-and foot-ulcer specific quality of life instrument (NeuroQoL): Translation, cross-cultural adaptation and content validation in Hindi

BACKGROUND

Neuropathy-and foot ulcer-specific quality of life instrument (NeuroQoL) is not available in Hindi language which is one of the most spoken languages in the world. Hence, we aimed to translate, cross-culturally adapt and content validate NeuroQoL into Hindi language (NeuroQoL-H).

METHODS

NeuroQoL is translated and cross-culturally adapted into Hindi language (NeuroQoL-H) using the guidelines proposed by Beaton and colleagues. Both item and scale levels content validity index (I-CVI & S-CVI) were reported for the validation and pretesting was performed with, n = 51 patients with diabetes. Floor and ceiling effects were reported.

RESULT

Total score of content validated and pretested, NeuroQoL-H with, n = 51 patients with diabetes is, 60 (36,109). For all the items in NeuroQoL-H, I-CVI ≥ 0.8 and S-CVI = 0.95. The floor effect is (24 %) while the ceiling effect is (1 %).

CONCLUSION

NeuroQoL-H scale is a valid outcome measure among Hindi speaking patients with diabetes.

Modelling a System to Help Improve It Comment on "Insights Gained From a Re-analysis of Five Improvement Cases in Healthcare Integrating System Dynamics Into Action Research"

The article that this commentary considers describes the use of systems modelling in an action research (AR) project that helped improvement teams to understand the dynamics of their service as a system. This commentary seeks to make the complex article easier to understand for those unfamiliar with the subjects. It describes the advantages, disadvantages and benefits, and suggests developments of this approach for research and practice using digital technologies. The conclusion of the commentary is that dynamic system modelling combined with AR is useful for certain purposes and can produce benefits in terms of a more sophisticated understanding of systems and feedback loops for practitioners. However, there are challenges for researchers unfamiliar with AR and dynamic system modelling as well workshop facilitation expertise.

Developing a medication adherence technologies repository: proposed structure and protocol for an online real-time Delphi study

INTRODUCTION

An online interactive repository of available medication adherence technologies may facilitate their selection and adoption by different stakeholders. Developing a repository is among the main objectives of the European Network to Advance Best practices and technoLogy on medication adherencE (ENABLE) COST Action (CA19132). However, meeting the needs of diverse stakeholders requires careful consideration of the repository structure.

METHODS AND ANALYSIS

A real-time online Delphi study by stakeholders from 39 countries with research, practice, policy, patient representation and technology development backgrounds will be conducted. Eleven ENABLE members from 9 European countries formed an interdisciplinary steering committee to develop the repository structure, prepare study protocol and perform it. Definitions of medication adherence technologies and their attributes were developed iteratively through literature review, discussions within the steering committee and ENABLE Action members, following ontology development recommendations. Three domains (product and provider information (D1), medication adherence descriptors (D2) and evaluation and implementation (D3)) branching in 13 attribute groups are proposed: product and provider information, target use scenarios, target health conditions, medication regimen, medication adherence management components, monitoring/measurement methods and targets, intervention modes of delivery, target behaviour determinants, behaviour change techniques, intervention providers, intervention settings, quality indicators and implementation indicators. Stakeholders will evaluate the proposed definition and attributes' relevance, clarity and completeness and have multiple opportunities to reconsider their evaluations based on aggregated feedback in real-time. Data collection will stop when the predetermined response rate will be achieved. We will quantify agreement and perform analyses of process indicators on the whole sample and per stakeholder group.

ETHICS AND DISSEMINATION

Ethical approval for the COST ENABLE activities was granted by the Malaga Regional Research Ethics Committee. The Delphi protocol was considered compliant regarding data protection and security by the Data Protection Officer from University of Basel. Findings from the Delphi study will form the basis for the ENABLE repository structure and related activities.

Establishing a Multidisciplinary Framework for an Emergency Food Supply System Using a Modified Delphi Approach

A scientific food emergency supply system is helpful for assuring food supplies continuity, improving response efficiency, and reducing disaster losses. However, the framework for a food emergency supply system is currently an understudied area in emergency management post-disaster. In this study, a comprehensive literature review of major databases was performed to identify potential indicators for the emergency food supply system, followed by a two-round modified Delphi with a multidisciplinary expert panel (n = 17) to verify the proposed framework. The effective response rate of questionnaires ranged from 94.4% (17/18) to 100% (17/17) and the authority coefficient of experts was 0.88, indicating high positivity and reliability of the experts. Furthermore, the p-values of Kendall’s W were < 0.01 and the Cronbach’s α were > 0.7 for all domains and indicators, indicating a high reliability and validity for the proposed framework. Finally, a consensus was reached on all eight domains and 81 indicators. In conclusion, this study introduced and verified a multidisciplinary framework for the food emergency supply system, which could provide a theoretical basis for emergency responders to make corresponding commands and decisions post-disaster.

Neurological update: consult the neurosurgical oracle for a standard operating procedure

Standard operating procedures (SOPs) contain general instructions and principles to standardize care, to improve effective and safe healthcare. Developing new, or updating current, SOPs is, however, challenging in fields where high-level evidence is limited. Still, SOPs alone have been shown to result in less complications. In this narrative review, we describe the process of creating a consensus-based SOP that is pragmatic for clinical practice since it can be created regardless of the current level of evidence. Through live audience engagement platforms, a group of experts will be able to both anonymously respond to a created questionnaire, and (subsequently) discuss the results within the same meeting. This modified Digital Delphi method as described here can be used as a tool toward consensus-based healthcare.