Using a Delphi process to establish consensus on emergency medicine clerkship competencies

Identifying competencies for an advanced practice physiotherapy role within an interprofessional chronic pain clinic

BACKGROUND

Advanced practice physiotherapy (APP) models of care have shown success in access, safety, satisfaction, and care quality for musculoskeletal pain conditions in various settings. Yet, there is a gap in defining competencies for physiotherapists to be the initial point of contact for people with chronic pain. This study aims to identify and agree upon the competencies necessary for a physiotherapist to fulfill the role of an APP in an interprofessional chronic pain clinic.

METHODS

Three focus groups were conducted using a Nominal Group Technique and a modified Delphi process. Consensus on the competency, defined as agreement by ≥ 75% of participants, was sought.

RESULTS

Twenty-three experts (17 healthcare providers and six individuals with chronic pain) participated in the focus group discussions. Twenty completed the follow-up Delphi surveys. Ten essential competencies for an APP role in interprofessional chronic pain clinics were identified and achieved consensus: 1) use an evidence-based approach to practice; 2) communicate effectively with the patient; 3) perform a comprehensive assessment; 4) determine pain-related diagnoses; 5) develop therapeutic relationships; 6) provide appropriate care; 7) support patients through transitions in care; 8) collaborate with members of the interprofessional team; 9) advocate for the needs of the patients; and 10) use a reflective approach to practice.

CONCLUSION

This study identified ten competencies essential for physiotherapists to fulfill an APP role within interprofessional chronic pain clinics. These competencies serve as a foundation for informing a training program and future research evaluating the effectiveness of the APP model in this setting.

Virtual reality vs. tablet for procedural comfort using an identical game in children undergoing venipuncture: a randomized clinical trial

Introduction

Recent research has explored the effectiveness of interactive virtual experiences in managing pain and anxiety in children during routine medical procedures, compared to conventional care methods. However, the influence of the specific technology used as an interface, 3-dimensions (D) immersive virtual reality (VR) vs. 2D touch screens, during pediatric venipuncture, remains unexamined. This study aimed to determine if immersive VR is more effective than a tablet in reducing pain and anxiety during short procedures.

Methods

An interactive game was designed by clinicians and psychologists, expert in pain theory, hypnosis, and procedural pain and anxiety relief, and was tailored for both VR and tablet use. Fifty patients were randomly assigned to either the Tablet or VR group. The primary outcome measures were pain and anxiety levels during the procedure. Secondary outcome measures included the need for physical restraint, duration of the procedure, enjoyment levels, and satisfaction ratings from both parents and nurses.

Results

Participants, in both groups, had low levels of pain and anxiety. Physical restraint was infrequently used, procedures were brief, and high satisfaction levels were reported by patients, parents, and nurses.

Discussion

This study suggests that the type of technology used as a support for the game has a minimal effect on the child's experience, with both groups reporting low pain and anxiety levels, minimal physical restraint, and high enjoyment. Despite immersive VR's technological advancements, this study underscores the value of traditional tablets with well-designed interactive games in enhancing children's wellbeing during medical procedures.

Clinical Trial Registration

[ClinicalTrials.gov], identifier [NCT05065307].

Expert consensus on the attributes and competencies required for rural and remote junior physicians to work effectively in isolated indonesian communities

Indonesian physicians working in rural and remote areas must be equipped not only with generic competencies but also with the attributes and skills necessary to provide health care services without compromising quality. This study sought to reach a consensus on the attributes and competencies that are viewed as essential and important for working effectively as an early career doctor in rural and remote practice in Indonesia. A two-round Delphi study was conducted by reference to 27 consenting physicians working in rural and remote Indonesia. Forty-three items covering 9 attributes and 34 competencies were sent to these physicians to be rated on a Likert scale ranging from 1 to 5 in terms of their importance for effective rural and remote practice. Nine attributes and 29 competencies progressed to Round 2. All nine attributes and 29 competencies were identified as essential or important for junior physicians' ability to be effective in their practice. The essential attributes included professional quality related to prioritising the rural community. The essential competencies included medical skills, professional behaviour, interprofessional skills, health promotion and connection to the rural community. The consensus thus reached on these essential and important attributes and competencies can inform curriculum development for the undergraduate and postgraduate training of junior rural and remote physicians.

Defining a competency framework for health and social professionals to promote healthy aging throughout the lifespan: an international Delphi study

The promotion of healthy aging has become a priority in most parts of the world and should be promoted at all ages. However, the baseline training of health and social professionals is currently not adequately tailored to these challenges. This paper reports the results of a Delphi study conducted to reach expert agreement about health and social professionals' competencies to promote healthy aging throughout the lifespan within the SIENHA project. Materials and methods: This study was developed following the CREDES standards. The initial version of the competence framework was based on the results of a scoping review and following the CanMEDS model. The expert panel consisted of a purposive sample of twenty-two experts in healthy aging with diverse academic and clinical backgrounds, fields and years of expertise from seven European countries. Agreement was reached after three rounds. The final framework consisted of a set of 18 key competencies and 80 enabling competencies distributed across six domains. The SIENHA competence framework for healthy aging may help students and educators enrich their learning and the academic content of their subjects and/or programs and incentivize innovation.

Identifying teaching competencies for medical residents using a modified Delphi method

PURPOSE

This study aimed to identify the teaching competencies of medical residents.

METHODS

A modified Delphi study was conducted from January to March 2017. Twenty-four panelists (six medical educators, program directors, chief residents, and residents each) from various facilities in Japan participated in the study. The consensus criterion for this study was that more than 80% of the panelists gave a rating of 6 or higher on the 7-point Likert scale ("not at all important" to "extremely important") without any comments. The modified Delphi approach resulted in a list of 27 resident teaching competencies after three rounds. These competencies were categorized based on Harden and Crosby's 12 roles of medical teachers.

RESULTS

Our study revealed that, of the 12 roles, residents were primarily viewed as "clinical or practical teachers," "teaching role models," "on-the-job role models," "learning facilitators," and "student assessors."

CONCLUSION

The 27 resident teaching competencies indicate the importance of educational proximity for residents as teachers. It is expected that this finding will contribute to competency-based resident-as-teacher education.

Development and delivery of a higher diploma in emergency medicine and critical care for clinical officers in Kenya

The provision of emergency medicine and critical care in a cost-efficient manner has the potential to address many preventable deaths in low- and middle-income countries. Here, utilising Kern's framework for curriculum development, we describe the origins, development and implementation of the Emergency Medicine and Critical Care Clinical Officer training program; Kenya's first training programme for clinical officers in emergency medicine and critical care. Graduates are scattered across the country in diverse settings, ranging from national referral hospitals in the capital, Nairobi, to rural hospitals in northern Kenya. In these locations, they provide clinical care, leadership, and teaching. Similar programmes could be replicated in other locations to help plug the gap in critical care provision in Sub-Saharan Africa.

Development and validation of a pharmacist-led education model in allergic rhinitis management: a multi-phase study

BACKGROUND

Patient education is identified as one of the core and fundamental management strategies in the management of allergic rhinitis. The Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines developed guidance for the management of allergic respiratory disease, and the guidelines are applicable to the international context. The ARIA guidelines for the pharmacy have specifically encouraged the creation of local pharmacist-led intervention in allergic rhinitis management. This study aims to develop a pharmacist-led educational model using a multi-phase study approach.

METHOD

In phase one, we conducted a literature review using four databases to extract relevant articles and clinical practice guidelines published between 2017 and 2022. The information was structured into a questionnaire consisting of patient education material (10 domains with 130 items) and pharmacist counseling scopes (15 domains with 43 items), with each item having a rating scale ranging from 1 (lowest) to 9 (highest) level of agreement. Fifty-two panellists, including otorhinolaryngologists and pharmacists, were invited to complete the questionnaire. A consensus agreement was considered when at least 70% of panellists scored 7 to 9 (critically important). A two-round survey was conducted, and descriptive analysis, inter-rater reliability (≥ 0.5-1 indicate moderate to excellent reliability), variation in the relative interquartile (VRIR < 0.3 indicate good stability), and variation in the coefficient of variation (VCV < 40% considered consensus achieved) were performed. In phase two, patient education material was developed into audio-visual format, and in phase three, patients rated its understandability and actionability using a validated Patient Education Materials Assessment Tool.

RESULTS

In the round one Delphi survey, 43 panellists responded, with 171 out of 173 items achieving "consensus agreement" (75.4-100%). In the second survey, 32 out of 43 panellists responded, with most items (171 out of 173 items) stable across rounds and all items had acceptable internal consistency (VCV: - 12.21-15.81). Two items did not achieve "consensus agreement" (64%) but improved in round two (92.9%), however, instability was observed (VRIR: 0.36). These two items were retained in the model due to achieving the minimum level of agreement and internal consistency (VCV = 15.81). Inter-rater reliability was 0.608 and 0.970 in the respective rounds. Patients rated the educational material as understandable (81.8-100%) and actionable (100%).

CONCLUSION

The validated pharmacist-led education model, with its educational materials tested on end-users, provides structured patient education and pharmaceutical care in assisting patients with allergic rhinitis. The educational material allows the delivery of standardized information by the healthcare providers to the patients. Further research on the effectiveness of this model in improving patients' symptom control and quality of life is warranted.

The Shouldice Method: an expert's consensus

BACKGROUND

Hernia repairs are a commonly performed surgical procedure and the Shouldice Repair of inguinal hernias has been well described before in terms of its outcomes. However, the purpose of this paper was to understand what experts from Shouldice Hospital consider to be the essential steps to the Shouldice Method.

METHODS

Utilizing a Delphi Methodology, surgeons at Shouldice Hospital who are considered content experts, provided their answers on what is essential to the Shouldice Method. The median, interquartile range, and percent agreement from the responses as well as the Delphi's overall Cronbach's Alpha were determined. All Delphi items were ranked on a five-point Likert scale and consensus was reached when Cronbach's Alpha was ≥ 0.8. The items from the survey that ranked as 5-completely agree or 4-partially agree by ≥ 80% of participants on the five-point Likert scale were included in the final framework.

RESULTS

The final consensus for the Shouldice Method included 39 items with 7 overarching steps: preoperative preparation of the patient, anesthetic component, incision and nerve identification, treatment of the cremasteric muscles, hernia identification and treatment, reconstruction of the posterior wall, and post-operative management of patients.

CONCLUSION

The results of this consensus provide a step-by-step approach to the Shouldice Method, as well as information that is timely and can be utilized by surgeons incorporating non-mesh hernia repairs into their practice.

Competency Framework for Podiatric Medicine Training in Canada: An Adapted Delphi Study

PURPOSE

Podiatrists are generally defined as professionals with high-level skills in the prevention and management of local foot conditions that are not systemic diseases. Across countries, different academic trainings are implemented due to the specific context and practice of podiatric medicine. It is thus essential to support country-specific podiatry education for the development of highly skilled podiatrists. Therefore, we report the development of a podiatric medicine competency framework to support training in Canada.

PARTICIPANTS AND METHODS

A Delphi process was conducted by 12 stakeholders (including 8 podiatry experts) from the University of Québec at Trois-Rivières which is the only university offering the degree of Doctor of Podiatric Medicine (DPM) in Canada. The developed framework is (1) based on the seven key roles of the Canadian medical education directives of specialists (CanMEDs) and, (2) closely aligned with the requirement of the College of Podiatrists of Québec which sets the standards of entry to practice in Québec.

RESULTS

The developed framework represents the state of the development process and the consensus of the podiatry experts. It reflects the expected profile of the institution's DPM graduates based on seven key roles (podiatry expert, communicator, collaborator, health advocate, leader and manager, scholar, and professional). This developed framework is an arborescence of complex skills defined in tangible indicators that characterize each expected part of a core competency. Twenty-four core competencies have been determined and divided into 84 enabling competencies and 288 observable indicators.

CONCLUSION

This competency framework has been designed to support high-quality education and to develop podiatry. Next steps include: (1) validation of this framework by external experts, (2) development of rigorous evaluation methods and, (3) concrete actions for its implementation and assessment. This framework would help to define the scope of practice and capabilities of podiatric medicine, both in Canada and internationally.

Development of a psychosocial oncology core curriculum for multidisciplinary education and training: Initial content validation using the modified Delphi Method

OBJECTIVE

Despite its four decade history, the multidisciplinary specialty of psychosocial oncology (PSO) has no official consensus on core content. In 2014, the American Psychosocial Oncology Society (APOS) Board charged the APOS Professional Education Committee with outlining curricular content needed for core competence.

METHODS

Content validation was completed using a four-phase modified Delphi Method. During Phase I, a Professional Education Committee subgroup proposed domains and items, which were rated by the APOS Fellows and Board via online survey. During Phase II, Fellows completed a second, revised survey. Phase III incorporated early career members. Developmental and diversity items were integrated into each domain. In Phase IV, a larger group of subject matter experts were surveyed, with feedback incorporated. Validation across phases involved average rating thresholds, intraclass correlations, and final percent agreement.

RESULTS

The Delphi Method supported 12 content domains: Cancer Basics, Psychosocial Oncology, Professional Development, Ethics, Emotional/Psychological Concerns, Sexuality and Relationship Concerns, Spiritual/Religious Concerns, Healthcare Communication and Decision Making, Social/Practical Problems, Caregiver Concerns, Cognitive Concerns, Physical Symptoms and Psychosocial Assessment/Treatment. High levels of agreement were achieved across domains (86%-100%) and items, with no significant rating differences by discipline.

CONCLUSIONS

This proposed core content can enhance and standardize education and training in PSO including APOS' Virtual Psychosocial Oncology Core Curriculum, focused on broadly expanding the PSO workforce, particularly in underserved areas. Next steps include development of core competencies and establishment of online training modules based on this content validation.

Structured evaluation of stress triggers in prehospital emergency medical care : An analysis by questionnaire regarding the professional groups

BACKGROUND

Emergency medical services work in the environment of high responsibility teams and have to act under unpredictable working conditions. Stress occurs and has potential of negative effects on tasks, teamwork, prioritization processes and cognitive control. Stress is not exclusively dictated by the situation-the individuals rate the situation of having the necessary skills that a particular situation demands. There are different occupational groups in the emergency medical services in Germany. Training, tasks and legal framework of these groups vary.

OBJECTIVE

The aim of this study was to identify professional group-specific stressors for emergency medical services. These stress situations can be used to design skills building tools to enable individuals to cope with these stressors.

MATERIAL AND METHODS

The participants were invited to the study via posters and social media. An expert group (minimum 6 months of experience) developed a set of items via a two-step online Delphi survey. The experts were recruited from all professional groups represented in the German emergency medical service. We evaluated the resulting parameters for relevance and validity in a larger collective. Lastly, we identified stress factors that could be grouped in relevant scales. In total 1017 participants (paramedics, physicians) took part in the final validation survey.

RESULTS

After validation, we identified a catalogue of stressors with 7 scales and 25 items for EMT (Emergency Medical Technician) paramedics (KMO [Kayser-Meyer-Olkin criterion] 0.81), 6 scales and 24 items for advanced paramedics (KMO 0.82) and 6 scales and 24 items for EMS (Emergency Medical Service) physicians (KMO 0.82). For the professional group of EMT basic, the quality parameters did not allow further processing of the items. Professional group-specific scales for EMT paramedics are "professional limitations", "organizational framework", "expectations" and "questions of meaning". For advanced paramedics "appreciation", "exceptional circumstances" and "legal certainty" were identified. The EMT physicians named "handling third parties", "tolerance to ambiguity", "task management" and "pressure to act". A scale that is representative for all professional groups is "teamwork". Organizational circumstances occur in all groups. The item "unnecessary missions" for EMT paramedics and "legal concerns with the application of methods" for advanced paramedics are examples.

DISCUSSION

Different stressors are relevant for the individual professional groups in the German emergency medical service. The developed catalogue can be used in the future to evaluate the subjective stress load of emergency service professionals. There are stressors that are inherent in the working environment (e.g. pressure to act) and others that can be improved through training (teamwork). We recommend training of general resistance as well as training of specific items (e.g., technical, nontechnical skills). All professionals mentioned items with respect to organizational factors. The responsible persons can identify potential for improvement based on the legal and organizational items. The EMT basic requires further subdivision according to task areas due to its variable applicability.

Using the Delphi technique to determine objectives and topical outline for a pharmaceutical care course: an experience from the Cuban higher education system

BACKGROUND

Being pharmaceutical care one of the four areas defined by the International Pharmaceutical Federation's Global Competence Framework, the curriculum redesigned scheme is a priority task to perform a pharmaceutical workforce capable to contribute significantly to the appropriate use of medicines. Therefore, the pharmacy curriculum should be adapted, in order to provide pharmacists with new knowledge and skills to provide pharmaceutical care services. This study used a modified Delphi technique to define objectives and topical outlines for a pharmaceutical care course on a pharmacy curriculum.

METHODS

A modified Delphi process was used to determine a consensus among proposed course objectives and topical outlines. The preliminary phase of the study included a compilation of prospective objectives and outline topics on which to structure informational flow through the Delphi. A two-round modified Delphi process were completed by the participants in the study. The Delphi questionnaire was organized using six domains: the theoretical program foundation; recommended for teaching literature; instructional and educational objectives of the program (course structure); teaching methods; knowledge, skills and professional values considered; and students' performance assessments. Nineteen items for evaluation within the referred domains were considered.

RESULTS

Consensus was achieved among 15 participants regarding 10 objectives, and eleven topical outlines related to pharmaceutical care teaching in an undergraduate pharmacy course. Despite this favorable valuation and considering the qualitative evaluations provided by the participants, it was believed appropriate to analyze the recommendation for the inclusion of literature for the teaching of the course in Spanish language (73%). It resulted in a project proposal for the elaboration of a book by a group of authors from all the faculties of pharmacy in the country.

CONCLUSION

A Delphi expert panel achieved consensus on topical outline and objectives for a pharmaceutical care course. The results of this study can be used to underline the didactic guidance for pharmaceutical care teaching and learning useful for future pharmacy curriculum upgrades.

A Modified Delphi Involving Laboratory Faculty to Define Essential Skills for Pharmacy Graduates

Objective. To define essential skills for Doctor of Pharmacy (PharmD) graduates that are needed in the four most common sectors of pharmacy practice as determined by expert faculty who instruct within pharmacy skills laboratories.Methods. A three-round Delphi method was used to establish consensus. In the first round, participants were asked what skills were needed by students at entry to practice in community, health-system, ambulatory care, and managed care pharmacy settings. In rounds two and three, participants were asked to rate each skill with a level of importance using a 10-point Likert scale (1=not important to 10=very important).Results. In round one, participants produced a collective list of 289 essential skills. These skill statements were sent to participants in rounds two and three to assign a level of importance. After the third round, participants reached consensus using a mean level of importance for a final list of 69 community pharmacy skills, 47 health-system, 60 ambulatory care, and 15 managed care skills. These skills were then mapped to entrustable professional activities domains for schools and colleges pharmacy to use as a resource when assessing core competency development in the curriculum.Conclusion. The Delphi technique was used successfully with expert pharmacy skills laboratory faculty to identify laboratory-focused essential skills that recent PharmD graduates should have prior to entering community, health-system, ambulatory care, or managed care pharmacy practice. These essential skills can be used to guide curriculum development, develop milestone markers, and help ensure students are practice ready.

Identifying Cancer Care Team Competencies to Improve Care Coordination in Multiteam Systems: A Modified Delphi Study

PURPOSE

Identifying nontechnical, teamwork competencies (knowledge, skills, and attitudes) underlying coordination within and across the network of teams-or multiteam system (MTS) involved in cancer care is foundational to optimizing high-quality cancer care.

METHODS

A multidisciplinary group of cancer care stakeholders refined an initial list of competency statements during three rounds of a web-based modified Delphi survey.

RESULTS

Panelists reached consensus on a final list of four domains and 20 associated team-based competencies important for effective coordination in cancer care MTS.

CONCLUSION

This study provides an initial foundation for testing, modifying, measuring and evaluating the impact of identified competencies on care coordination, outcomes, and costs, for people being screened, treated, or surviving cancer.

Core Curricular Priorities in the Care of Children With Medical Complexity: A North American Modified Delphi Study

BACKGROUND

Previous studies of pediatric residents have identified educational gaps in caring for children with medical complexity. Training opportunities in complex care vary across residency programs. Defining core curricular topics in complex care is a priority in medical education.

OBJECTIVE

To identify core topics to include in a standard complex care curriculum for pediatric residents.

METHODS

An initial topic list was generated through literature review and proceedings of national meetings. Expert panelists were identified based on experience in complex care and residency education. A modified Delphi method was used to determine group consensus by asking participants to rate the importance of complex care curricular topics for pediatric residents. Consensus was defined as >70% of experts identifying a topic as essential. There was a predetermined maximum of 3 iterative, electronic survey rounds, with feedback provided to participants between each round.

RESULTS

Sixteen experts participated. Response rate was 100% for all rounds. Experts were from the United States (44%) and Canada (56%); most were affiliated with an academic medical center (96%) involved in both inpatient and outpatient care (69%). Eleven topics were identified as essential across 3 rounds: feeding difficulties, pain/irritability, transition, feeding tube management, difficult discussions, team management/care coordination, dysmotility, aspiration, safety/emergency planning, neuromuscular/skeletal issues, and advocacy. Essential topics were organized according to the International Classification of Functioning, Disability and Health.

CONCLUSIONS

Eleven curricular priorities in complex care were identified across multiple domains of the International Classification of Functioning, Disability and Health framework, serving as a guide for standardized curriculum development for future pediatricians.

Identifying appropriate outcomes to help evaluate the impact of the Canadian Guideline for Safe and Effective Use of Opioids for Non-Cancer Pain

BACKGROUND

The Canadian Guideline for Safe and Effective Use of Opioids for Chronic Non-Cancer Pain (COG) was developed in response to increasing rates of opioid-related hospital visits and deaths in Canada, and uncertain benefits of opioids for chronic non-cancer pain (CNCP). Following publication, we developed a list of evaluable outcomes to assess the impact of this guideline on practice and patient outcomes.

METHODS

A working group at the National Pain Centre at McMaster University used a modified Delphi process to construct a list of clinical and patient outcomes important in assessing the uptake and application of the COG. An advisory group then reviewed this list to determine the relevance and feasibility of each outcome, and identified potential data sources. This feedback was reviewed by the National Faculty for the Guideline, and a National Advisory Group that included the creators of the COG, resulting in the final list of 5 priority outcomes.

RESULTS

Five outcomes were judged clinically important and feasible to measure: 1) Effects of opioids for CNCP on quality of life, 2) Assessment of patient's risk of addiction before starting opioid therapy, 3) Monitoring patients on opioid therapy for aberrant drug-related behaviour, 4) Mortality rates associated with prescription opioid overdose and 5) Use of treatment agreements with patients before initiating opioid therapy for CNCP. Data sources for these outcomes included patient's medical charts, e-Opioid Manager, prescription monitoring programs and administrative databases.

CONCLUSION

Measuring the impact of best practice guidelines is infrequently done. Future research should consider capturing the five outcomes identified in this study to evaluate the impact of the COG in promoting evidence-based use of opioids for CNCP.

A Delphi consensus study for teaching "Basic Trauma Management" to third-year medical students

Background

The Basic-Trauma Management (BTM) course has been taught to third-year medical students in small groups for many years without substantial changes. With the introduction of a new curriculum for Swiss medical students, it was necessary to revise the BTM content and re-align it. Our aim was to identify core competencies for the revised BTM course.

Methods

We applied a three-round step-wise Delphi consensus. First, we asked open-ended questions on what were the most important competencies to be taught for BTM; the second round used Likert scales to ensure agreement on the competencies; and the final round reached out for consensus on these BTM competencies. Stakeholders were selected based on their long-standing experience in teaching BTM and in managing trauma patients.

Results

Consensus was found on 29 competencies out of an initial 130 proposals. “Human Factors”, which had not been taught previously, scored relatively high, at 22%. The sole specific trauma skill agreed upon was the use of tourniquets.

Conclusions

This is an example of curricular revision of a clinical skills course after the introduction of a regulatory framework for undergraduate medical education. The revised course curriculum tailors the concepts and skills in trauma that fulfill stakeholder needs, and are in agreement with the new Swiss learning outcomes.

Validating curricular competencies in innovation and entrepreneurship for biomedical research trainees: A modified Delphi approach

INTRODUCTION

Biomedical researchers need skills in innovation and entrepreneurship (I&E) to efficiently translate scientific discoveries into products and services to be used to improve health.

METHODS

In 2016, the European Union identified and published 15 entrepreneurial competencies (EntreComp) for the general population. To validate the appropriateness of these competencies for I&E training for biomedical researchers and to identify program content, we conducted six modified Delphi panels of 45 experts (6-9 per panel). Participating experts had diverse experience, representing such fields as entrepreneurship, academic research, venture capital, and industry.

RESULTS

The experts agreed that all 15 EntreComp competencies were important for biomedical research trainees and no additional competencies were identified. In a two-round Delphi process, the experts identified 120 topics to be included in a training curriculum. They rated the importance of each topic using a 5-point scale from not at all important (1) to extremely important (5) for two student groups: entrepreneurs (those interested in starting their own ventures) and intrapreneurs (those wanting to be innovative and strategic within academia or industry). Consensus (mean importance score >4) was reached that 85 (71%) topics were of high importance for the curriculum. Four topics were identified by multiple panels for both student groups: resiliency, goal setting, team management, and communication skills.

CONCLUSIONS

I&E training for biomedical trainees should address all 15 EntreComp competencies, including "soft skills," and be flexible to accommodate the needs of trainees on different career trajectories.

Instructors for on-the-job training of advanced paramedics - definition of competencies and development of a quality management tool for a "High Responsibility Organization"

Objective: The psychological demands placed upon the emergency medical services, assures them of their place amongst High Responsibility Organizations. A high pressure to act and an irreversibility of situations are integral features of their workplaces. After the emergency services' job profiles were restructured in Germany, the practical stage of paramedic training is now undertaken in these conditions. That is, they are trained by a supervising instructor whilst caring for critically ill patients. This paper aims to describe the requisite skills for such an instructor, formulate the associated competences as learning objectives, and develop a quality-measuring instrument for the description of training situations. Methods: The compilation of a competence catalogue was done via a two-step process: following a Delphi survey with an expert panel of practical trainers and trainees, a large cohort of parameters were validated in terms of their relevance. Those factors that formed scales together were identified. Results: After validating the results of the Delphi analysis, six scales (composed of 25 items in toto) were defined. They included the areas of "Training during times of action", "Training during periods of calm", "Background and practical relevance", "Character and personality traits", "Pedagogical competencies" and "Organizational behaviour". Conclusion: For the first time, a competency catalogue has been developed for instructors working in the emergency medical services from German-speaking countries. The catalogue focuses upon clinical training during the acute care of critically ill patients. The scales and items can be used for training-the-trainers, and also quality monitoring. Further research needs to focus on the application of the catalogue in clinical practice and evaluate the need for situational customization.

Integration of Entrustable Professional Activities with the Milestones for Emergency Medicine Residents

INTRODUCTION

Medical education is moving toward a competency-based framework with a focus on assessment using the Accreditation Council for Graduate Medical Education Milestones. Assessment of individual competencies through milestones can be challenging. While competencies describe characteristics of the person, the entrustable professional activities (EPAs) concept refers to work-related activities. EPAs would not replace the milestones but would be linked to them, integrating these frameworks. Many core specialties have already defined EPAs for resident trainees, but EPAs have not yet been created for emergency medicine (EM). This paper describes the development of milestone-linked EPAs for EM.

METHODS

Ten EM educators from across North America formed a consensus working group to draft EM EPAs, using a modified Glaser state-of-the-art approach. A reactor panel with EPA experts from the United States, Canada and the Netherlands was created, and an iterative process with multiple revisions was performed based on reactor panel input. Following this, the EPAs were sent to the Council of Residency Directors for EM (CORD-EM) listserv for additional feedback.

RESULTS

The product was 11 core EPAs that every trainee from every EM program should be able to perform independently by the time of graduation. Each EPA has associated knowledge, skills, attitudes and behaviors (KSAB), which are either milestones themselves or KSABs linked to individual milestones. We recognize that individual programs may have additional focus areas or work-based activities they want their trainees to achieve by graduation; therefore, programs are also encouraged to create additional program-specific EPAs.

CONCLUSION

This set of 11 core, EM-resident EPAs can be used as an assessment tool by EM residency programs, allowing supervising physicians to document the multiple entrustment decisions they are already making during clinical shifts with trainees. The KSAB list within each EPA could assist supervisors in giving specific, actionable feedback to trainees and allow trainees to use this list as an assessment-for-learning tool. Linking each KSAB to individual EM milestones allows EPAs to directly inform milestone assessment for clinical competency committees. These EPAs serve as another option for workplace-based assessment, and are linked to the milestones to create an integrated framework.

Use of a Modified Delphi Process to Determine Course Objectives for a Model Global Health Course in a Pharmacy Curriculum

Objective. To define course objectives for a model global health course in a pharmacy curriculum. Methods. A modified Delphi process was used to determine a consensus among proposed course objectives. A three-round email panel was sent to members of three special interest groups (SIGs) within the American Association of Colleges of Pharmacy (Public Health, Global Pharmacy Education, and Health Disparities and Cultural Competence) to recruit educators broadly interested or engaged in this area of education. An initial list of 80 potential course objectives across 11 domains was proposed for inclusion. Objectives that were cumulatively rated as either "extremely important" or "very important" by at least 75%, 80%, and 85% of respondents in each of the three rounds, respectively, were moved forward (first and second rounds) or accepted (third round). Results. Responses were received from 87, 73, and 70 faculty panel members in the three consecutive rounds. The initial list of proposed objectives was narrowed to 65 objectives (19% reduction), and 38 objectives (53%) after the first and second rounds, respectively. The final list was composed of 20 objectives from seven domains. Global burden of disease and social/environmental determinants of health contained the most objectives selected by consensus. Conclusion. The process identified a consensus for course objectives for a model global health education course. These objectives can be used by pharmacy faculty to align global health education in the profession.

Core Competencies in Evidence-Based Practice for Health Professionals: Consensus Statement Based on a Systematic Review and Delphi Survey

IMPORTANCE

Evidence-based practice (EBP) is necessary for improving the quality of health care as well as patient outcomes. Evidence-based practice is commonly integrated into the curricula of undergraduate, postgraduate, and continuing professional development health programs. There is, however, inconsistency in the curriculum content of EBP teaching and learning programs. A standardized set of minimum core competencies in EBP that health professionals should meet has the potential to standardize and improve education in EBP.

OBJECTIVE

To develop a consensus set of core competencies for health professionals in EBP.

EVIDENCE REVIEW

For this modified Delphi survey study, a set of EBP core competencies that should be covered in EBP teaching and learning programs was developed in 4 stages: (1) generation of an initial set of relevant EBP competencies derived from a systematic review of EBP education studies for health professionals; (2) a 2-round, web-based Delphi survey of health professionals, selected using purposive sampling, to prioritize and gain consensus on the most essential EBP core competencies; (3) consensus meetings, both face-to-face and via video conference, to finalize the consensus on the most essential core competencies; and (4) feedback and endorsement from EBP experts.

FINDINGS

From an earlier systematic review of 83 EBP educational intervention studies, 86 unique EBP competencies were identified. In a Delphi survey of 234 participants representing a range of health professionals (physicians, nurses, and allied health professionals) who registered interest (88 [61.1%] women; mean [SD] age, 45.2 [10.2] years), 184 (78.6%) participated in round 1 and 144 (61.5%) in round 2. Consensus was reached on 68 EBP core competencies. The final set of EBP core competencies were grouped into the main EBP domains. For each key competency, a description of the level of detail or delivery was identified.

CONCLUSIONS AND RELEVANCE

A consensus-based, contemporary set of EBP core competencies has been identified that may inform curriculum development of entry-level EBP teaching and learning programs for health professionals and benchmark standards for EBP teaching.

Development of "Core Syllabus" for Facial Anatomy Teaching to Aesthetic Physicians: A Delphi Consensus

Background:

A detailed understanding of facial anatomy, specifically the vascular framework, is crucial for delivering safe nonsurgical aesthetic procedures. To date, there is no core document based on consensus for the teaching of facial anatomy to aesthetic physicians exists. The aim of this study was to ascertain the most critical anatomical structures for avoiding disastrous complications during nonsurgical aesthetic procedures.

Methods:

After a detailed literature review, Delphi questionnaire was developed listing 154 anatomical structures of the face and neck for consensus review. Thirty-five international experts in surgical and nonsurgical facial aesthetics were invited to complete an online survey designed to rank the relevance of each anatomical element on a Likert scale from 1 (not at all important) to 4 (very important) or 5 (no opinion). Consensus for items included in the core syllabus was predefined as achieving a Cronbach’s α ≥ 0.90 and an agreement score of ≥ 80%.

Results:

Thirty-four international experts (97.14%) completed the iterative online Delphi survey. The consensus among the specialists polled was achieved after 1 round of the survey (Cronbach’s α = 0.94). The expert panel reached an agreement score of ≥ 80% on 137 of the 154 anatomical structures listed.

Conclusion:

The outcome of this Delphi study represents an essential first step in systematizing an evidence-based facial anatomy “Core Syllabus” for the teaching of aesthetic physicians and practitioners. This “Core Syllabus” will act as a blueprint for the educators while crafting a program.

Defining curricular priorities for student-as-teacher programs: A National Delphi Study

BACKGROUND

"Student-as-Teacher" (SaT) programs have been growing in number to prepare medical students for their teaching roles in residency and beyond, but it remains unknown what content areas should be covered in SaT curricula.

AIM

To determine five to ten "essential" content areas for inclusion in SaT curricula using expert opinion.

METHODS

Using a three-round Delphi process, moderators iteratively surveyed a panel of 28 medical educators (25 academy directors and three individuals identified as having expertise in undergraduate medical education) representing 25 medical schools in the United States. This "SaT Delphi Working Group" was tasked with rating topics for inclusion in SaT curricula on a 3-point scale (i.e. 1. "essential," 2. "important, but not essential" 3. "not important"). Topics achieving ≥70% consensus as "essential," "important" or "not important" were accepted by the moderators and removed from subsequent rounds.

RESULTS

Hundred per cent response rate (n = 28) was achieved for all survey rounds. Five content areas reached consensus as "essential" for inclusion in a SaT curriculum: feedback, bedside teaching and clinical precepting, small-group teaching, case-based teaching and professionalism as a medical educator.

CONCLUSION

This consensus from a group of leaders in medical education is a first step toward the implementation of more developmentally-appropriate SaT competencies.

Failure to flow: An exploration of learning and teaching in busy, multi-patient environments using an interpretive description method

INTRODUCTION

As patient volumes continue to increase, more attention must be paid to skills that foster efficiency without sacrificing patient safety. The emergency department is a fertile ground for examining leadership and management skills, especially those that concern prioritization in multi-patient environments. We sought to understand the needs of emergency physicians (EPs) and emergency medicine junior trainees with regards to teaching and learning about how best to handle busy, multi-patient environments.

METHOD

A cognitive task analysis was undertaken, using a qualitative approach to elicit knowledge of EPs and residents about handling busy emergency department situations. Ten experienced EPs and 10 junior emergency medicine residents were interviewed about their experiences in busy emergency departments. Transcripts of the interviews were analyzed inductively and iteratively by two independent coders using an interpretive description technique.

RESULTS

EP teachers and junior residents differed in their perceptions of what makes an emergency department busy. Moreover, they focused on different aspects of patient care that contributed to their busyness: EP teachers tended to focus on volume of patients, junior residents tended to focus on the complexity of certain cases. The most important barrier to effective teaching and learning of managerial skills was thought to be the lack of faculty development in this skill set.

CONCLUSIONS

This study presents qualitative data that helps us elucidate how patient volumes affect our learning environments, and how clinical teachers and residents operate within these environments.

An international survey and modified Delphi process revealed editors' perceptions, training needs, and ratings of competency-related statements for the development of core competencies for scientific editors of biomedical journals

Background: Scientific editors (i.e., those who make decisions on the content and policies of a journal) have a central role in the editorial process at biomedical journals. However, very little is known about the training needs of these editors or what competencies are required to perform effectively in this role. Methods: We conducted a survey of perceptions and training needs among scientific editors from major editorial organizations around the world, followed by a modified Delphi process in which we invited the same scientific editors to rate the importance of competency-related statements obtained from a previous scoping review. Results: A total of 148 participants completed the survey of perceptions and training needs. At least 80% of participants agreed on six of the 38 skill and expertise-related statements presented to them as being important or very important to their role as scientific editors. At least 80% agreed on three of the 38 statements as necessary skills they perceived themselves as possessing (well or very well).  The top five items on participants' list of top training needs were training in statistics, research methods, publication ethics, recruiting and dealing with peer reviewers, and indexing of journals. The three rounds of the Delphi were completed by 83, 83, and 73 participants, respectively, which ultimately produced a list of 23 "highly rated" competency-related statements and another 86 "included" items. Conclusion: Both the survey and the modified Delphi process will be critical for understanding knowledge and training gaps among scientific editors when designing curriculum around core competencies in the future.

Making the McMOST out of Milestones and Objectives: Reimagining Standard Setting Using the McMaster Milestones and Objectives Stratification Technique

BACKGROUND

As we enter the era of milestones and competency-based medical education (CBME), there is an increasing need to examine the procedures for stratifying objectives into levels of achievement. Most techniques used to date (e.g., Delphi surveys) involve some sort of consensus-based process, essentially crowd-sourcing wisdom of multiple educators to set anticipated milestones. In most graduate education settings, however, many simply use the judgment of one or two educators when setting educational objectives. Meanwhile, standard-setting procedures have been historically used in medical education for setting cut-points to determine levels of acceptable performance and do so in a more robust manner.

OBJECTIVES

Inspired by these standard-setting procedures, the authors sought to develop a new way to stratify objectives into three relative levels of achievement (junior [ACGME level 1], intermediate [ACGME level 2-3], senior [ACGME level 4]).

METHODS

The authors describe a novel, stepwise method that is composed of four steps. There are four steps to the McMOST procedure: 1) sorting objectives with a group of experienced teachers, 2) factor analysis to group preferences, 3) labeling components and reorganizing groupings, and 4) confirmation and final review by educational and content experts.

RESULTS

Using McMOST method resulted in a change of placement for 15 of 34 (44%) of the milestones and improved agreement in two of three levels (intermediate from intraclass correlation of 0.56 to 0.80; senior from 0.69 to 0.79).

CONCLUSIONS

The authors describe a novel protocol for stratifying objectives that may be useful to stratify and sort competencies into various levels of achievement (e.g., milestones) in this era of CBME.

A Modified Delphi Process to Define Competencies for Assessment Leads Supporting a Doctor of Pharmacy Program

Objective. To define the competencies for individuals designated as assessment leads in colleges and schools of pharmacy. Methods. Twenty-three assessment experts in pharmacy participated in a modified Delphi process to describe competencies for an assessment lead, defined as the individual responsible for curricular assessment and assessment-related to doctor of pharmacy program accreditation. Round 1 asked open-ended questions about knowledge, skills, and attitudes. Round 2 grouped responses for comment and rating for consensus, which was prospectively set at 80%. Results. Twelve competencies were defined and grouped into 3 areas: Context for Assessment, Managing the Process of Assessment, and Leadership of Assessment Activities. In order to verify the panel's work, assessment competencies from other disciplines were reviewed and compared. Conclusions. The competencies describe roles for assessment professionals as experts, managers, and leaders of assessment processes. They can be used by assessment professionals in self-assessing areas for professional development and by administrators in selecting, developing, and supporting designated leads.

An expert consensus on core competencies in integrated care for psychiatrists

OBJECTIVE

All psychiatry residents in Canada are required to train in integrated care (also known as "shared care" or "collaborative care"). We sought to define the competencies required for integrated care practice, with an emphasis on those competencies necessary for all psychiatric postgraduate learners regardless of their intended future practice setting or population.

METHOD

We conducted a mixed methods study including qualitative interviews with nine psychiatrists practicing integrated care across Canada and a quantitative survey of 35 experts using a modified Delphi method.

RESULTS

Our participants believed that integrated care aims to build capacity for improved quality of mental health care in unspecialized settings, and as such, its practice requires broad clinical expertise as well as competencies in interprofessional teamwork, collaborative leadership, knowledge exchange and program consultation. All psychiatrists require knowledge of evidence-based models of integrated care and the ability to work with organizations to implement these models.

CONCLUSION

Psychiatrists are best prepared for integrated care practice through clinical exposure to primary care and/or community settings, as well as didactic teaching regarding the evidence for integrated care, quality improvement methods, leadership, health systems and population health.

Precepting at the time of a natural disaster

BACKGROUND

Natural disasters strike communities that have varied degrees of preparedness, both physical and psychological. Rural communities may be particularly vulnerable as they often do not have the infrastructure or resources to prepare in advance. The psychological impact of a natural disaster is amplified in learners who may be temporary members of the community and therefore cannot draw on personal support during the crisis. They may turn to their clinical preceptors for guidance.

CONTEXT

The Slave Lake fire (population 6782) in May 2011 and the High River flood (population 12 920) in June 2013 are examples of natural disasters that have occurred in rural Alberta, Canada. At the time of these critical incidents, three medical students and one family medicine resident from the two provincial medical schools were participating in rotations in these communities.

INNOVATION

Although disasters occur rarely, there is a need for guidelines for preceptors from the learner perspective. Accordingly, using a modified Delphi approach, we captured the experiences of learners that were then refined into two themes, each containing three recommendations: considerations for action during a natural disaster and considerations for action after the acute crisis has passed. Although disasters occur rarely, there is a need for guidelines for preceptors from the learner perspective IMPLICATIONS: Our recommendations provide suggestions for practical solutions that build on the usual expectations of mentors and may benefit the student-teacher relationship at the time of a disaster and beyond. They are meant to initiate discussion regarding further study aimed towards creating recommendations for preceptor response that may cross disciplines.

Core competencies for health professionals' training in pediatric behavioral sleep care: a Delphi study

The need to train non-sleep-specialist health professionals in evidence-based pediatric behavioral sleep care is well established. The objective of the present study was to develop a list of core competencies for training health professionals in assisting families of 1- to 10-year old children with behavioral insomnia of childhood. A modified Delphi methodology was employed, involving iterative rounds of surveys that were administered to 46 experts to obtain consensus on a core competency list. The final list captured areas relevant to the identification and treatment of pediatric behavioral sleep problems. This work has the potential to contribute to the development of training materials to prepare non-sleep-specialist health professionals to identify and treat pediatric behavioral sleep problems, ideally within stepped-care frameworks.

Content Validation of a Novel Screening Tool to Identify Emergency Department Patients With Significant Palliative Care Needs

BACKGROUND

The emergency department (ED) is increasingly used by patients with life-limiting illness. These patients are frequently admitted to the hospital, where they suffer from poorly controlled symptoms and are often subjected to marginally effective therapies. Palliative care (PC) has emerged as the specialty that cares for patients with advanced illness. PC has been shown to reduce symptoms, improve quality of life, and decrease resource utilization. Unfortunately, most patients who could benefit from PC are never identified. At present, there exists no validated screening tool to identify significant unmet PC needs among ED patients with life-limiting illness.

OBJECTIVES

The objective was to develop a simple, content-valid screening tool for use by ED providers to identify ED patients with significant PC needs. A positive screen would result in an inpatient PC consultation.

METHODS

An initial screening tool was developed based on a critical review of the literature. Content validity was determined by a two-round modified Delphi technique using a panel of PC experts. The expert panel reviewed the items of the tool for accuracy and necessity using a Likert scale and provided narrative feedback. Expert's responses were aggregated and analyzed to revise the tool until consensus was achieved. Greater than 80% agreement, as well as meeting Lawshe's critical values, was required to achieve consensus.

RESULTS

Fifteen experts completed two rounds of surveys to reach consensus on the content validity of the tool. Three screening items were accepted with minimal revisions. The remaining items were revised, condensed, or eliminated. The final tool contains 13 items divided into three steps: 1) presence of a life-limiting illness, 2) unmet PC needs, and 3) hospital admission. The majority of panelists (86%) endorsed adoption of the final screening tool.

CONCLUSIONS

Use of a modified Delphi technique resulted in the creation of a content-validated screening tool for identification of ED patients with significant unmet PC needs. Further validation testing of the instrument is warranted.

Operationalising elaboration theory for simulation instruction design: a Delphi study

OBJECTIVE

The aim of this study was to assess the feasibility of incorporating the Delphi process within the simplifying conditions method (SCM) described in elaboration theory (ET) to identify conditions impacting the complexity of procedural skills for novice learners.

METHODS

We generated an initial list of conditions impacting the complexity of lumbar puncture (LP) from key informant interviews (n = 5) and a literature review. Eighteen clinician-educators from six different medical specialties were subsequently recruited as expert panellists. Over three Delphi rounds, these panellists rated: (i) their agreement with the inclusion of the simple version of the conditions in a representative ('epitome') training scenario, and (ii) how much the inverse (complex) version increases LP complexity for a novice. Cronbach's α-values were used to assess inter-rater agreement.

RESULTS

All panellists completed Rounds 1 and 2 of the survey and 17 completed Round 3. In Round 1, Cronbach's α-values were 0.89 and 0.94 for conditions that simplify and increase LP complexity, respectively; both values increased to 0.98 in Rounds 2 and 3. With the exception of 'high CSF (cerebral spinal fluid) pressure', panellists agreed with the inclusion of all conditions in the simplest (epitome) training scenario. Panellists rated patient movement, spinal anatomy, patient cooperativeness, body habitus, and the presence or absence of an experienced assistant as having the greatest impact on the complexity of LP.

CONCLUSIONS

This study demonstrated the feasibility of using expert consensus to establish conditions impacting the complexity of procedural skills, and the benefits of incorporating the Delphi method into the SCM. These data can be used to develop and sequence simulation scenarios in a progressively challenging manner. If the theorised learning gains associated with ET are realised, the methods described in this study may be applied to the design of simulation training for other procedural and non-procedural skills, thereby advancing the agenda of theoretically based instruction design in health care simulation.

Frontline over ivory tower: key competencies in community-based curricula

BACKGROUND

The Royal College of Physicians and Surgeons of Canada mandates that community experiences be incorporated into medicine-based specialties. Presently there is wide variability in community endocrine experiences across Canadian training programs. This is complicated by the paucity of literature providing guidance on what constitutes a 'community' rotation.

METHOD

A modified Delphi technique was used to determine the CanMEDS competencies best taught in a community endocrinology curriculum. The Delphi technique is a qualitative-research method that uses a series of questionnaires sent to a group of experts with controlled feedback provided by the researchers after each survey round. The experts in this study included endocrinology program directors, community endocrinologists, endocrinology residents and recent endocrinology graduates.

RESULTS

Thirty four out of 44 competencies rated by the panel were deemed suitable for a community curriculum. The experts considered the "Manager" role best taught in the community, while they considered the community least suitable to learn the "Medical Expert" competency.

CONCLUSIONS

To our knowledge, this is the first time the content of a community-based subspecialty curriculum was determined using the Delphi process in Canada. These findings suggest that community settings have potential to fill in gaps in residency training in regards to the CanMEDS Manager role. The results will aid program directors in designing competency-based community endocrinology rotations and competency-based community rotations in other medical subspecialty programs.

Consensus development for healthcare professionals

Consensus development sprang from a desire to synthesize clinician and expert opinions on clinical practice and research agendas in the 1950s. And since the American Institute of Medicine formally defined "guidelines" in 1990, there has been a proliferation of clinical practice guidelines (CPG) both formally and informally. This modern decision-making tool used by both physicians and patients, requires extensive planning to overcome the challenges of consensus development while reaping its rewards. Consensus allows for a group approach of multiple experts sharing ideas to form consensus on topics ranging from appropriateness of procedures to research agenda development. Disagreements can shed light on areas of controversy and launch further discussions. It has five main components: three inputs (defining the task, participant identification and recruitment, and information synthesis), the approach (consensus development by explicit or implicit means), and the output (dissemination of results). Each aspect requires extensive planning a priori as they influence the entire process, from how information will be interpreted, the interaction of participants, the resulting judgment, to whether there will be uptake of results. Implicit approaches utilize qualitative methods and/or a simple voting structure of majority wins, and are used in informal consensus development methods and consensus development conferences. Explicit approaches aggregate results or judgments using explicit rules set a priori with definitions of "agreement" or consensus. Because the implicit process can be more opaque, unforeseen challenges can emerge such as the undue influence of a minority. And yet, the logistics of explicit approaches may be more time consuming and not appropriate when speed is a priority. In determining which method to use, it is important to understand the pros and cons of different approaches and how it will affect the overall input, approach, and outcome.

What adult electrocardiogram (ECG) diagnoses and/or findings do residents in emergency medicine need to know?

Objective

There is no evidence-based description of electrocardiogram (ECG) interpretation competencies for emergency medicine (EM) trainees. The first step in defining these competencies is to develop a prioritized list of adult ECG findings relevant to EM contexts. The purpose of this study was to categorize the importance of various adult ECG diagnoses and/or findings for the EM trainee.

Methods

We developed a list of potentially important adult ECG diagnoses/findings and conducted a Delphi opinion-soliciting process. Participants used a 4-point Likert scale to rate the importance of each diagnosis for EM trainees. Consensus was defined as a minimum of 75% agreement at the second round or later. In the absence of consensus, stability was defined as a shift of 20% or less after successive rounds.

Results

A purposive sampling of 22 emergency physicians participated in the Delphi process, and 16 (72%) completed the process. Of those, 15 were from 11 different EM training programs across Canada and one was an expert in EM electrocardiography. Overall, 78 diagnoses reached consensus, 42 achieved stability and one diagnosis achieved neither consensus nor stability. Out of 121 potentially important adult ECG diagnoses, 53 (44%) were considered “must know” diagnoses, 61 (50%) “should know” diagnoses, and 7 (6%) “nice to know” diagnoses.

Conclusion

We have categorized adult ECG diagnoses within an EM training context, knowledge of which may allow clinical EM teachers to establish educational priorities. This categorization will also facilitate the development of an educational framework to establish EM trainee competency in ECG interpretation.

An approach toward the development of core syllabuses for the anatomical sciences

There is increasingly a call for clinical relevance in the teaching of the biomedical sciences within all health care programs. This presupposes that there is an understanding of what is "core" material within the curriculum. To date, the anatomical sciences have been poorly served by the development of core syllabuses, although there have been commendable attempts to define a core syllabus for gross anatomy in medicine and for some medical specialties. The International Federation of Associations of Anatomists and the European Federation for Experimental Morphology aim to formulate, on an international basis, core syllabuses for all branches of the anatomical sciences. This is being undertaken at the initial stage using Delphi Panels consisting of a team of anatomists, scientists, and clinicians who evaluate syllabus content and accord each element/topic "essential," "important," "acceptable," or "not required" status. Their initial conjectures, published on the International Federation of Associations of Anatomists' website, provide merely a framework to enable anatomical (and other cognate learned) societies and individual anatomists, clinicians, and students to comment upon the syllabuses. This article presents the concepts and methodological approaches underlying the hybrid Delphi process employed. Preliminary findings relating to the development of a neuroanatomy core syllabus are provided to illustrate the methods initially employed by a Delphi Panel. The approach is novel in that it is international in scope, is conceptually democratic, and is developmentally fluid in terms of availability for amendment. The aim is to set internationally recognized standards and thus to provide guidelines concerning anatomical knowledge when engaged in course development.

Determination of clinically relevant content for a musculoskeletal anatomy curriculum for physical medicine and rehabilitation residents

To address the need for more clinical anatomy training in residency education, many postgraduate programs have implemented structured anatomy courses into their curriculum. Consensus often does not exist on specific content and level of detail of the content that should be included in such curricula. This article describes the use of the Delphi method to identify clinically relevant content to incorporate in a musculoskeletal anatomy curriculum for Physical Medicine and Rehabilitation (PM&R) residents. A two round modified Delphi involving PM&R experts was used to establish the curricular content. The anatomical structures and clinical conditions presented to the expert group were compiled using multiple sources: clinical musculoskeletal anatomy cases from the PM&R residency program at the University of Toronto; consultation with PM&R experts; and textbooks. In each round, experts rated the importance of each curricular item to PM&R residency education using a five-point Likert scale. Internal consistency (Cronbach's alpha) was used to determine consensus at the end of each round and agreement scores were used as an outcome measure to determine the content to include in the curriculum. The overall internal consistency in both rounds was 0.99. A total of 37 physiatrists from across Canada participated and the overall response rate over two rounds was 97%. The initial curricular list consisted of 361 items. After the second iteration, the list was reduced by 44%. By using a national consensus method we were able to objectively determine the relevant anatomical structures and clinical musculoskeletal conditions important in daily PM&R practice.

Guiding principles for student leadership development in the doctor of pharmacy program to assist administrators and faculty members in implementing or refining curricula

OBJECTIVE

To assist administrators and faculty members in colleges and schools of pharmacy by gathering expert opinion to frame, direct, and support investments in student leadership development.

METHODS

Twenty-six leadership instructors participated in a 3-round, online, modified Delphi process to define doctor of pharmacy (PharmD) student leadership instruction. Round 1 asked open-ended questions about leadership knowledge, skills, and attitudes to begin the generation of student leadership development guiding principles and competencies. Statements were identified as guiding principles when they were perceived as foundational to the instructional approach. Round 2 grouped responses for agreement rating and comment. Group consensus with a statement as a guiding principle was set prospectively at 80%. Round 3 allowed rating and comment on guidelines, modified from feedback in round 2, that did not meet consensus. The principles were verified by identifying common contemporary leadership development approaches in the literature.

RESULTS

Twelve guiding principles, related to concepts of leadership and educational philosophy, were defined and could be linked to contemporary leadership development thought. These guiding principles describe the motivation for teaching leadership, the fundamental precepts of student leadership development, and the core tenets for leadership instruction.

CONCLUSIONS

Expert opinion gathered using a Delphi process resulted in guiding principles that help to address many of the fundamental questions that arise when implementing or refining leadership curricula. The principles identified are supported by common contemporary leadership development thought.

Competencies for student leadership development in doctor of pharmacy curricula to assist curriculum committees and leadership instructors

OBJECTIVE

To assist curriculum committees and leadership instructors by gathering expert opinion to define student leadership development competencies for pharmacy curricula.

METHODS

Twenty-six leadership instructors participated in a 3-round, online, modified Delphi process to define competencies for student leadership development in pharmacy curricula. Round 1 asked open-ended questions about leadership knowledge, skills, and attitudes. Round 2 grouped responses for agreement rating and comment. Round 3 allowed rating and comment on competencies not yet meeting consensus, which was prospectively set at 80%.

RESULTS

Eleven competencies attained 80% consensus or higher and were grouped into 3 areas: leadership knowledge, personal leadership commitment, and leadership skill development. Connections to contemporary leadership development literature were outlined for each competency as a means of verifying the panel's work.

CONCLUSIONS

The leadership competencies will aid students in addressing: What is leadership? Who am I as a leader? What skills and abilities do I need to be effective? The competencies will help curriculum committees and leadership instructors to focus leadership development opportunities, identify learning assessments, and define program evaluation.

Early identification of children at risk for critical care: standardizing communication for inter-emergency department transfers

BACKGROUND

Interfacility transfers occur frequently and often involve critically ill patients. Clear communication at the time of patient referral is essential for patient safety.

OBJECTIVES

The objective of this work was to study whether a standardized inter-emergency department (ED) transfer communication template for pediatric patients with respiratory complaints identifies patients that require intensive care unit (ICU) admission.

METHODS

We created a template to structure the communication between referring and receiving providers involved in inter-ED transfers of children with respiratory complaints. The template was designed for use by nonphysicians to prompt specific questions that would trigger notification of the ED attending based on signs of critical illness. The template was retrospectively applied to determine whether it would have properly triggered attending physician notification of a child ultimately requiring ICU admission.

RESULTS

Of 285 transferred children, 61 (21%) were admitted to an ICU from the receiving ED. The sensitivity of the communication template in predicting the need for ICU admission was 84% (95% confidence interval [CI], 72%-92%), negative predictive value of 95% (95% CI, 90%-97%), specificity of 77% (95% CI, 71%-82%), positive predictive value of 50% (95% CI, 40%-60%). Of the 10 patients admitted to an ICU who were not identified by the tool, none were critically ill upon arrival. Of the individual communication elements, the sensitivity and negative predictive value ranged from 3% to 38% and from 79% to 86%, respectively.

CONCLUSIONS

A standardized communication template for inter-ED transfers can identify children with respiratory complaints who require ICU admission. Next steps include real-time application to judge screening performance compared with current nonstandardized intake protocols.

A consensus definition and core competencies for being an advocate for pharmacy

OBJECTIVE

To develop a consensus definition for "advocacy for the profession of pharmacy" and core competencies for doctor of pharmacy (PharmD) graduates to be effective advocates for the profession.

METHODS

A 3-round modified Delphi process was conducted using a panel of 9 experts. Participants revised a definition for "advocacy for the profession" and ultimately rated their agreement using a 5-point Likert scale. Competency statements were developed and subsequently rated for importance for being an advocate and importance to address in PharmD curricula.

RESULTS

A consensus-derived definition was developed. Two competency statements achieved consensus for both measures of importance. Four competency statements achieved consensus for only 1 measure and another 4 did not reach consensus for either measure.

CONCLUSION

A consensus-derived definition was developed describing advocacy for the profession of pharmacy and began laying the groundwork for the knowledge and skills necessary to be an effective advocate for the profession of pharmacy.

Competencies of specialised wound care nurses: a European Delphi study

Health care professionals responsible for patients with complex wounds need a particular level of expertise and education to ensure optimum wound care. However, uniform education for those working as wound care nurses is lacking. We aimed to reach consensus among experts from six European countries as to the competencies for specialised wound care nurses that meet international professional expectations and educational systems. Wound care experts including doctors, wound care nurses, lecturers, managers and head nurses were invited to contribute to an e-Delphi study. They completed online questionnaires based on the Canadian Medical Education Directives for Specialists framework. Suggested competencies were rated on a 9-point Likert scale. Consensus was defined as an agreement of at least 75% for each competence. Response rates ranged from 62% (round 1) to 86% (rounds 2 and 3). The experts reached consensus on 77 (80%) competences. Most competencies chosen belonged to the domain 'scholar' (n = 19), whereas few addressed those associated with being a 'health advocate' (n = 7). Competencies related to professional knowledge and expertise, ethical integrity and patient commitment were considered most important. This consensus on core competencies for specialised wound care nurses may help achieve a more uniform definition and education for specialised wound care nurses.

Delphi approach to select rare diseases for a European representative survey. The BURQOL-RD study

OBJECTIVES

The BURQOL-RD project is intended to develop a disease based model capable of quantifying the socio-economic burden and health-related quality of life for patients with rare diseases (RDs) and their caregivers in Europe. We described the methodology used to select a set of 10 RDs to be approached in a pilot study.

METHODS

BURQOL-RD project includes 23 partners from 8 European countries: Spain, UK, France, Germany, Sweden, Italy, Hungary and Bulgaria. A two-round Delphi panels in combination with Carroll diagram was used to generate consensus in the selection of the 10 RDs among the project participants.

RESULTS

The two Delphi rounds yielded a prioritised list, to which the Carroll diagram was applied, taking into account three determinants: prevalence, availability of effective treatment and need for carer. The final set of RD to be studied was obtained: cystic fibrosis, Prader-Willi syndrome, haemophilia, duchenne muscular dystrophy, epidermolysis bullosa, fragile X syndrome, scleroderma, mucopolysaccharidosis, juvenile idiopathic arthritis and histiocytosis.

CONCLUSIONS

This methodology permitted the generation of an equilibrated set of RDs for the pilot study of BURQOL-RD project. The model will be suitable for application in a wide range of RDs.