Assessment of Competence in EVAR Procedures: A Novel Rating Scale Developed by the Delphi Technique

Roles, Responsibilities, and Expectations of Medical Students on Surgical Subinternships: A Modified Delphi Consensus Study

PURPOSE

Surgical subinternships are important rotations for students preparing for a career in general surgery; however, these rotations often vary by institution and service. This modified Delphi study was conducted to reach a consensus set of roles, responsibilities, and expectations of fourth-year medical students on their surgical subinternships.

METHOD

Candidate statements on roles, responsibilities, and expectations of subinterns were categorized into 7 domains: rotation structure, rounding and patient care, operating room conduct, technical skills, knowledge base, clinic, and professionalism. Expert panels were assembled of key stakeholders: program directors, clerkship directors, other education faculty, trainees, and recent subinterns. Three Delphi rounds were conducted from January to April 2023 to reach consensus defined a priori as a Cronbach α ≥ 0.8 and 80% or greater panel agreement.

RESULTS

Forty-six expert panelists were recruited to participate in Delphi rounds, with 100%, 95.7%, and 97.8% response rates in the first, second, and third rounds, respectively. By the third round, 67 statements reached consensus as essential roles, responsibilities, and expectations of surgical subinterns. Key themes from these 67 statements included subinterns approximating the role of an intern with respect to work hours, patient care responsibilities, basic technical skills, and knowledge base. Panelists rated rounding and patient care as the most important domain, followed closely by professionalism. Additional key domains for evaluation in descending order were knowledge base, operating room conduct, clinic, and technical skills. By the third round, notable disagreements in the Delphi process included technical skills and rounding and patient care (93.3% and 88.9% agreement, respectively).

CONCLUSIONS

This study provides a national consensus on core roles, responsibilities, and expectations for medical students completing surgical subinternships. Students can use these recommendations to prepare for subinternships, whereas faculty as well as residents and fellows can use them to evaluate applicants for general surgery residency positions.

Development of the Technical Assistance Engagement Scale: a modified Delphi study

BACKGROUND

Technical assistance (TA) is a tailored approach to capacity building that is commonly used to support implementation of evidence-based interventions. Despite its widespread applications, measurement tools for assessing critical components of TA are scant. In particular, the field lacks an expert-informed measure for examining relationship quality between TA providers and recipients. TA relationships are central to TA and significantly associated with program implementation outcomes. The current study seeks to address the gap in TA measurement tools by providing a scale for assessing TA relationships.

METHODS

We utilized a modified Delphi approach involving two rounds of Delphi surveys and a panel discussion with TA experts to garner feedback and consensus on the domains and items that compose the TA Engagement Scale.

RESULTS

TA experts represented various U.S. organizations and TA roles (e.g., provider, recipient, researcher) with 25 respondents in the first survey and 26 respondents in the second survey. The modified Delphi process resulted in a scale composed of six domains and 22 items relevant and important to TA relationships between providers and recipients.

CONCLUSION

The TA Engagement Scale is a formative evaluation tool intended to offer TA providers the ability to identify strengths and areas for growth in the provider-recipient relationship and to communicate about ongoing needs. As a standard measurement tool, it lends a step toward more systematic collection of TA data, the ability to generate a more coherent body of TA evidence, and enables comparisons of TA relationships across settings.

Construction of PARI public health education programs for Chinese undergraduates: a Delphi study

Objectives

The objective of this study is to develop a consensus among experts on a comprehensive and scientifically sound physical activity-related injuries (PARI) public health education program specifically tailored for undergraduates.

Methods

This study designed three rounds of expert consultation by using a Delphi method. A panel of 30 experts from the fields of public health education, sports medicine, anesthesia pain, emergency medicine, and emergency nursing participated in the study.

Results

This study successfully established a consensus among experts on the goals, content, teaching methods, and time allocation for the PARI Public Health Education Program for undergraduates. The program encompasses 10 objectives divided into 2 main categories: professional knowledge and skill goals. In terms of content, it includes 5 primary indicators, 22 secondary indicators, and 56 detailed tertiary indicators. Six teaching methods were identified as suitable. Additionally, a typical 60-min educational session was segmented into eight parts, with a proposed time arrangement for each, ensuring comprehensive coverage of all topics.

Conclusion

The consensus achieved in this study on the PARI Public Health Education Program for undergraduates lays a crucial foundation for the advancement of health literacy and proactive health management within this demographic. We presented a comprehensive framework for PARI public health education, integrating diverse learning methods and content areas. This systematic approach not only enriched the resources available for undergraduate health education, especially of PARI but also had the potential to significantly impact the implementation and effectiveness of health promotion strategies.

Certification of Basic Skills in Endovascular Aortic Repair Through a Modular Simulation Course With Real Time Performance Assessment

OBJECTIVE

Endovascular aortic repair (EVAR) is being used increasingly for the treatment of infrarenal abdominal aortic aneurysms. Improvement in educational strategies is required to teach future vascular surgeons EVAR skills, but a comprehensive, pre-defined e-learning and simulation curriculum remains to be developed and tested. EndoVascular Aortic Repair Assessment of Technical Expertise (EVARATE), an assessment tool for simulation based education (SBE) in EVAR, has previously been designed to assess EVAR skills, and a pass limit defining mastery level has been set. However, EVARATE was developed for anonymous video ratings in a research setting, and its feasibility for real time ratings in a standardised SBE programme in EVAR is unproven. This study aimed to test the effect of a newly developed simulation based modular course in EVAR. In addition, the applicability of EVARATE for real time performance assessments was investigated.

METHODS

The European Society of Vascular Surgery (ESVS) and Copenhagen Certification Programme in EVAR (ENHANCE-EVAR) was tested in a prospective cohort study. ENHANCE-EVAR is a modular SBE programme in EVAR consisting of e-learning and hands-on SBE. Participants were rated with the EVARATE tool by experienced EVAR surgeons.

RESULTS

Twenty-four physicians completed the study. The mean improvement in EVARATE score during the course was +11.8 (95% confidence interval 9.8 - 13.7) points (p < .001). Twenty-two participants (92%) passed with a mean number of 2.8 ± 0.7 test attempts to reach the pass limit. Cronbach's alpha coefficient was 0.91, corresponding to excellent reliability of the EVARATE scale. Differences between instructors' EVARATE ratings were insignificant (p = .16), with a maximum variation between instructors of ± 1.3 points.

CONCLUSION

ENHANCE-EVAR, a comprehensive certifying EVAR course, was proven to be effective. EndoVascular Aortic Repair Assessment of Technical Expertise (EVARATE) is a trustworthy tool for assessing performance within an authentic educational setting, enabling real time feedback.

Developing Gender-Affirming Surgery Curricula for Plastic Surgery Residency and Fellowship via Delphi Consensus

BACKGROUND

Gender-affirming surgery (GAS) is a rapidly growing field within plastic surgery, and residents and fellows must receive appropriate training. However, there are no standardized surgical training curricula. The objective of this study was to identify core curricula within the field of GAS.

METHODS

Four GAS surgeons from different academic institutions identified initial curricular statements within six categories: (1) comprehensive GAS care, (2) gender-affirming facial surgery, (3) masculinizing chest surgery, (4) feminizing breast augmentation, (5) masculinizing genital GAS, and (6) feminizing genital GAS. Expert panelists consisting of plastic surgery residency program directors and GAS surgeons were recruited for three rounds of the Delphi-consensus process. The panelists decided whether each curriculum statement was appropriate for residency, fellowship, or neither. A statement was included in the final curriculum when Cronbach α value was greater than or equal to 0.8, meaning that 80% or more of the panel agreed on inclusion.

RESULTS

A total of 34 panelists (14 plastic surgery residency program directors and 20 GAS surgeons representing 28 US institutions) participated. The response rate was 85% for the first round, 94% for the second, and 100% for the third. Out of 124 initial curriculum statements, 84 reached consensus for the final GAS curricula, 51 for residency, and 31 for fellowship.

CONCLUSIONS

A national consensus on core GAS curriculum for plastic surgery residency and GAS fellowship was achieved by a modified Delphi method. Implementation of this curriculum will ensure that trainees in plastic surgery are adequately prepared in the field of GAS.

Development of a Rubric to Evaluate Implementation Quality of Simulation-Based Courses: A Consensus Study

INTRODUCTION

Simulation-based education is a recognized way of developing medical competencies, and there is overwhelming scientific evidence to support its efficacy. However, it is still underused, which can often be related to poor implementation process. In addition, best practices for implementation of simulation-based courses based on implementation science are not widely known nor applied. The purpose of this study was to develop a rubric, the Implementation Quality Rubric for Simulation (IQR-SIM), to evaluate the implementation quality of simulation-based courses.

METHODS

A 3-round, modified Delphi process involving international simulation and implementation experts was initiated to gather and converge opinions regarding criteria for evaluating the implementation quality of simulation-based courses. Candidate items for Round 1 were developed based on the Adapted Implementation Model for Simulation. Items were revised and expanded to include descriptive anchors for evaluation in Round 2. Criterion for inclusion was 70% of respondents selecting an importance rating of 4 or 5/5. Round 3 provided refinement and final approval of items and anchors.

RESULTS

Thirty-three experts from 9 countries participated. The initial rubric of 32 items was reduced to 18 items after 3 Delphi rounds, resulting in the IQR-SIM: a 3-point rating scale, with nonscored options "Don't know/can't assess" and "Not applicable," and a comments section.

CONCLUSIONS

The IQR-SIM is an operational tool that can be used to evaluate the implementation quality of simulation-based courses and aid in the implementation process to identify gaps, monitor the process, and promote the achievement of desired implementation and learning outcomes.

Facilitators and Barriers to Implementation of Simulation Based Education in Vascular Surgery in Europe

OBJECTIVE

This study explored the status and availability of simulation based education (SBE) for learning vascular surgical procedures identified in the 2019 General Needs Assessment in vascular surgery in Europe (GNA-2019) and identified facilitators and barriers to SBE implementation in vascular surgery.

METHODS

A three round iterative survey was distributed via the European Society for Vascular Surgery and the Union Européenne des Médecins Spécialistes. Members from leading committees and organisations within the European vascular surgical community were invited to participate as key opinion leaders (KOLs). Three online survey rounds explored demographics, SBE availability, and facilitators and barriers to SBE implementation.

RESULTS

Overall, 147 KOLs (target population 338) accepted invitation to round 1, representing 30 European countries. The dropout rates for rounds 2 and 3 were 29% and 40%, respectively. Most respondents (88%) were senior, consultant level or higher. No mandatory SBE training was required in their department before training on patients, according to 84% of the KOLs. There was high consensus on the need for structured SBE (87%) and mandatory SBE (81%). SBE is available for the top three prioritised procedures in GNA-2019 (basic open skills, basic endovascular skills, and vascular imaging interpretation) in 24, 23, and 20 of the 30 represented European countries, respectively. The highest ranking facilitators were structured SBE programmes, availability of simulation equipment locally and regionally, good quality simulators, and having a dedicated person running the SBE. The highest ranked barriers were lack of structured SBE curriculum, equipment costs, lack of SBE culture, no or limited dedicated time for faculty to teach in SBE, and clinical work overload.

CONCLUSION

Based largely on the opinions of KOLs in vascular surgery in Europe, this study revealed that SBE is needed in vascular surgery training and that systematic and structured programmes are required to ensure successful implementation.

Assessment of Technical Competence in Distal Radius Fracture Fixation by a Volar Locking Plate: A Global Delphi Consensus Study

PURPOSE

Volar locking plate fixation of distal radius fractures is a common orthopedic procedure and should be mastered by graduating orthopedic residents. Surgical education is transitioning from a traditional time-based approach to competency-based medical education. Valid and objective assessment is essential for successful transition. The purpose of this study was to develop a comprehensive, procedure-specific assessment tool to evaluate technical competence in volar locking plate osteosynthesis of a distal radius fracture.

METHODS

International orthopedic/trauma experts involved in resident education participated as panelists in a four-round online Delphi process to reach consensus on the content of the assessment tool. Round 1 was an item-generating round, in which the panelists identified potential assessment parameters. In round 2, the panelists rated the importance of each suggested assessment parameter and reached consensus on which to include in the assessment tool. Round 3 yielded specific assessment score intervals for specific bone and fracture models and is not reported in this study. In round 4, the panelists assigned weights to the assessment parameters on a 1-10 scale to determine how each parameter should have an impact on the overall results.

RESULTS

Eighty-seven surgeons, representing 42 countries, participated in the study. Round 1 resulted in 45 assessment parameters, grouped into five procedural steps. After round 2, the number of parameters was reduced to 39. After the final round, an additional parameter was removed and weights were assigned to the remaining parameters.

CONCLUSIONS

Using a systematic methodology, a preliminary assessment tool to evaluate technical competence in distal radius fracture fixation was developed. A consensus of international experts supports the content validity of the assessment tool.

CLINICAL RELEVANCE

This assessment tool represents the first step in the evidence-based assessment essential for competency-based medical education. Before implementation, further studies exploring validity of variations of the assessment tool in different educational contexts are required.

Assessing endovascular team performances in a hybrid room using the Black Box system: a prospective cohort study

BACKGROUND

The hybrid room (HR) is a complex, high-risk environment, requiring teams (surgeons, anesthesiologists, nurses, technologists) to master various skills, including the 'As Low As Reasonably Achievable' principle of radiation safety. This prospective single center cohort reports the first use of the Operating Room Black Box (ORBB) in a HR. This medical data recording system captures procedural and audio-visual data to facilitate structured team performance analysis.

METHODS

Patients planned for endovascular repair of an infrarenal abdominal aortic aneurysm (EVAR) or treatment of symptomatic iliac-femoral-popliteal atherosclerotic disease (Peripheral Vascular Interventions or PVI) were included. Validated measures and established assessment tools were used to assess (non-)technical skills, radiation safety performance and environmental distractions.

RESULTS

Six EVAR and sixteen PVI procedures were captured. Technical performance for one EVAR was rated 19/35 on the procedure-specific scale, below the 'acceptable' score of 21. Technical skills were rated above acceptable in all PVI procedures. Shared decision making and leadership were rated highly in 12/22 cases, whereas surgical communication and nurses' task management were rated low in 14/22 cases. Team members rarely stepped back from the C-arm during digital subtraction angiography. Radiation safety behavior was scored below 'acceptable' in 14/22 cases. A median (interquartile range) number of 12 (6-23) auditory distractions was observed per procedure.

CONCLUSIONS

The ORBB facilitates holistic workplace-based assessment of endovascular performance in a HR by combining objective assessment parameters and rating scale-based evaluations. Strengths and weaknesses were identified in team members' (non-)technical and radiation safety practices. This technology has the potential to improve vascular surgical practice, though human input remains crucial. (NCT04854278).

Development of Content for a Virtual Reality Simulation to Understand and Mitigate Moral Distress in Healthcare Workers

Background In high-stakes situations, healthcare workers are prone to suffer moral injury, the psychological, social, and spiritual impact of events involving betrayal or transgression of one's own deeply held moral beliefs and values. As a result, this may negatively impact their capacity to provide adequate levels of care to patients. There is a lack of educational resources catered to help healthcare workers navigate ethical situations in clinical settings that may lead to or worsen moral distress. The aim of this report is to describe the methodology of development and resulting outcomes in the form of an educational resource that includes a virtual reality (VR) simulation to help healthcare workers understand and mitigate moral distress as a result of internal and external constraints at their workplaces. Methodology A study using a method outlining a set of constraint parameters, followed by ideation utilizing design thinking (DT), and concluding with a consensus-building exercise using Delphi methodology (DM) with a group of 13 experts in healthcare simulation, VR, psychiatry, psychology, and nursing. The constraints parameters included technology use (VR), use of experiential learning theory, and duration of the intervention (15 minutes). A DT process was performed to generate and expand on ideas on the scenario and intervention of a possible VR simulation which were funneled into a three-round DM to define the foundations of the VR simulation. Average, standard deviations, and free-text comments in the DM were used to assess the inclusion of the produced requirements. Finally, a focus group interview was conducted with the same experts to draft the VR simulation. Results Within the specified constraints, the DT process produced 33 ideas for the VR simulation scenario and intervention that served as a starting point to short-list the requirements in Round 1. In Rounds 1 to 2, 25 items were removed, needed revising, and/or were retained for the subsequent rounds, which resulted in eight items at the end of Round 2. Round 2 also required specialists to provide descriptions of potential scenarios and interventions, in which five were submitted. In Round 3, experts rated the descriptions as somewhat candidate to use in the final VR simulation, and the open feedback in this round proposed combining the elements from each of the descriptions. Using this data, a prototype of the VR simulation was developed by the project team together with VR designers. Conclusions This development demonstrated the feasibility of using the constraints-ideation-consensus approach to define the content of a possible VR simulation to serve as an educational resource for healthcare workers on how to understand and mitigate moral distress in the workplace. The methodology described in this development may be applied to the design of simulation training for other skills, thereby advancing healthcare training and the quality of care delivered to the greater society.

Resident Readiness for Senior Level Decision Making: Identifying the Domains for Formative Assessment and Feedback

OBJECTIVE

To establish expert consensus regarding the domains and topics for senior surgery residents (PGY-4) to make critical decisions and assume senior-level responsibilities, and to develop the formative American College of Surgeons Senior Resident Readiness Assessment (ACS SRRA) Program.

DESIGN

The American College of Surgeons (ACS) education leadership team conducted a focus group with surgical experts to identify the content for an assessment tool to evaluate senior residents' readiness for their increased levels of responsibility. After the focus group, national experts were recruited to develop consensus on the topics through three rounds of surveys using Delphi methodology. The Delphi participants rated topics using Likert-type scales and their comments were incorporated into subsequent rounds. Consensus was defined as ≥ 80% agreement with internal-consistency reliability (Cronbach's alpha) ≥ 0.8. In a stepwise fashion, topics that did not achieve consensus for inclusion were removed from subsequent survey rounds.

SETTING

The surveys were administered via an online questionnaire.

PARTICIPANTS

Twelve program directors and assistant program directors made up the focus group. The 39 Delphi participants represented seven different surgical subspecialties and were from diverse practice settings. The median length of experience in general surgery resident education was 20 years (IQR 14.3-30.0) with 64% of the experts being either current or past general surgery residency program directors.

RESULTS

The response rate was 100% and Cronbach's alpha was ≥ 0.9 for each round. The Delphi participants contributed a large number of comments. Of the 201 topics that were evaluated initially, 120 topics in 25 core clinical areas were included to create the final domains of ACS SRRA.

CONCLUSIONS

National consensus on the domain of the ACS SRRA has been achieved via the modified Delphi method among expert surgeon educators. ACS SRRA will identify clinical topics and areas in which each senior resident needs improvement and provide data to residents and residency programs to develop individualized learning plans. This would help in preparing the senior residents to assume their responsibilities and support their readiness for future fellowship training or surgical practice.

Construction of life-and-death education contents for the elderly: a Delphi study

BACKGROUND

Life-and-death education is intimately related to palliative-and-hospice care. It should be implemented among groups of all ages, especially for the elderly. This study aims to establish expert consensus on a set of scientific and systematic life-and-death education contents for the elderly and provide reference for the practice on the elderly.

METHODS

This study designed three rounds of expert consultation by using a Delphi method. A panel of 22 experts from the fields of palliative-and-hospice care, life-and-death education, geriatric nursing, humanities and ethics, and geriatric psychology participated in the study.

RESULTS

This study finally reached expert consensus on the contents of life-and-death education for the elderly, containing 4 first-level items, Life-and-death literacy promotion in the elderly; Life-and-death concept establishment of the elderly; Life-and-death planning of the elderly; Life-and-death thoughts of the elderly with affiliated 9 second-level items, and corresponding 23 detailed third-level items.

CONCLUSIONS

The life-and-death education contents for the elderly offer a basis for publicity for health professionals, promote dialogues on death, preparation, and planning for death and dying. The life-and-death education contents system was clear in coherence containing definite and comprehensive contents, which enriched life-and-death education resources globally. The results could assist in the planning of palliative-and-hospice care services to improve quality of death of the elderly.

Reliable and valid assessment of procedural skills in resuscitative endovascular balloon occlusion of the aorta

BACKGROUND

Valid and reliable assessment of skills is essential for improved and evidence-based training concepts. In a recent study, we presented a novel tool to assess procedural skills in resuscitative endovascular balloon occlusion of the aorta (REBOA), REBOA-RATE, based on international expert consensus. Although expert consensus is a strong foundation, the performance of REBOA-RATE has not been explored. The study aimed to examine the reliability and validity of REBOA-RATE.

METHODS

This was an experimental simulation-based study. We enrolled doctors with three levels of expertise to perform two REBOA procedures in a simulated scenario of out-of-hospital cardiac arrest. Procedures were video-recorded, and videos were blinded and randomized. Three clinical experts independently rated all procedures using REBOA-RATE. Data were analyzed using Messick's framework for validity evidence, including generalizability analysis of reliability and determination of a pass/fail standard.

RESULTS

Forty-two doctors were enrolled: 16 novices, 13 anesthesiologists, and 13 endovascular experts. They all performed two procedures, yielding 84 procedures and 252 ratings. The REBOA-RATE assessment tool showed high internal consistency (Cronbach's α = 0.95) and excellent interrater reliability (intraclass correlation coefficient, 0.97). Assessment using one rater and three procedures could ensure overall reliability suitable for high-stakes testing (G-coefficient >0.80). Mean scores (SD) for the three groups in the second procedure were as follows: novices, 32% (24%); anesthesiologists, 55% (29%); endovascular experts, 93% (4%) (p < 0.001). The pass/fail standard was set at 81%, which all experts but no novices passed.

CONCLUSION

Data strongly support the reliability and validity of REBOA-RATE, which successfully discriminated between all experience levels. The REBOA-RATE assessment tool requires minimal instruction, and one rater is sufficient for reliable assessment. Together, these are strong arguments for the use of REBOA-RATE to assess REBOA skills, allowing for competency-based training and certification concepts.

LEVEL OF EVIDENCE

Diagnostic test, no or poor gold standard, level V.

Assessment of competence in local anaesthetic thoracoscopy: development and validity investigation of a new assessment tool

Background

The aims of the study were to develop an assessment tool in local anaesthetic thoracoscopy (LAT), investigate validity evidence, and establish a pass/fail standard.

Methods

Validity evidence for the assessment tool was gathered using the unified Messick framework. The tool was developed by five experts in respiratory medicine and medical education. Doctors with varying experience performed two consecutive procedures in a standardized, simulation-based setting using a newly developed thorax/lung silicone model. Performances were video-recorded and assessed by four expert raters using the new tool. Contrasting groups’ method was used to set a pass/fail standard.

Results

Nine novices and 8 experienced participants were included, generating 34 recorded performances and 136 expert assessments. The tool had a high internal consistency (Cronbach’s alpha =0.94) and high inter-rater reliability (Cronbach’s alpha =0.91). The total item score significantly correlated with the global score (r_s=0.86, P<0.001). Participants’ first performance correlated to second performance (test-retest reliability) with a Pearson’s r of 0.93, P<0.001. Generalisability (G) study showed a G-coefficient of 0.92 and decision (D) study estimated that one performance assessed by two raters or four performances assessed by one rater are needed to reach an acceptable reliability, i.e., G-coefficient >0.80. The tool was able to discriminate between the two groups in both performances: experienced mean score =30.8±4.2; novice mean score =15.8±2.3, P<0.001. Pass/fail standard was set at 22 points.

Conclusions

The newly developed assessment tool showed solid evidence of validity and can be used to ensure competence in LAT.

Developing a tool to assess competence in resuscitative endovascular balloon occlusion of the aorta: An international Delphi consensus study

BACKGROUND

Resuscitative endovascular balloon occlusion of the aorta (REBOA) is an emergency procedure that is potentially lifesaving in major noncompressible torso hemorrhage. It may also improve outcome in nontraumatic cardiac arrest. However, the procedure can be technically challenging and requires the immediate presence of a qualified operator. Thus, evidence-based training and assessment of operator skills are essential for successful implementation and patient safety. A prerequisite for this is a valid and reliable assessment tool specific for the procedure. The aim of this study was to develop a tool for assessing procedural competence in REBOA based on best-available knowledge from international experts in the field.

METHODS

We invited international REBOA experts from multiple specialties to participate in an anonymous three-round iterative Delphi study to reach consensus on the design and content of an assessment tool. In round 1, participants suggested items to be included. In rounds 2 and 3, the relevance of each suggested item was evaluated by all participants to reach consensus. Interround data processing was done systematically by a steering group.

RESULTS

Forty panelists representing both clinical and educational expertise in REBOA from 16 countries (in Europe, Asia, and North and South America) and seven different specialties participated in the study. After 3 Delphi rounds and 532 initial item suggestions, the panelists reached consensus on a 10-item assessment tool with behaviorally anchored rating scales. It includes assessment of teamwork, procedure time, selection and preparation of equipment, puncture technique, guidewire handling, sheath handling, placement of REBOA catheter, occlusion, and evaluation.

CONCLUSION

We present the REBOA-RATE assessment tool developed systematically by international experts in the field to optimize content validity. Following further studies of its validity and reliability, this tool represents an important next step in evidence-based training programs in REBOA, for example, using mastery learning.

LEVEL OF EVIDENCE

Therapeutic, level V.

Unifying the Hepatopancreatobiliary Surgery Fellowship Curriculum via Delphi Consensus

BACKGROUND

Hepatopancreatobiliary (HPB) Fellowship training in the Americas consists of 3 distinctive routes with variable curricula: Surgical Oncology Fellowship via the Society of Surgical Oncology (SSO), Abdominal Transplant Surgery Fellowship via the American Society of Transplant Surgeons (ASTS), and HPB Fellowship via the Americas Hepato-Pancreato-Biliary Association (AHPBA). Our objective was to establish a pan-American consensus among HPB surgeons, surgical oncologists, abdominal transplant surgeons, and general surgery residency program directors (GSPDs) on a core knowledge curriculum for HPB fellowship, and to identify topics appropriate for general surgery residency and subspecialty beyond HPB fellowship.

STUDY DESIGN

A 3-round modified Delphi process was used. Baseline statements were developed by the Education and Training Committee of the AHPBA, in collaboration with representatives of the SSO, ASTS, and GSPDs. The expert panel, consisting of members of the 3 societies together with GSPDs, rated the statements on a 5-point Likert scale and suggested editing or adding new statements. A statement was included in the final curriculum when Cronbach's alpha value was ≥ 0.8 and ≥ 80% of the panel agreed on inclusion.

RESULTS

The response rate was 100% for the first round, and 98% for the second and third rounds. Eighty-nine of 138 proposed statements were included in the final HPB fellowship curriculum. Curricula for general surgery residency and subspecialty beyond HPB fellowship included 50 and 29 statements, respectively.

CONCLUSIONS

A multinational consensus on core knowledge for an HPB fellowship curriculum was achieved via the modified Delphi method. This core curriculum may be used to standardize HPB fellowship training across different pathways in the Americas.

Evaluation of tools to assess operative competence in endovascular procedures: a systematic review

BACKGROUND

With an increase in the use of endovascular interventions as an alternative to open surgery and the unique technical skills required, current methods for assessing the competence of vascular surgery trainees may not be optimal, suggesting a need for a shift in assessment modalities. We conducted this systematic review to explore current assessment methods used in vascular surgery training to assess competence specific to endovascular procedures.

METHODS

A comprehensive literature search was performed with a structured search strategy using terms focusing on endovascular procedures and assessment. Inclusion and exclusion criteria were used in order to screen for suitable articles.

RESULTS

We identified 54 articles that satisfied the inclusion criteria. These included a single randomized controlled trial, a single systematic review, a single narrative review and a single literature review, with the vast majority having level 2 evidence. Global rating scales, proficiency assessments and written/oral examinations were described as standard current assessment tools. These modalities lack reproducibility and objectivity, neglecting the needs of assessment of endovascular procedures requiring specialized decision making and finger dexterity. Novel methods such as high fidelity simulation and virtual reality promote reproducible and objective assessment methods in the context of endovascular surgery, and have a promising future.

CONCLUSION

While current assessment methods in vascular surgery are widely supported the changing skills required of a vascular surgery trainee warrants a shift in assessment modalities to better align to these requirements. High fidelity simulations show promise, although they require more extensive research to understand their relative merits.

Evaluation of competence in ultrasound-guided procedures-a generic assessment tool developed through the Delphi method

OBJECTIVES

To develop a generic and objective tool for assessing competence in percutaneous ultrasound-guided procedures.

METHODS

Interventional ultrasound experts from the Nordic countries were invited to participate in a three-round Delphi process. A steering committee was formed to manage the process. In round 1, the experts were asked to suggest all aspects to consider when assessing competence in US-guided procedures. Suggestions were analyzed and condensed into assessment items. In round 2, the expert panel rated these items on a 1-5 scale and suggested new items. Items with a mean rating of ≤ 3.5 were excluded. In round 3, the expert panel rated the list items and suggested changes to the items.

RESULTS

Twenty-five experts were invited, and response rates in the three rounds were 68% (17 out of 25), 100% (17 out of 17), and 100% (17 out of 17). The three-round Delphi process resulted in a 12-item assessment tool, using a five-point rating scale. The final assessment tool evaluates pre-procedural planning, US technique, procedural technique, patient safety, communication, and teamwork.

CONCLUSIONS

Expert consensus was achieved on a generic tool for assessment of competence in percutaneous ultrasound-guided procedures-the Interventional Ultrasound Skills Evaluation (IUSE). This is the initial step in ensuring a valid and reliable method for assessment of interventional US skill.

KEY POINTS

• Through a Delphi process, expert consensus was achieved on the content of an assessment tool for percutaneous ultrasound-guided procedures-the Interventional Ultrasound Skills Evaluation (IUSE) tool. • The IUSE tool is comprehensive and covers pre-procedural planning, US technique, procedural technique, patient safety, communication, and teamwork. • This is an important step in ensuring valid and reliable assessment of interventional US skills.

Key components of a hepatobiliary surgery curriculum for general surgery residents: results of the FULCRUM International Delphi consensus

BACKGROUND

In general surgery residency, hepatobiliary training varies significantly across the world. The aim of this study was to establish an international consensus among hepatobiliary surgeons on components of a hepatobiliary curriculum for general surgery residents.

METHODS

A three-round modified Delphi technique was employed. Fifty-two hepatobiliary surgeons involved in general surgery training programs were invited. An initial questionnaire was developed by a group of experts in hepatobiliary and educational research after a systematic literature review. It comprised 90 statements about knowledge, technical skills, attitudes, and postoperative care. Panelists could add or alter items. The survey was delivered electronically and the panel was instructed to score the items based on 5-point Likert scale. Consensus was reached when at least 80% of panelists agreed on a statement with Cronbach's alpha value >0.8.

RESULTS

Forty-one (79%) experts have participated. Sixteen panelists are based in Asia, 14 in Europe, and 11 in the Americas. Eighty percent of all proposed skills (81/101) were considered fundamental including knowledge (39/43), technical skills (16/32), attitude (15/15), and postoperative care (11/11).

CONCLUSION

An international consensus was achieved on components of a hepatobiliary curriculum. Acquiring broad knowledge is fundamental during residency. Advanced liver resection techniques require specialized hepatobiliary training.

Radiation Safety Performance is More than Simply Measuring Doses! Development of a Radiation Safety Rating Scale

PURPOSE

Radiation safety performance is often evaluated using dose parameters measured by personal dosimeters and/or the C-arm, which provide limited information about teams' actual radiation safety behaviors. This study aimed to develop a rating scale to evaluate team radiation safety behaviors more accurately and investigate its reliability.

MATERIALS AND METHODS

A modified Delphi consensus was organized involving European vascular surgeons (VS), interventional radiologists, and interventional cardiologists. Initial items and anchors were drafted a priori and rated using five-point Likert scales. Participants could suggest additional items or adjustments. Consensus was defined as ≥ 80% agreement (rating ≥ 4) with Cronbach's alpha ≥ .80. Two VS with expertise in radiation safety evaluated 15 video-recorded endovascular repairs of infrarenal aortic aneurysms (EVAR) to assess usability, inter and intrarater reliability.

RESULTS

Thirty-one of 46 invited specialists completed three rating rounds to generate the final rating scale. Five items underwent major adjustments. In the final round, consensus was achieved for all items (alpha = .804; agreement > 87%): 'Pre-procedural planning', 'Preparation in angiosuite/operating room', 'Shielding equipment', 'Personal protective equipment', 'Position of operator/team', 'Radiation usage awareness', 'C-arm handling', 'Adjusting image quality', 'Additional dose reducing functions', 'Communication/leadership', and 'Overall radiation performance and ALARA principle'. All EVARs were rated, yielding excellent Cronbach's alpha (.877) with acceptable interrater and excellent intrarater reliability (ICC = .782; ICC = .963, respectively).

CONCLUSION

A reliable framework was developed to assess radiation safety behaviors in endovascular practice and provide teams with formative feedback. The final scale is provided in this publication.

Learn EVAR sizing from scratch: The results of a one-day intensive course in EVAR sizing and stent graft selection for vascular trainees

Background and aim

Recognition of structured training in endovascular aortic repair (EVAR) for vascular trainees is increasing. Nevertheless, how trainees can achieve sufficient skills in EVAR sizing and graft selection is sparsely described. The aim of this study was to investigate the effect of systematic training in basic EVAR sizing and graft selection on vascular surgery trainees using a validated assessment tool.

Methods

Sixteen vascular surgery trainees were included in an intensive 6-h hands-on workshop in aortic sizing and stent graft selection for EVAR with a trainer-to-trainee ratio of 1:2. After 1-h lecture, participants did 5 h of supervised training on increasingly complex cases. Finally, the participants were tested using a validated assessment tool.

Results

All participants were able to size the test-case and select a stent graft combination in 24:35 (13:30–48:20) min (median and range). The participants’ overall test scores (lower is better) were in median 17.9 (11.9–28.4). This did not differ from the scores of experienced EVAR operators 14.7 (11.7–25.2) (<200 EVAR’s) ( p = .32) but was inferior to the score of EVAR experts 11.2 (9.8 –18.7) (≥200 EVAR’s) ( p = .01). The sub-score for anatomical measurements was 10.6 (3.9–18.8) and comparable with the experienced group 9.7 (8.1–12.8) ( p = .83) but inferior to the expert operators 6.5 (5.2–10.2) ( p = .04). The sub-score for stent graft selection was 7.5 (4.9–14.1) and comparable with experienced operators scoring 4.5 (3.6–12.3) ( p = .09) but inferior to the expert operators score of 5.0 (3.6–8.4) ( p = .01).

Conclusion

This study presents the results of a standardised one-day basic EVAR sizing and graft selection workshop. Vascular surgery trainees with no prior EVAR experience learned to size and select stent grafts for a simple infra-renal AAA on par with experienced EVAR operators.

A Binational Need Assessment to Define the Level of Endovascular Expertise Required by Vascular Surgical Trainees

BACKGROUND

There has been a shift toward competency-based surgical education programs to improve trainee performance and achieve better patient outcomes. Endovascular procedures comprise a significant volume of vascular surgery, but the current methods for assessing the endovascular competence of vascular trainees in Australia and New Zealand are suboptimal. The objective of this study was to perform a need assessment to define the scope of endovascular expertise required by vascular surgical trainees to later aid in the development of novel surgical training assessment tools.

METHODS

A modified Delphi method was used to achieve expert consensus. Fifty-three key stakeholders in vascular surgical education and training (SET) in Australia and New Zealand were invited to take part in the 2-stage survey. Experts were asked which procedures they considered to be requisite for vascular surgery trainees and at which SET level competence should be achieved. The results were reiterated to the expert panel in the second stage, and consensus considered achieved if over 75% of experts were in agreement.

RESULTS

In the first stage 25 experts reached consensus that competence in 18 of the 26 procedures should be requisite for SET trainees. Twenty-two experts responded to the second stage and consensus was achieved for 12 out of 14 of the procedural items with mean percentage of experts in agreement being 90%.

CONCLUSIONS

A need assessment using a modified Delphi method has achieved consensus among experts in vascular surgery regarding the endovascular procedures considered to be requisite for vascular surgery trainees in Australia and New Zealand.