Defining the Key Competencies in Radiation Protection for Endovascular Procedures: A Multispecialty Delphi Consensus Study

Editor's Choice - Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions

OBJECTIVE

Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI).

METHODS

A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases.

RESULTS

The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case.

CONCLUSION

An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.

Development, Feasibility, and Knowledge Impact of a Massive Open Online Course on Radiation Safety: A Multicentre Prospective Cohort Study

OBJECTIVE

This study reports the development of an innovative, interactive Massive Open Online Course (MOOC) teaching radiation safety principles in the vascular workplace, using stepwise e-learning with multiple choice question tests (MCQs), educational videos, and a serious game. The aim was to study the MOOC impact on radiation safety knowledge and assess its feasibility and acceptability.

METHODS

An international multicentre prospective study included team members active in the hybrid operating room. The MOOC was offered voluntarily via a secure online learning platform. A standardised MCQ test (15 questions) assessed radiation safety knowledge pre- and post-course (range 0 - 100%). Acceptability and feasibility were tested via the previously validated, Evaluating e-learning system success (EESS) model, using five point Likert scales.

RESULTS

In eight centres across four countries, 150 of 203 invited endovascular team members consented. Over a seven week study period, surgeons (28%, including vascular surgery trainees and consultants), nurses (27%, including scrub, circulating and anaesthetic nurses), anaesthetists (43%, including trainees and consultants), and radiographers (3%) participated. Of those, 67% completed the course. The average radiation knowledge improved by 22.8% (95% CI 19.5 - 26.0%; p < .001) after MOOC completion, from 48% to 71% (standard deviation [SD] 14 and 15% respectively), requiring a mean time investment of 169 minutes (SD 89 minutes). In centres with a radiographer, mean knowledge gain after MOOC completion was significantly smaller (14%, SD 19% vs. 24%, SD 16%, p = .036). The course was deemed feasible and acceptable according to the EESS model with a total mean score of 3.68/5.

CONCLUSION

This newly developed, multimodal MOOC was deemed feasible and effective across multiple international centres. The MOOC significantly contributes to radiation safety education of the entire endovascular team, improving radiation safety knowledge. The course may optimise workplace radiation safety behaviour and therefore enhance team and patient safety.

Focus on Radiation Protection Improves Both Correct Behavior and Procedural Performance During Simulation-Based Training - A Randomized Comparison

BACKGROUND

To explore whether simulation-based endovascular training with focus on radiation safety could improve correct behavior without jeopardizing the learning of procedural skills.

METHODS

Twenty-four residents without previous endovascular experience completed 10 clinical scenarios on a virtual-reality endovascular simulator with software for peripheral endovascular interventions. Participants were randomized to receive feedback (n = 12) or not (n = 12) on radiation protection (RP) performance after each case. Expert assessments were done at the first, second, fourth, seventh, and 10th case on RP and endovascular skills (ES). Automatic simulator metrics on procedure time, contrast dose, handling errors, and estimated radiation exposure to patient and operator were registered. Outcome metrics were analyzed by two-way mixed analysis of variance pairwise comparisons with independent t-tests. Correlations were explored using Pearson's r for internal consistency reliability.

RESULTS

The RP performance was similar in both groups at their first attempt (P = 0.61), but the feedback group significantly outperformed the control group over time (P < 0.001 for all comparisons). The feedback group was however slower to learn the ES at start (P = 0.047 at second performance), but after 7 attempts no difference was shown (P = 0.59). The feedback group used more time (19.5 vs. 15.3 min; P = 0.007) but less contrast (60 vs. 100 mL; P < 0.001). The number of errors was the same in both groups, but all metrics regarding radiation exposure favored the feedback group (P-values from 0.001 to 0.008).

CONCLUSIONS

Simulation-based training (SBT) is effective to acquire basic endovascular intervention skills and concurrently learn RP behavior when feedback on radiation culture is provided.

Resident Readiness for Senior Level Decision Making: Identifying the Domains for Formative Assessment and Feedback

OBJECTIVE

To establish expert consensus regarding the domains and topics for senior surgery residents (PGY-4) to make critical decisions and assume senior-level responsibilities, and to develop the formative American College of Surgeons Senior Resident Readiness Assessment (ACS SRRA) Program.

DESIGN

The American College of Surgeons (ACS) education leadership team conducted a focus group with surgical experts to identify the content for an assessment tool to evaluate senior residents' readiness for their increased levels of responsibility. After the focus group, national experts were recruited to develop consensus on the topics through three rounds of surveys using Delphi methodology. The Delphi participants rated topics using Likert-type scales and their comments were incorporated into subsequent rounds. Consensus was defined as ≥ 80% agreement with internal-consistency reliability (Cronbach's alpha) ≥ 0.8. In a stepwise fashion, topics that did not achieve consensus for inclusion were removed from subsequent survey rounds.

SETTING

The surveys were administered via an online questionnaire.

PARTICIPANTS

Twelve program directors and assistant program directors made up the focus group. The 39 Delphi participants represented seven different surgical subspecialties and were from diverse practice settings. The median length of experience in general surgery resident education was 20 years (IQR 14.3-30.0) with 64% of the experts being either current or past general surgery residency program directors.

RESULTS

The response rate was 100% and Cronbach's alpha was ≥ 0.9 for each round. The Delphi participants contributed a large number of comments. Of the 201 topics that were evaluated initially, 120 topics in 25 core clinical areas were included to create the final domains of ACS SRRA.

CONCLUSIONS

National consensus on the domain of the ACS SRRA has been achieved via the modified Delphi method among expert surgeon educators. ACS SRRA will identify clinical topics and areas in which each senior resident needs improvement and provide data to residents and residency programs to develop individualized learning plans. This would help in preparing the senior residents to assume their responsibilities and support their readiness for future fellowship training or surgical practice.

Step, Error, and Event Frameworks in Endovascular Aortic Repair

OBJECTIVE

Competency-based surgical education requires detailed and actionable feedback to ensure adequate and efficient skill development. Comprehensive operative capture systems such as the Operating Room Black Box (ORBB; Surgical Safety Technologies, Inc), which continuously records and synchronizes multiple sources of intraoperative data, have recently been integrated into hybrid rooms to provide targeted feedback to endovascular teams. The objective of this study is to develop step, error, and event frameworks to evaluate technical performance in elective endovascular aortic repair (EVAR) comprehensively captured by the ORBB (Surgical Safety Technologies, Inc; Toronto, Canada).

METHODS

This study is based upon a modified Delphi consensus process to create evaluation frameworks for steps, errors, and events in EVAR. International experts from Vascular Surgery and Interventional Radiology were identified, based on their records of publications and invited presentations, or serving on relevant journal editorial boards. In an initial open-ended survey round, experts were asked to volunteer a comprehensive list of steps, errors, and events for a standard EVAR of an infrarenal aorto-iliac aneurysm (AAA). In subsequent survey rounds, the identified items were presented to the expert panel to rate on a 5-point Likert scale. Delphi survey rounds were repeated until the process reached consensus with a predefined agreement threshold (Cronbach α>0.7). The final frameworks were constructed with items achieving an agreement (responses of 4 or 5) from greater than 70% of experts.

RESULTS

Of 98 invited proceduralists, 38 formed the expert consensus panel (39%), consisting of 29 vascular surgeons and 9 interventional radiologists, with 34% from North America and 66% from Europe. Consensus criteria were met following the third round of the Delphi consensus process (Cronbach α=0.82-0.93). There were 15, 32, and 25 items in the error, step, and event frameworks, respectively (within-item agreement=74%-100%).

CONCLUSION

A detailed evaluation tool for the procedural steps, errors, and events in infrarenal EVAR was developed. This tool will be validated on recorded procedures in future work: It may focus skill development on common errors and hazardous steps. This tool might be used to provide high-quality feedback on technical performance of trainees and experienced surgeons alike, thus promoting surgical mastery.

Unifying the Hepatopancreatobiliary Surgery Fellowship Curriculum via Delphi Consensus

BACKGROUND

Hepatopancreatobiliary (HPB) Fellowship training in the Americas consists of 3 distinctive routes with variable curricula: Surgical Oncology Fellowship via the Society of Surgical Oncology (SSO), Abdominal Transplant Surgery Fellowship via the American Society of Transplant Surgeons (ASTS), and HPB Fellowship via the Americas Hepato-Pancreato-Biliary Association (AHPBA). Our objective was to establish a pan-American consensus among HPB surgeons, surgical oncologists, abdominal transplant surgeons, and general surgery residency program directors (GSPDs) on a core knowledge curriculum for HPB fellowship, and to identify topics appropriate for general surgery residency and subspecialty beyond HPB fellowship.

STUDY DESIGN

A 3-round modified Delphi process was used. Baseline statements were developed by the Education and Training Committee of the AHPBA, in collaboration with representatives of the SSO, ASTS, and GSPDs. The expert panel, consisting of members of the 3 societies together with GSPDs, rated the statements on a 5-point Likert scale and suggested editing or adding new statements. A statement was included in the final curriculum when Cronbach's alpha value was ≥ 0.8 and ≥ 80% of the panel agreed on inclusion.

RESULTS

The response rate was 100% for the first round, and 98% for the second and third rounds. Eighty-nine of 138 proposed statements were included in the final HPB fellowship curriculum. Curricula for general surgery residency and subspecialty beyond HPB fellowship included 50 and 29 statements, respectively.

CONCLUSIONS

A multinational consensus on core knowledge for an HPB fellowship curriculum was achieved via the modified Delphi method. This core curriculum may be used to standardize HPB fellowship training across different pathways in the Americas.

Physician impact on use of fluoroscopy during endovascular procedures to improve radiation safety

OBJECTIVE

To determine whether differences exist in fluoroscopy time and radiation exposure during lower extremity endovascular procedures performed by fellowship trained vascular surgeons vs general surgeons, to minimize radiation exposure to operating room staff.

METHODS

A retrospective review of all lower extremity endovascular procedures was performed from August 1, 2014, to January 29, 2016. The procedures were performed by the surgical department's four surgeons with endovascular privileges: two vascular surgeons and two general surgeons. Only procedures involving lower extremity arterial angiograms with balloon angioplasty, stenting, or atherectomy were included. The operative records were reviewed for each case. The total fluoroscopy time and total radiation dose for each procedure were recorded. Procedures were grouped according to the number of endovascular interventions as one to two interventions, three to four interventions, and five or more interventions performed. Statistical analysis was performed with a P value of less than .05 considered significant.

RESULTS

About 271 lower extremity endovascular procedures were performed during the study period by 4 surgeons. The average age of the patient population was 70 years. The total number of procedures performed over the study period were 112, 45, 91, and 25 for surgeons 1 through 4, respectively. On average, 3.24 interventions were performed during each procedure. Vascular surgeons were found to have shorter fluoroscopy time for procedures involving one to two (7.8 vs 30.1; P < .01), three to four (9.3 vs 34.2; P < .01), and five or more (11.5 vs 51.9; P < .01) interventions. Vascular surgeons were also found to have less radiation exposure compared with general surgeons in procedures with one to two (1.69 vs 3.53; P = .001) and five or more (2.3 vs 5.4; P = .003) interventions. There was no significant difference in radiation exposure between vascular and general surgeons for procedures with three to four interventions (5.86 vs 5.59; P = .95).

CONCLUSIONS

In this small series at our institution, lower extremity endovascular procedures performed by specialty trained vascular surgeons were associated with both decreased operative fluoroscopy time and decreased radiation exposure when compared with general surgeons.

Key components of a hepatobiliary surgery curriculum for general surgery residents: results of the FULCRUM International Delphi consensus

BACKGROUND

In general surgery residency, hepatobiliary training varies significantly across the world. The aim of this study was to establish an international consensus among hepatobiliary surgeons on components of a hepatobiliary curriculum for general surgery residents.

METHODS

A three-round modified Delphi technique was employed. Fifty-two hepatobiliary surgeons involved in general surgery training programs were invited. An initial questionnaire was developed by a group of experts in hepatobiliary and educational research after a systematic literature review. It comprised 90 statements about knowledge, technical skills, attitudes, and postoperative care. Panelists could add or alter items. The survey was delivered electronically and the panel was instructed to score the items based on 5-point Likert scale. Consensus was reached when at least 80% of panelists agreed on a statement with Cronbach's alpha value >0.8.

RESULTS

Forty-one (79%) experts have participated. Sixteen panelists are based in Asia, 14 in Europe, and 11 in the Americas. Eighty percent of all proposed skills (81/101) were considered fundamental including knowledge (39/43), technical skills (16/32), attitude (15/15), and postoperative care (11/11).

CONCLUSION

An international consensus was achieved on components of a hepatobiliary curriculum. Acquiring broad knowledge is fundamental during residency. Advanced liver resection techniques require specialized hepatobiliary training.

Radiation Safety Performance is More than Simply Measuring Doses! Development of a Radiation Safety Rating Scale

PURPOSE

Radiation safety performance is often evaluated using dose parameters measured by personal dosimeters and/or the C-arm, which provide limited information about teams' actual radiation safety behaviors. This study aimed to develop a rating scale to evaluate team radiation safety behaviors more accurately and investigate its reliability.

MATERIALS AND METHODS

A modified Delphi consensus was organized involving European vascular surgeons (VS), interventional radiologists, and interventional cardiologists. Initial items and anchors were drafted a priori and rated using five-point Likert scales. Participants could suggest additional items or adjustments. Consensus was defined as ≥ 80% agreement (rating ≥ 4) with Cronbach's alpha ≥ .80. Two VS with expertise in radiation safety evaluated 15 video-recorded endovascular repairs of infrarenal aortic aneurysms (EVAR) to assess usability, inter and intrarater reliability.

RESULTS

Thirty-one of 46 invited specialists completed three rating rounds to generate the final rating scale. Five items underwent major adjustments. In the final round, consensus was achieved for all items (alpha = .804; agreement > 87%): 'Pre-procedural planning', 'Preparation in angiosuite/operating room', 'Shielding equipment', 'Personal protective equipment', 'Position of operator/team', 'Radiation usage awareness', 'C-arm handling', 'Adjusting image quality', 'Additional dose reducing functions', 'Communication/leadership', and 'Overall radiation performance and ALARA principle'. All EVARs were rated, yielding excellent Cronbach's alpha (.877) with acceptable interrater and excellent intrarater reliability (ICC = .782; ICC = .963, respectively).

CONCLUSION

A reliable framework was developed to assess radiation safety behaviors in endovascular practice and provide teams with formative feedback. The final scale is provided in this publication.

Percutaneous AXillary Artery (PAXA) Access at the First Segment During Fenestrated and Branched Endovascular Aortic Procedures

OBJECTIVE

The aim of this study was to assess the feasibility and safety of percutaneous axillary access with vessel closure device closure after puncturing the first segment during endovascular treatment of complex aneurysms with fenestrated and branched endografts (F/BEVAR).

METHODS

The PAXA (Percutaneous AXillary Access) study is a physician initiated, single centre, ambispective, non-randomised study (clinicaltrials.gov: NCT03223311). The primary endpoint was the closure success rate defined as the absence of any vascular injury to achieve haemostasis at the puncture site, requiring any treatment other than manual compression or adjunctive endovascular ballooning. The secondary endpoints were minor access complications, cerebrovascular complication rate, short term access vessel patency, and study cohort anatomical evaluation of the axillary artery.

RESULTS

Sixty-four patients required an upper extremity access during F/BEVAR procedure during study period (November 2016 to July 2019) and were screened for the PAXA study: 59 patients (47 males; median age: 75 years, IQR 69-78) met the study inclusion criteria and were enrolled (one patient had bilateral access). Closure success was obtained in 54 cases (90%) with no open conversion required: five patients received a bare or covered stenting to the AXA and in one patient mechanical failure of the delivery system was recorded. No 30 day permanent peripheral nerve injuries and two non-ipsilateral ischaemic strokes (3.4%) were recorded. Seven patients (12%) had access haematomas managed conservatively associated with closure failure (p = .002), oral anticoagulants therapy (p = .005) and procedure length (p = .028). At short term follow up (6 months), no late complications were observed, and all access vessels were patent.

CONCLUSION

PAXA on the first segment using a large sheath (10-16F) is technically feasible, relatively safe with no need for open conversion but it may require access related secondary endovascular procedures. Further prospective studies are needed to modify the device instruction for use and to put the procedure on label.