Transatlantic Multispecialty Consensus on Fundamental Endovascular Skills: Results of a Delphi Consensus Study

Expert consensus on the attributes and competencies required for rural and remote junior physicians to work effectively in isolated indonesian communities

Indonesian physicians working in rural and remote areas must be equipped not only with generic competencies but also with the attributes and skills necessary to provide health care services without compromising quality. This study sought to reach a consensus on the attributes and competencies that are viewed as essential and important for working effectively as an early career doctor in rural and remote practice in Indonesia. A two-round Delphi study was conducted by reference to 27 consenting physicians working in rural and remote Indonesia. Forty-three items covering 9 attributes and 34 competencies were sent to these physicians to be rated on a Likert scale ranging from 1 to 5 in terms of their importance for effective rural and remote practice. Nine attributes and 29 competencies progressed to Round 2. All nine attributes and 29 competencies were identified as essential or important for junior physicians' ability to be effective in their practice. The essential attributes included professional quality related to prioritising the rural community. The essential competencies included medical skills, professional behaviour, interprofessional skills, health promotion and connection to the rural community. The consensus thus reached on these essential and important attributes and competencies can inform curriculum development for the undergraduate and postgraduate training of junior rural and remote physicians.

Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions

OBJECTIVE

Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI).

METHODS

A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases.

RESULTS

The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case.

CONCLUSION

An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.

Adult essential extracorporeal membrane oxygenation (ECMO) skills for use in an e-learning program for ICU physicians, nurses and perfusionists: a consensus by a modified Delphi questionnaire

BACKGROUND

Education in ECMO starts with basic theory and physiology. For this type of training, self-assessment e-learning modules may be beneficial. The aim of this study was to generate consensus on essential ECMO skills involving various professional groups involved in caring for ECMO patients. These skills can be used for educational purposes: development of an e-learning program and fine-tuning of ECMO-simulation programs.

METHODS

Experts worldwide received an e-mail inviting them to participate in the modified Delphi questionnaire. A mixture of ECMO experts was contacted. The expert list was formed based on their scientific track record mainly in adult ECMO (research, publications, and invited presentations). This survey consisted of carefully designed questionnaires, organized into three categories, namely knowledge skills, technical skills, and attitudes. Each statement considered a skill and was rated on a 5-point Likert-scale and qualitative comments were made if needed. Based on the summarized information and feedback, the next round Delphi questionnaire was developed. A statement was considered as a key competency when at least 80% of the experts agreed or strongly agreed (rating 4/5 and 5/5) with the statement. Cronbach's Alpha score tested internal consistency. Intraclass correlation coefficient was used as reliability index for interrater consistency and agreement.

RESULTS

Consensus was achieved in two rounds. Response rate in the first round was 45.3% (48/106) and 60.4% (29/48) completed the second round. Experts had respectively for the first and second round: a mean age of 43.7 years (8.2) and 43.4 (8.8), a median level of experience of 11.0 years [7.0-15.0] and 12.0 years [8.3-14.8]. Consensus was achieved with 29 experts from Australia (2), Belgium (16), France (1), Germany (1), Italy (1), Russia (2), Spain (1), Sweden, (1), The Netherlands (4). The consensus achieved in the first round was 90.9% for the statements about knowledge, 54.5% about technical skills and 75.0% about attitudes. Consensus increased in the second round: 94.6% about knowledge skills, 90.9% about technical skills and 75.0% about attitudes.

CONCLUSION

An expert consensus was accomplished about the content of "adult essential ECMO skills". This consensus was mainly created with participation of physicians, as the response rate for nurses and perfusion decreased in the second round.

Validity Evidence for Procedure-Specific Competency Assessment Tools in Cardiovascular and Thoracic Surgery: A Scoping Review

OBJECTIVE

Surgical education has shifted from a time-based approach to the achievement and demonstration of procedural competency. High quality, objective assessment instruments are required to support this new approach. This study comprehensively reviewed the literature to identify and evaluate available procedure-specific assessment instruments in cardiothoracic and vascular surgery.

DESIGN

A systematic search of 8 databases identified studies containing procedure-specific operative assessment instruments in cardiothoracic and vascular surgery. Generic global rating scales were excluded, unless modified to be procedure-specific. Two reviewers independently evaluated the validity evidence, methodological rigour and educational utility of each instrument using objective scoring criteria. Validity evidence was evaluated with a scoring tool aligned with the contemporary framework of validity. Methodological rigour was evaluated using the Medical Education Research Study Quality Instrument. Educational utility was evaluated according to the Accreditation Council for Graduate Medical Education (ACGME) framework.

RESULTS

There were 2130 unique studies describing procedure-specific assessment in surgery. Of these, 9 studies evaluating 8 procedure-specific assessment instruments met inclusion criteria for cardiothoracic and vascular surgery. Four instruments were identified in thoracic surgery, 2 in cardiac surgery, and 2 in vascular surgery. Only 1 instrument was designed to evaluate surgeon performance, with the remainder designed to evaluate residents. No single instrument scored the maximum score of 15 for validity evidence. The highest score was 11, with 62.5% (n = 5) of instruments scoring greater than 10. All tools attained high scores in content validity, with minimal evidence generally presented regarding the consequences of assessment using a particular instrument. All but 1 instrument scored greater than 11 out of a maximum 16.5 points for methodological rigour. Very few studies reported on the ACGME domains of educational utility.

CONCLUSIONS

In an era where surgical education is shifting towards the demonstration of procedural competency, objective procedure-specific assessment is critical. This review identified that few procedure-specific assessment instruments in cardiothoracic and vascular surgery exist, emphasizing the need for such instruments to ensure the success of competency-based education models.

A Multilayer and Multimodal-Fusion Architecture for Simultaneous Recognition of Endovascular Manipulations and Assessment of Technical Skills

The clinical success of the percutaneous coronary intervention (PCI) is highly dependent on endovascular manipulation skills and dexterous manipulation strategies of interventionalists. However, the analysis of endovascular manipulations and related discussion for technical skill assessment are limited. In this study, a multilayer and multimodal-fusion architecture is proposed to recognize six typical endovascular manipulations. The synchronously acquired multimodal motion signals from ten subjects are used as the inputs of the architecture independently. Six classification-based and two rule-based fusion algorithms are evaluated for performance comparisons. The recognition metrics under the determined architecture are further used to assess technical skills. The experimental results indicate that the proposed architecture can achieve the overall accuracy of 96.41%, much higher than that of a single-layer recognition architecture (92.85%). In addition, the multimodal fusion brings significant performance improvement in comparison with single-modal schemes. Furthermore, the K -means-based skill assessment can obtain an accuracy of 95% to cluster the attempts made by different skill-level groups. These hopeful results indicate the great possibility of the architecture to facilitate clinical skill assessment and skill learning.

Step, Error, and Event Frameworks in Endovascular Aortic Repair

OBJECTIVE

Competency-based surgical education requires detailed and actionable feedback to ensure adequate and efficient skill development. Comprehensive operative capture systems such as the Operating Room Black Box (ORBB; Surgical Safety Technologies, Inc), which continuously records and synchronizes multiple sources of intraoperative data, have recently been integrated into hybrid rooms to provide targeted feedback to endovascular teams. The objective of this study is to develop step, error, and event frameworks to evaluate technical performance in elective endovascular aortic repair (EVAR) comprehensively captured by the ORBB (Surgical Safety Technologies, Inc; Toronto, Canada).

METHODS

This study is based upon a modified Delphi consensus process to create evaluation frameworks for steps, errors, and events in EVAR. International experts from Vascular Surgery and Interventional Radiology were identified, based on their records of publications and invited presentations, or serving on relevant journal editorial boards. In an initial open-ended survey round, experts were asked to volunteer a comprehensive list of steps, errors, and events for a standard EVAR of an infrarenal aorto-iliac aneurysm (AAA). In subsequent survey rounds, the identified items were presented to the expert panel to rate on a 5-point Likert scale. Delphi survey rounds were repeated until the process reached consensus with a predefined agreement threshold (Cronbach α>0.7). The final frameworks were constructed with items achieving an agreement (responses of 4 or 5) from greater than 70% of experts.

RESULTS

Of 98 invited proceduralists, 38 formed the expert consensus panel (39%), consisting of 29 vascular surgeons and 9 interventional radiologists, with 34% from North America and 66% from Europe. Consensus criteria were met following the third round of the Delphi consensus process (Cronbach α=0.82-0.93). There were 15, 32, and 25 items in the error, step, and event frameworks, respectively (within-item agreement=74%-100%).

CONCLUSION

A detailed evaluation tool for the procedural steps, errors, and events in infrarenal EVAR was developed. This tool will be validated on recorded procedures in future work: It may focus skill development on common errors and hazardous steps. This tool might be used to provide high-quality feedback on technical performance of trainees and experienced surgeons alike, thus promoting surgical mastery.

Pan-Canadian initiative on Fundamental Competencies for Transcatheter Cardiac Surgery: A modified Delphi consensus study

OBJECTIVE

Transcatheter cardiac procedures have generated increasing interest in trainees and training programs alike. Using the modified Delphi method, we sought to clarify the transcatheter competencies that cardiac surgery residents should be expected to attain by the completion of training.

METHODS

Individuals with expertise in transcatheter structural heart and aortic procedures were recruited across Canada. A questionnaire was prepared using a 5-point Likert scale. During 2 rounds, participants rated the competencies that they thought cardiac surgery residents should be required to achieve to perform transcatheter procedures. Data were analyzed and presented to participants between rounds. Competencies rated 4 or higher by at least 80% of respondents after the second round were considered fundamental to transcatheter cardiac surgical training.

RESULTS

A total of 46 individuals participated in the study, including 23 cardiac surgeons, 17 interventional cardiologists, and 6 vascular surgeons. Participants with relevant experience performed a median of 75 (interquartile range, 40-100) transcatheter aortic valve implantations in the prior year as primary or secondary operator and 15 (interquartile range, 11-35) thoracic endovascular aortic repairs in the prior 2 years as primary operator. Median clinical and teaching experience consisted of 13 (interquartile range, 7-19.5) years in practice and 8.5 (interquartile range, 5-15) residents taught per year, respectively. Of the included competencies, 53 were considered fundamental to transcatheter cardiac surgical training.

CONCLUSIONS

The identified fundamental competencies can be used to develop educational strategies during transcatheter cardiac surgery training. Future efforts should focus on collecting evidence for their validity.

Definition of Proficiency Level by a Virtual Simulator as a First Step Toward a Curriculum on Fundamental Skills for Endovascular Aneurysm Repair (EVAR)

OBJECTIVE

At present, there is no proficiency-based curriculum for endovascular treatment of aortic aneurysm repair (EVAR) using virtual reality (VR) surgical simulators, whereas such curricula are available for the treatment of iliac and/or superficial femoral artery disease. The purpose of this work was to compute proficiency, defined by a benchmark level determined by the performance of experts, using a commercial VR simulator as a first step of a curriculum on EVAR.

MATERIALS AND METHODS

Expert endovascular surgeons (with more than 150 EVAR cases as first operators) from 12 major Italian centers completed three cases of EVAR of increasing difficulty level 3 times each, using the Angio Mentor simulator (by Simbionix) and Gore devices. Proficiency level was based on performance of expert surgeons, as assessed by metrics from a VR simulator.

RESULTS

The participating surgeons had a median of 20 years of experience and executed a median of 440 EVAR. For the 3 simulated cases, the following proficiency values were respectively obtained: total procedure time: 22 minutes 32 seconds, 23 minutes 05 seconds, and 20 minutes 32 seconds; total amount of contrast injected: 85.16 mL, 89.97 mL, and 98.01 mL total fluoroscopy time: 10 minutes 39 seconds, 12 minutes 22 seconds, and 10 minutes 17 seconds; time to contralateral gate cannulation: 5 minutes 51 seconds, 7 minutes 09 seconds, and 3 minutes 32 seconds.

CONCLUSIONS

We computed proficiency levels for 3 simulated cases of EVAR using a VR simulator. Our next step is to determine whether surgical residents can reach this level. Translational research will then be required to assess the impact of such training on real patients.

Key components of a hepatobiliary surgery curriculum for general surgery residents: results of the FULCRUM International Delphi consensus

BACKGROUND

In general surgery residency, hepatobiliary training varies significantly across the world. The aim of this study was to establish an international consensus among hepatobiliary surgeons on components of a hepatobiliary curriculum for general surgery residents.

METHODS

A three-round modified Delphi technique was employed. Fifty-two hepatobiliary surgeons involved in general surgery training programs were invited. An initial questionnaire was developed by a group of experts in hepatobiliary and educational research after a systematic literature review. It comprised 90 statements about knowledge, technical skills, attitudes, and postoperative care. Panelists could add or alter items. The survey was delivered electronically and the panel was instructed to score the items based on 5-point Likert scale. Consensus was reached when at least 80% of panelists agreed on a statement with Cronbach's alpha value >0.8.

RESULTS

Forty-one (79%) experts have participated. Sixteen panelists are based in Asia, 14 in Europe, and 11 in the Americas. Eighty percent of all proposed skills (81/101) were considered fundamental including knowledge (39/43), technical skills (16/32), attitude (15/15), and postoperative care (11/11).

CONCLUSION

An international consensus was achieved on components of a hepatobiliary curriculum. Acquiring broad knowledge is fundamental during residency. Advanced liver resection techniques require specialized hepatobiliary training.

Standardization of dermoscopic terminology and basic dermoscopic parameters to evaluate in general dermatology (non-neoplastic dermatoses): an expert consensus on behalf of the International Dermoscopy Society

BACKGROUND

Over the last few years, several articles on dermoscopy of non-neoplastic dermatoses have been published, yet there is poor consistency in the terminology among different studies.

OBJECTIVES

We aimed to standardize the dermoscopic terminology and identify basic parameters to evaluate in non-neoplastic dermatoses through an expert consensus.

METHODS

The modified Delphi method was followed, with two phases: (i) identification of a list of possible items based on a systematic literature review and (ii) selection of parameters by a panel of experts through a three-step iterative procedure (blinded e-mail interaction in rounds 1 and 3 and a face-to-face meeting in round 2). Initial panellists were recruited via e-mail from all over the world based on their expertise on dermoscopy of non-neoplastic dermatoses.

RESULTS

Twenty-four international experts took part in all rounds of the consensus and 13 further international participants were also involved in round 2. Five standardized basic parameters were identified: (i) vessels (including morphology and distribution); (ii) scales (including colour and distribution); (iii) follicular findings; (iv) 'other structures' (including colour and morphology); and (v) 'specific clues'. For each of them, possible variables were selected, with a total of 31 different subitems reaching agreement at the end of the consensus (all of the 29 proposed initially plus two more added in the course of the consensus procedure).

CONCLUSIONS

This expert consensus provides a set of standardized basic dermoscopic parameters to follow when evaluating inflammatory, infiltrative and infectious dermatoses. This tool, if adopted by clinicians and researchers in this field, is likely to enhance the reproducibility and comparability of existing and future research findings and uniformly expand the universal knowledge on dermoscopy in general dermatology. What's already known about this topic? Over the last few years, several papers have been published attempting to describe the dermoscopic features of non-neoplastic dermatoses, yet there is poor consistency in the terminology among different studies. What does this study add? The present expert consensus provides a set of standardized basic dermoscopic parameters to follow when evaluating inflammatory, infiltrative and infectious dermatoses. This consensus should enhance the reproducibility and comparability of existing and future research findings and uniformly expand the universal knowledge on dermoscopy in general dermatology.

Defining the Key Competencies in Radiation Protection for Endovascular Procedures: A Multispecialty Delphi Consensus Study

OBJECTIVES

Radiation protection training courses currently focus on broad knowledge topics which may not always be relevant in daily practice. The goal of this study was to determine the key competencies in radiation protection that every endovascular team member should possess and apply routinely, through multispecialty clinical content expert consensus.

METHODS

Consensus was obtained through a two round modified Delphi methodology. The expert panel consisted of European vascular surgeons, interventional radiologists, and interventional cardiologists/angiologists experienced in endovascular procedures. An initial list of statements, covering knowledge skills, technical skills and attitudes was created, based on a literature search. Additional statements could be suggested by the experts in the first Delphi round. Each of the statements had to be rated on a 5- point Likert scale. A statement was considered to be a key competency when the internal consistency was greater than alpha = 0.80 and at least 80% of the experts agreed (rating 4/5) or strongly agreed (rating 5/5) with the statement. Questionnaires were emailed to panel members using the Surveymonkey service.

RESULTS

Forty-one of 65 (63.1%) invited experts agreed to participate in the study. The response rates were 36 out of 41 (87.8%): overall 38 out of 41(92.6%) in the first round and 36 out of 38 (94.7%) in the second round. The 71 primary statements were supplemented with nine items suggested by the panel. The results showed excellent consensus among responders (Cronbach's alpha = 0.937 first round; 0.958 s round). Experts achieved a consensus that 30 of 33 knowledge skills (90.9%), 23 of 27 technical skills (82.1%), and 15 of 20 attitudes (75.0%) should be considered as key competencies.

CONCLUSIONS

A multispecialty European endovascular expert panel reached consensus about the key competencies in radiation protection. These results may serve to create practical and relevant radiation protection training courses in the future, enhancing radiation safety for both patients and the entire endovascular team.

A Proficiency Based Stepwise Endovascular Curricular Training (PROSPECT) Program Enhances Operative Performance in Real Life: A Randomised Controlled Trial

OBJECTIVES

Healthcare evolution requires optimisation of surgical training to provide safe patient care. Operating room performance after completion of proficiency based training in vascular surgery has not been investigated.

DESIGN

A randomised controlled trial evaluated the impact of a Proficiency based Stepwise Endovascular Curricular Training program (PROSPECT) on the acquisition of endovascular skills and the transferability of these skills to real life interventions.

MATERIALS

All subjects performed two endovascular interventions treating patients with symptomatic iliac and/or superficial femoral artery stenosis under supervision. Primary outcomes were technical performances (Global Rating Scale [GRS]; Examiner Checklist), operative metrics, and patient outcomes, adjusted for case difficulty and trainee experience. Secondary outcomes included knowledge and technical performance after 6 weeks and 3 months.

METHODS

Thirty-two general surgical trainees were randomised into three groups. Besides traditional training, the first group (n = 11) received e-learning and simulation training (PROSPECT), the second group (n = 10) only had access to e-learning, while controls (n = 11) did not receive supplementary training.

RESULTS

Twenty-nine trainees (3 dropouts) performed 58 procedures. Trainees who completed PROSPECT showed superior technical performance (GRS 39.36 ± 2.05; Checklist 63.51 ± 3.18) in real life with significantly fewer supervisor takeovers compared with trainees receiving e-learning alone (GRS 28.42 ± 2.15; p = .001; Checklist 53.63 ± 3.34; p = .027) or traditional education (GRS 23.09 ± 2.18; p = .001; Checklist 38.72 ± 3.38; p = .001). Supervisors felt more confident in allowing PROSPECT trained physicians to perform basic (p = .006) and complex (p = .003) procedures. No differences were detected in procedural parameters (such as fluoroscopy time, DAP, procedure time, etc.) or complications. Proficiency levels were maintained up to 3 months.

CONCLUSIONS

A structured, stepwise, proficiency based endovascular curriculum including e-learning and simulation based training should be integrated early into training programs to enhance trainee performance.

How to define the hemodynamic significance of an equivocal iliofemoral artery stenosis: Review of literature and outcomes of an international questionnaire

Purpose

The goal of the study was to review current literature regarding the diagnosis of equivocal (50–70%) iliofemoral artery stenosis and compare these findings with the daily practice of an international panel of endovascular experts.

Methods

The Medline Database was searched for relevant publications, and an electronic survey was sent to experts in the field covering the following topics: definition of an equivocal iliofemoral artery stenosis, angiographic visualization and investigation protocols of an equivocal stenosis, intra-arterial pressure measurements, and definition of hemodynamic significance of an equivocal iliofemoral artery stenosis using a physiologic measure.

Results

Of the 37 invited endovascular experts, 21 (53.8%) agreed to participate in the survey. Analysis of existing literature shows that the level of evidence for diagnosing equivocal iliofemoral artery stenosis is mediocre and is not being implemented by experts in the field.

Conclusion

Studies have shown that a stenosis of between 50% and 70% iliofemoral lumen diameter reduction shows a wide range of trans-stenotic pressure gradients. Equivocal iliofemoral artery stenosis can best be identified using three-dimensional quantitative vascular analysis software. Although evidence for a clear hemodynamic cutoff point is weak, performing trans-lesion intra-arterial pressure measurements at rest and during maximal hyperemia is preferred. Diagnosing iliofemoral artery stenosis solely on lumen diameter reduction is inadequate.