Assessment Tools for Use During Anesthesia-Centric Pediatric Advanced Life Support Training and Evaluation

A Consensus-Driven Approach to Redesigning Graduate Medical Education: The Pediatric Anesthesiology Delphi Study

BACKGROUND

Pediatric anesthesiology fellowship education has necessarily evolved since Accreditation Council for Graduate Medical Education (ACGME) accreditation in 1997. Advancements in perioperative and surgical practices, emerging roles in leadership, increasing mandates by accreditation and certification bodies, and progression toward competency-based education-among other things-have created pressure to enrich the current pediatric anesthesiology training system. The Society for Pediatric Anesthesia (SPA) formed a Task Force for Pediatric Anesthesiology Graduate Medical Education that included key leaders and subject matter experts from the society. A key element of the Task Force's charge was to identify curricular and evaluative enhancements for the fellowship program of the future.

METHODS

The Task Force executed a nationally representative, stakeholder-based Delphi process centered around a fundamental theme: "What makes a pediatric anesthesiologist?" to build consensus among a demographically varied and broad group of anesthesiologists within the pediatric anesthesiology community. A total of 37 demographically and geographically varied pediatric anesthesiologists participated in iterative rounds of open- and close-ended survey work between August 2020 and July 2021 to build consensus on the current state, known deficiencies, anticipated needs, and strategies for enhancing national educational offerings and program requirements.

RESULTS

Participation was robust, and consensus was almost completely achieved by round 2. This work generated a compelling Strengths, Weaknesses, Opportunities, and Threats (SWOT) analysis that suggests more strengths and opportunities in the current Pediatric Anesthesiology Graduate Medical Education program than weaknesses or threats. Stakeholders agreed that while fellows matriculate with some clinical knowledge and procedural gaps, a few clinical gaps exist upon graduation. Stakeholders agreed on 8 nonclinical domains and specific fundamental and foundational knowledge or skills that should be taught to all pediatric anesthesiology fellows regardless of career plans. These domains include (1) patient safety, (2) quality improvement, (3) communication skills, (4) supervision skills, (5) leadership, (6) medical education, (7) research basics, and (8) practice management. They also agreed that a new case log system should be created to better reflect modern pediatric anesthesia practice. Stakeholders further identified the need for the development of standardized and validated formative and summative assessment tools as part of a competency-based system. Finally, stakeholders noted that significant departmental, institutional, and national organizational support will be necessary to implement the specific recommendations.

CONCLUSIONS

A Delphi process achieved robust consensus in assessing current training and recommending future directions for pediatric anesthesiology graduate medical education.

Assessing Pediatric Life Support Skills Using Augmented Reality Medical Simulation With Eye Tracking: A Pilot Study

BACKGROUND

Augmented reality (AR) and eye tracking are promising adjuncts for medical simulation, but they have remained distinct tools. The recently developed Chariot Augmented Reality Medical (CHARM) Simulator combines AR medical simulation with eye tracking. We present a novel approach to applying eye tracking within an AR simulation to assess anesthesiologists during an AR pediatric life support simulation. The primary aim was to explore clinician performance in the simulation. Secondary outcomes explored eye tracking as a measure of shockable rhythm recognition and participant satisfaction.

METHODS

Anesthesiology residents, pediatric anesthesiology fellows, and attending pediatric anesthesiologists were recruited. Using CHARM, they participated in a pediatric crisis simulation. Performance was scored using the Anesthesia-centric Pediatric Advanced Life Support (A-PALS) scoring instrument, and eye tracking data were analyzed. The Simulation Design Scale measured participant satisfaction.

RESULTS

Nine each of residents, fellows, and attendings participated for a total of 27. We were able to successfully progress participants through the AR simulation as demonstrated by typical A-PALS performance scores. We observed no differences in performance across training levels. Eye tracking data successfully allowed comparisons of time to rhythm recognition across training levels, revealing no differences. Finally, simulation satisfaction was high across all participants.

CONCLUSIONS

While the agreement between A-PALS score and gaze patterns is promising, further research is needed to fully demonstrate the use of AR eye tracking for medical training and assessment. Physicians of multiple training levels were satisfied with the technology.

Tools for Assessing the Performance of Pediatric Perioperative Teams During Simulated Crises: A Psychometric Analysis of Clinician Raters' Scores

INTRODUCTION

The pediatric perioperative setting is a dynamic clinical environment where multidisciplinary interprofessional teams interact to deliver complex care to patients. This environment requires clinical teams to possess high levels of complex technical and nontechnical skills. For perioperative teams to identify and maintain clinical competency, well-developed and easy-to-use measures of competency are needed.

METHODS

Tools for measuring the technical and nontechnical performance of perioperative teams were developed and/or identified, and a group of raters were trained to use the instruments. The trained raters used the tools to assess pediatric teams managing simulated emergencies. A psychometric analysis of the trained raters' scores using the different instruments was performed and the agreement between the trained raters' scores and a reference score was determined.

RESULTS

Five raters were trained and scored 96 recordings of perioperative teams managing simulated emergencies. Scores from both technical skills assessment tools demonstrated significant reliability within and between ratings with the scenario-specific performance checklist tool demonstrating greater interrater agreement than scores from the global rating scale. Scores from both technical skills assessment tools correlated well with the other and with the reference standard scores. Scores from the Team Emergency Assessment Measure nontechnical assessment tool were more reliable within and between raters and correlated better with the reference standard than scores from the BARS tool.

CONCLUSIONS

The clinicians trained in this study were able to use the technical performance assessment tools with reliable results that correlated well with reference scores. There was more variability between the raters' scores and less correlation with the reference standard when the raters used the nontechnical assessment tools. The global rating scale used in this study was able to measure the performance of teams across a variety of scenarios and may be generalizable for assessing teams in other clinical scenarios. The Team Emergency Assessment Measure tool demonstrated reliable measures when used to assess interprofessional perioperative teams in this study.

The impact of simulation-based medical education on resident management of emergencies in pediatric anesthesiology

BACKGROUND

Resident education in pediatric anesthesiology is challenging. Traditional curricula for anesthesiology residency programs have included a combination of didactic lectures and mentored clinical service, which can be variable. Limited pediatric medical knowledge, technical inexperience, and heightened resident anxiety further challenge patient care. We developed a pediatric anesthesia simulation-based curriculum to address crises related to hypoxemia and dysrhythmia management in the operating room as an adjunct to traditional didactic and clinical experiences.

AIMS

The primary objective of this trial was to evaluate the impact of a simulation curriculum designed for anesthesiology residents on their performance during the management of crises in the pediatric operating room. A secondary objective was to compare the retention of learned knowledge by assessment at the eight-week time point during the rotation.

METHODS

In this prospective, observational trial 30 residents were randomized to receive simulation-based education on four perioperative crises (Laryngospasm, Bronchospasm, Supraventricular Tachycardia (SVT), and Bradycardia) during the first week (Group A) or fifth week (Group B) of an eight-week rotation. Assessment sessions that included two scenarios (Laryngospasm, SVT) were performed in the first week, fifth week, and the eighth week of their rotation for all residents. The residents were assessed in real time and by video review using a 7-point checklist generated by a modified Delphi technique of senior pediatric anesthesiology faculty.

RESULTS

Residents in Group A showed improvement between the first week and fifth week assessment as well as between first week and eighth week assessments without decrement between the fifth week and eighth week assessments for both the laryngospasm and SVT scenarios. Residents in Group B showed improvement between the first week and eighth week assessments for both scenarios and between the fifth week and eighth week assessment for the SVT scenario.

CONCLUSION

This adjunctive simulation-based curriculum enhanced the learner's management of laryngospasm and SVT management and is a reasonable addition to didactic and clinical curricula for anesthesiology residents.