1
|
Castellanos-Ortega Á, Broch MJ, Palacios-Castañeda D, Gómez-Tello V, Valdivia M, Vicent C, Madrid I, Martinez N, Párraga MJ, Sancho E, Fuentes-Dura MDC, Sancerni-Beitia MD, García-Ros R. Competency assessment of residents of Intensive Care Medicine through a simulation-based objective structured clinical evaluation (OSCE). A multicenter observational study. Med Intensiva 2022; 46:491-500. [PMID: 36057440 DOI: 10.1016/j.medine.2022.01.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 01/22/2022] [Indexed: 06/15/2023]
Abstract
OBJECTIVES The current official model of training in Intensive Care Medicine (ICM) in Spain is based on exposure to experiences through clinical rotations. The main objective was to determine the level of competency (I novice to V independent practitioner) achieved by the residents at the end of the 3rd year of training (R3) in ICM through a simulation-based OSCE. Secondary objectives were: (1) To identify gaps in performance, and (2) To investigate the reliability and feasibility of conducting simulation-based assessment at multiple sites. DESIGN Observational multicenter study. SETTING Thirteen Spanish ICU Departments. PARTICIPANTS Thirty six R3. INTERVENTION The participants performed on five, 15-min, high-fidelity crisis scenarios in four simulation centers. The performances were video recorded for later scoring by trained raters. MAIN VARIABLES OF INTEREST Via a Delphi technique, an independent panel of expert intensivists identified critical essential performance elements (CEPE) for each scenario to define the levels of competency. RESULTS A total of 176 performances were analyzed. The internal consistency of the check-lists were adequate (KR-20 range 0.64-0.79). Inter-rater reliability was strong [median Intraclass Correlation Coefficient across scenarios: 0.89 (0.65-0.97)]. Competency levels achieved by R3 were: Level I (18.8%), II (35.2%), III (42.6%), IV/V (3.4%). Overall, a great heterogeneity in performance was observed. CONCLUSION The expected level of competency after one year in the ICU was achieved only in half of the performances. A more evidence-based educational approach is needed. Multiple center simulation-based assessment showed feasibility and reliability as an evaluation method of competency. TRIAL REGISTRATION COBALIDATION. NCT04278976. (https://register. CLINICALTRIALS gov).
Collapse
Affiliation(s)
- Á Castellanos-Ortega
- Intensive Care Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | - M J Broch
- Intensive Care Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | | | - V Gómez-Tello
- Intensive Care Department, University Hospital Moncloa, Madrid, Spain
| | - M Valdivia
- Intensive Care Department, Hospital Puerta de Hierro-Majadahonda, Spain
| | - C Vicent
- Intensive Care Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | - I Madrid
- Intensive Care Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | - N Martinez
- Intensive Care Department, Hospital Puerta de Hierro-Majadahonda, Spain
| | - M J Párraga
- Intensive Care Department, Hospital Universitario Morales Meseguer, Murcia, Spain
| | - E Sancho
- Intensive Care Department, Hospital Universitario y Politécnico La Fe, Valencia, Spain
| | - M D C Fuentes-Dura
- Department of Methodology of the Behavioral Sciences, University of Valencia, Spain
| | - M D Sancerni-Beitia
- Department of Methodology of the Behavioral Sciences, University of Valencia, Spain
| | - R García-Ros
- Department of Developmental and Educational Psychology, University of Valencia, Spain.
| |
Collapse
|
2
|
Castellanos-Ortega Á, Broch M, Palacios-Castañeda D, Gómez-Tello V, Valdivia M, Vicent C, Madrid I, Martinez N, Párraga M, Sancho E, Fuentes-Dura M, Sancerni-Beitia M, García-Ros R. Competency assessment of residents of Intensive Care Medicine through a simulation-based objective structured clinical evaluation (OSCE). A multicenter observational study. Med Intensiva 2022. [DOI: 10.1016/j.medin.2022.01.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
|
3
|
Sinz E, Banerjee A, Steadman R, Shotwell MS, Slagle J, McIvor WR, Torsher L, Burden A, Cooper JB, DeMaria S, Levine AI, Park C, Gaba DM, Weinger MB, Boulet JR. Reliability of simulation-based assessment for practicing physicians: performance is context-specific. BMC MEDICAL EDUCATION 2021; 21:207. [PMID: 33845837 PMCID: PMC8042680 DOI: 10.1186/s12909-021-02617-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/18/2020] [Accepted: 03/15/2021] [Indexed: 06/12/2023]
Abstract
INTRODUCTION Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established. METHODS Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters. RESULTS The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability. CONCLUSIONS The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.
Collapse
Affiliation(s)
- Elizabeth Sinz
- Penn State University College of Medicine, Hershey, PA, 17033, USA.
| | - Arna Banerjee
- Vanderbilt University School of Medicine, Nashville, TN, USA
| | | | | | - Jason Slagle
- Center for Research and Innovation in Systems Safety, Vanderbilt University, Nashville, TN, USA
| | - William R McIvor
- WISER Simulation Center, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA
| | | | - Amanda Burden
- Cooper Medical School of Rowan University, Cooper University Hospital, Camden, NJ, USA
| | - Jeffrey B Cooper
- Harvard Medical School, Massachusetts General Hospital, Senior Fellow, Center for Medical Simulation, Boston, MA, USA
| | - Samuel DeMaria
- Icahn School of Medicine at the Mt Sinai Medical Center, New York, NY, USA
| | - Adam I Levine
- Icahn School of Medicine at the Mt Sinai Medical Center, New York, NY, USA
| | - Christine Park
- Department of Medical Education, Simulation and Integrative Learning Institute, University of Illinois College of Medicine, Chicago, IL, USA
| | - David M Gaba
- Stanford University and Staff Physician and Founder/Co-Director Simulation Center, VA Palo Alto, Palo Alto, CA, USA
| | - Matthew B Weinger
- Center for Research and Innovation in Systems Safety (CRISS), Institute for Medicine and Public Health, Vanderbilt University Medical Center, Vanderbilt University School of Medicine, Nashville, TN, USA
| | - John R Boulet
- Foundation for the Advancement of International Medical Education and Research (FAIMER), Philadelphia, PA, USA
| |
Collapse
|
4
|
Koster MA, Soffler M. Navigate the Challenges of Simulation for Assessment: A Faculty Development Workshop. MEDEDPORTAL : THE JOURNAL OF TEACHING AND LEARNING RESOURCES 2021; 17:11114. [PMID: 33768146 PMCID: PMC7970643 DOI: 10.15766/mep_2374-8265.11114] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/25/2020] [Accepted: 12/11/2020] [Indexed: 06/12/2023]
Abstract
INTRODUCTION Given barriers to learner assessment in the authentic clinical environment, simulated patient encounters are gaining attention as a valuable opportunity for competency assessment across the health professions. Simulation-based assessments offer advantages over traditional methods by providing realistic clinical scenarios through which a range of technical, analytical, and communication skills can be demonstrated. However, simulation for the purpose of assessment represents a paradigm shift with unique challenges, including preservation of a safe learning environment, standardization across learners, and application of valid assessment tools. Our goal was to create an interactive workshop to equip educators with the knowledge and skills needed to conduct assessments in a simulated environment. METHODS Participants engaged in a 90-minute workshop with large-group facilitated discussions and small-group activities for practical skill development. Facilitators guided attendees through a simulated grading exercise followed by in-depth analysis of three types of assessment tools. Participants designed a comprehensive simulation-based assessment encounter, including selection or creation of an assessment tool. RESULTS We have led two iterations of this workshop, including an in-person format at an international conference and a virtual format at our institution during the COVID-19 pandemic, with a total of 93 participants. Survey responses indicated strong overall ratings and impactfulness of the workshop. DISCUSSION Our workshop provides a practical, evidence-based framework to guide educators in the development of a simulation-based assessment program, including optimization of the environment, design of the simulated case, and utilization of meaningful, valid assessment tools.
Collapse
Affiliation(s)
- Megan A. Koster
- Instructor in Medicine, Harvard Medical School and Division of Pulmonary and Critical Care Medicine, Mount Auburn Hospital
| | - Morgan Soffler
- Instructor in Medicine, Harvard Medical School and Division of Pulmonary, Critical Care and Sleep Medicine, Beth Israel Deaconess Medical Center; Director of Simulation Research, Carl J. Shapiro Center for Education and Research, Beth Israel Deaconess Medical Center
| |
Collapse
|
5
|
Abstract
Introduction: Anesthesiology requires procedure fulfillment, problem, and real-time crisis resolution, problem, and complications forecast, among others; therefore, the evaluation of its learning should center around how students achieve competence rather than solely focusing on knowledge acquisition. Literature shows that despite the existence of numerous evaluation strategies, these are still underrated in most cases due to unawareness.
Objective: The present article aims to explain the process of competency-based anesthesiology assessment, in addition to suggesting a brief description of the learning domains evaluated, theories of knowledge, instruments, and assessment systems in the area; and finally, to show some of the most relevant results regarding assessment systems in Colombia.
Methodology: The results obtained in “Characteristics of the evaluation systems used by anesthesiology residency programs stakeholders in the educational process, a fact that motivated the publishing of this discussion around the topic of competency-based assessment in anesthesiology. Following a bibliography search with the keywords through PubMed, OVID, ERIC, DIALNET, and REDALYC, 110 articles were reviewed and 75 were established as relevant for the research’s theoretical framework.
Results and conclusion: Anesthesiology assessment should be conceived from the competency’s multidimensionality; it must be longitudinal and focused on the learning objectives.
Collapse
|
6
|
Key performance gaps of practicing anesthesiologists: how they contribute to hazards in anesthesiology and proposals for addressing them. Int Anesthesiol Clin 2020; 58:13-20. [PMID: 31800410 DOI: 10.1097/aia.0000000000000262] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
|
7
|
Palee P, Wongta N, Khwanngern K, Jitmun W, Choosri N. Serious Game for Teaching Undergraduate Medical Students in Cleft lip and Palate Treatment Protocol. Int J Med Inform 2020; 141:104166. [PMID: 32570197 DOI: 10.1016/j.ijmedinf.2020.104166] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2019] [Revised: 03/30/2020] [Accepted: 04/30/2020] [Indexed: 12/31/2022]
Abstract
OBJECTIVES To develop and evaluate a serious game to deliver knowledge about the cleft lip with or without cleft palate (CL/P) protocol used in Craniofacial Center, Chiang Mai University. METHODS The game "Cleft Island" was developed then extensively evaluated using different criteria, including the measured knowledge gained by participants, and gameplay experience. A group of 53 fourth and fifth-year medical students (male: 24, female: 29) were recruited to take part in an experiment. They were from the Faculty of Medicine, Chiang Mai University, and had just rotated to the Department of Surgery. Three evaluations were conducted: 1) to test whether the players had competently gained CL/P knowledge; 2) to evaluate the usability of the game according to the System Usability Scale (SUS); and 3) to evaluate the game experience in terms of a Game Experience Questionnaire (GEQ) test. RESULTS The results indicate a statistically significant improvement of medical students' knowledge after performing the Wilcoxon Signed-Ranks test (p < 0.5) between pre-test and post-test scores of the same medical student group; the acceptable average SUS score (M = 55.28) of the serious game; a moderate degree of experience of the GEQ components including positive affect (M = 2.64), competence (M = 2.49), and immersion (M = 2.21). CONCLUSIONS Cleft Island can be used as an effective supplementary instructional material, which has the potential to provide significant knowledge of CL/P treatment protocol for the players. As far as the authors are aware, this is the first study to implement and assess a serious game for training in CL/P protocol.
Collapse
Affiliation(s)
- Patison Palee
- College of Arts, Media, and Technology, Chiang Mai University, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand
| | - Noppon Wongta
- College of Arts, Media, and Technology, Chiang Mai University, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand
| | - Krit Khwanngern
- Faculty of Medicine, Chiang Mai University, 110 Inthawarorot Rd., Suthep, Muang, Chiang Mai, 50200, Thailand; Center of Data Analytics and Knowledge Synthesis for Healthcare, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand
| | - Waritsara Jitmun
- College of Arts, Media, and Technology, Chiang Mai University, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand
| | - Noppon Choosri
- College of Arts, Media, and Technology, Chiang Mai University, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand; Center of Data Analytics and Knowledge Synthesis for Healthcare, 239 Huaykaew Rd., Suthep, Muang, Chiang Mai, 50200, Thailand.
| |
Collapse
|
8
|
Gugiu MR, Cash R, Rivard M, Cotto J, Crowe RP, Panchal AR. Development and Validation of Content Domains for Paramedic Prehospital Performance Assessment: A Focus Group and Delphi Method Approach. PREHOSP EMERG CARE 2020; 25:196-204. [PMID: 32243208 DOI: 10.1080/10903127.2020.1750743] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
BACKGROUND The chaotic and complex nature of delivering patient care in the prehospital setting complicates the provision of real-time formative feedback to paramedic students. Although the use of simulations is widespread in emergency medical services (EMS) education, a high degree of variability precludes consistent performance assessment in EMS. Objectives: The objective of this study was to define and validate key domains required to evaluate paramedic prehospital performance. Methods: We conducted a two-phase study that combined focus group and Delphi methodology. Participants were purposefully selected to attain diverse panels regarding sex, race, ethnicity, professional roles, levels of education, geographical area, and experience as a paramedic and educator. In Phase I, a panel of 11 subject matter experts (SMEs) were tasked with identifying the essential domains to be evaluated in a paramedic performance assessment. In Phase II, another panel of 11 SMEs and a four-round modified Delphi method with 28 paramedic program directors were used to validate the domains identified in Phase I. Results: The first focus group identified and achieved consensus on five domains: (1) effective communication, (2) scene management, (3) patient assessment, (4) patient management, and (5) professional behavior. These domains were validated by the second focus group. The first round of the Delphi process generated 64 content domains, which were reduced to nine unique content domains via thematic analysis. These nine content domains fit well within the broader domains identified by the focus groups with one specific area, critical thinking and reasoning, being listed in two key areas based on the definitions of the focus group domains of patient assessment and patient management. Conclusion: The content domains identified in this study provide EMS educators a theoretical framework for designing the performance assessment of newly trained professionals in the prehospital setting.
Collapse
|
9
|
Binstadt ES, Dahms RA, Carlson AJ, Hegarty CB, Nelson JG. When the Learner Is the Expert: A Simulation-Based Curriculum for Emergency Medicine Faculty. West J Emerg Med 2019; 21:141-144. [PMID: 31913834 PMCID: PMC6948691 DOI: 10.5811/westjem.2019.11.45513] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2019] [Accepted: 11/03/2019] [Indexed: 11/18/2022] Open
Abstract
Emergency physicians supervise residents performing rare clinical procedures, but they infrequently perform those procedures independently. Simulation offers a forum to practice procedural skills, but simulation labs often target resident learners, and barriers exist to faculty as learners in simulation-based training. Simulation-based curricula focused on improving emergency medicine (EM) faculty’s rare procedure skills were not discovered on review of published literature. Our objective was to create a sustainable, simulation-based faculty education curriculum for rare procedural skills in EM. Between 2012 and 2019, most EM teaching faculty at a single, urban, Level 1 trauma center completed an annual two-hour simulation-based rare procedure lab with small-group learning and guided hands-on instruction, covering 30 different procedural education sessions for faculty learners. A questionnaire administered before and after each session assessed EM faculty physicians’ self-perceived ability to perform these rare procedures. Participants’ self-reported confidence in their performance improved for all procedures, regardless of prior procedural experience. Faculty participation was initially mandatory, but is now voluntary. Diverse strategies were used to address barriers in this learner group including eliciting learner feedback, offering continuing medical education credits, gradual roll-out of checklist assessments, and welcoming expertise of faculty leaders from EM and other specialties and professions. Participants perceived training to be most helpful for the most rarely-encountered clinical procedures. Similar curricula could be implemented with minimal risk at other institutions.
Collapse
Affiliation(s)
- Emily S Binstadt
- University of Minnesota, Regions Hospital Emergency Department, St. Paul, Minnesota
| | - Rachel A Dahms
- University of Minnesota, Regions Hospital Emergency Department, St. Paul, Minnesota
| | - Amanda J Carlson
- St. Mary's Medical Center Essentia Health, Department of Emergency Medicine, Duluth, Minnesota
| | - Cullen B Hegarty
- University of Minnesota, Regions Hospital Emergency Department, St. Paul, Minnesota
| | - Jessie G Nelson
- University of Minnesota, Regions Hospital Emergency Department, St. Paul, Minnesota
| |
Collapse
|
10
|
Fleming M, McMullen M, Beesley T, Egan R, Field S. Simulation-based evaluation of anaesthesia residents: optimising resource use in a competency-based assessment framework. BMJ SIMULATION & TECHNOLOGY ENHANCED LEARNING 2019; 6:339-343. [DOI: 10.1136/bmjstel-2019-000504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 11/15/2019] [Indexed: 11/03/2022]
Abstract
IntroductionSimulation training in anaesthesiology bridges the gap between theory and practice by allowing trainees to engage in high-stakes clinical training without jeopardising patient safety. However, implementing simulation-based assessments within an academic programme is highly resource intensive, and the optimal number of scenarios and faculty required for accurate competency-based assessment remains to be determined. Using a generalisability study methodology, we examine the structure of simulation-based assessment in regard to the minimal number of scenarios and faculty assessors required for optimal competency-based assessments.MethodsSeventeen anaesthesiology residents each performed four simulations which were assessed by two expert raters. Generalisability analysis (G-analysis) was used to estimate the extent of variance attributable to (1) the scenarios, (2) the assessors and (3) the participants. The D-coefficient and the G-coefficient were used to determine accuracy targets and to predict the impact of adjusting the number of scenarios or faculty assessors.ResultsWe showed that multivariate G-analysis can be used to estimate the number of simulations and raters required to optimise assessment. In this study, the optimal balance was obtained when four scenarios were assessed by two simulation experts.ConclusionSimulation-based assessment is becoming an increasingly important tool for assessing the competency of medical residents in conjunction with other assessment methods. G-analysis can be used to assist in planning for optimal resource use and cost-efficacy.
Collapse
|
11
|
King CR, Abraham J, Kannampallil TG, Fritz BA, Ben Abdallah A, Chen Y, Henrichs B, Politi M, Torres BA, Mickle A, Budelier TP, McKinnon S, Gregory S, Kheterpal S, Wildes T, Avidan MS. Protocol for the Effectiveness of an Anesthesiology Control Tower System in Improving Perioperative Quality Metrics and Clinical Outcomes: the TECTONICS randomized, pragmatic trial. F1000Res 2019; 8:2032. [PMID: 32201572 PMCID: PMC7076336 DOI: 10.12688/f1000research.21016.1] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 11/12/2019] [Indexed: 01/25/2023] Open
Abstract
Introduction: Perioperative morbidity is a public health priority, and surgical volume is increasing rapidly. With advances in technology, there is an opportunity to research the utility of a telemedicine-based control center for anesthesia clinicians that assess risk, diagnoses negative patient trajectories, and implements evidence-based practices. Objectives: The primary objective of this trial is to determine whether an anesthesiology control tower (ACT) prevents clinically relevant adverse postoperative outcomes including 30-day mortality, delirium, respiratory failure, and acute kidney injury. Secondary objectives are to determine whether the ACT improves perioperative quality of care metrics including management of temperature, mean arterial pressure, mean airway pressure with mechanical ventilation, blood glucose, anesthetic concentration, antibiotic redosing, and efficient fresh gas flow. Methods and analysis: We are conducting a single center, randomized, controlled, phase 3 pragmatic clinical trial. A total of 58 operating rooms are randomized daily to receive support from the ACT or not. All adults (eighteen years and older) undergoing surgical procedures in these operating rooms are included and followed until 30 days after their surgery. Clinicians in operating rooms randomized to ACT support receive decision support from clinicians in the ACT. In operating rooms randomized to no intervention, the current standard of anesthesia care is delivered. The intention-to-treat principle will be followed for all analyses. Differences between groups will be presented with 99% confidence intervals; p-values <0.005 will be reported as providing compelling evidence, and p-values between 0.05 and 0.005 will be reported as providing suggestive evidence. Registration: TECTONICS is registered on ClinicalTrials.gov, NCT03923699; registered on 23 April 2019.
Collapse
Affiliation(s)
- Christopher R. King
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Joanna Abraham
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
- Institute for Informatics, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Thomas G. Kannampallil
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
- Institute for Informatics, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Bradley A. Fritz
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Arbi Ben Abdallah
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Yixin Chen
- Department of Computer Science and Engineering, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Bernadette Henrichs
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Mary Politi
- Department of Surgery, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Brian A. Torres
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Angela Mickle
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Thaddeus P. Budelier
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Sherry McKinnon
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Stephen Gregory
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Sachin Kheterpal
- Department of Anesthesiology, University of Michigan, Ann Arbor, MI, 48109, USA
| | - Troy Wildes
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - Michael S. Avidan
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
| | - TECTONICS Research Group
- Department of Anesthesiology, Washington University in St Louis, St Louis, MO, 63110, USA
- Institute for Informatics, Washington University in St Louis, St Louis, MO, 63110, USA
- Department of Computer Science and Engineering, Washington University in St Louis, St Louis, MO, 63110, USA
- Department of Surgery, Washington University in St Louis, St Louis, MO, 63110, USA
- Department of Anesthesiology, University of Michigan, Ann Arbor, MI, 48109, USA
| |
Collapse
|
12
|
Everett TC, McKinnon RJ, Ng E, Kulkarni P, Borges BCR, Letal M, Fleming M, Bould MD. Simulation-based assessment in anesthesia: an international multicentre validation study. Can J Anaesth 2019; 66:1440-1449. [PMID: 31559541 DOI: 10.1007/s12630-019-01488-4] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2018] [Revised: 06/14/2019] [Accepted: 06/14/2019] [Indexed: 11/27/2022] Open
Abstract
PURPOSE Simulated clinical events provide a means to evaluate a practitioner's performance in a standardized manner for all candidates that are tested. We sought to provide evidence for the validity of simulation-based assessment tools in simulated pediatric anesthesia emergencies. METHODS Nine centres in two countries recruited subjects to participate in simulated operating room events. Participants ranged in anesthesia experience from junior residents to staff anesthesiologists. Performances were video recorded for review and scored by specially trained, blinded, expert raters. The rating tools consisted of scenario-specific checklists and a global rating scale that allowed the rater to make a judgement about the subject's performance, and by extension, preparedness for independent practice. The reliability of the tools was classified as "substantial" (intraclass correlation coefficients ranged from 0.84 to 0.96 for the checklists and from 0.85 to 0.94 for the global rating scale). RESULTS Three-hundred and ninety-one simulation encounters were analysed. Senior trainees and staff significantly out-performed junior trainees (P = 0.04 and P < 0.001 respectively). The effect size of grade (junior vs senior trainee vs staff) on performance was classified as "medium" (partial η2 = 0.06). Performance deficits were observed across all grades of anesthesiologist, particularly in two of the scenarios. CONCLUSIONS This study supports the validity of our simulation-based anesthesiologist assessment tools in several domains of validity. We also describe some residual challenges regarding the validity of our tools, some notes of caution in terms of the intended consequences of their use, and identify opportunities for further research.
Collapse
Affiliation(s)
- Tobias C Everett
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children, University of Toronto, 555 University Avenue, Toronto, ON, M5G 1X8, Canada.
| | - Ralph J McKinnon
- Department of Anesthesia, Royal Manchester Children's Hospital, Manchester, United Kingdom
| | - Elaine Ng
- Department of Anesthesia and Pain Medicine, The Hospital for Sick Children, University of Toronto, 555 University Avenue, Toronto, ON, M5G 1X8, Canada
| | - Pradeep Kulkarni
- Department of Anesthesia, Stollery Children's Hospital, University of Alberta, Edmonton, AB, Canada
| | - Bruno C R Borges
- Department of Anesthesia, McMaster Children's Hospital, McMaster University, Hamilton, ON, Canada
| | - Michael Letal
- Department of Anesthesia, Alberta Children's Hospital, University of Calgary, Calgary, AB, Canada
| | - Melinda Fleming
- Department of Anesthesia, Queens University, Kingston, ON, Canada
| | - M Dylan Bould
- Department of Anesthesia, Children's Hospital of Eastern Ontario, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
13
|
Abstract
OBJECTIVES Develop a standardized simulation method to assess clinical skills of ICU providers. DESIGN Simulation assessment. SETTING Simulation laboratory. SUBJECTS Residents, Critical Care Medicine fellows, acute care nurse practitioner students. INTERVENTIONS Performance scoring in scenarios from multiple Critical Care Medicine competency domains. MEASUREMENTS AND MAIN RESULTS Three-hundred eighty-four performances by 48 participants were scored using checklists (% correct) and holistic "global" ratings (1 [unprepared] to 9 [expert]). One-hundred eighty were scored by two raters. Mean checklist and global scores (± SD) ranged from 65.0% (± 16.3%) to 84.5% (± 17.3%) and 4.7 (± 1.4) to 7.2 (± 1.2). Checklist and global scores for Critical Care Medicine fellows and senior acute care nurse practitioner students (Experienced group, n = 26) were significantly higher than those for the Novice acute care nurse practitioner students (Novice group, n = 14) (75.6% ± 15.6% vs 68.8% ± 21.0% and 6.1 ± 1.6 vs 5.4 ± 1.5, respectively; p < 0.05). Residents (Intermediate group, n = 8) scored between the two (75.4% ± 18.3% and 5.7 ± 1.7). 38.5% of the Experienced group scored in the top quartile for mean global score, compared with 12.5% of the Intermediate and 7.1% of the Novice groups. Conversely, 50% of the Novice group scored in the lower quartile (< 5.3), compared with 37.5% of the Intermediate and 11.5% of the Experienced groups. Psychometric analyses yielded discrimination values greater than 0.3 for most scenarios and reliability for the eight-scenario assessments of 0.51 and 0.60, with interrater reliability of 0.71 and 0.75, for checklist and global scoring, respectively. CONCLUSIONS The simulation assessments yielded reasonably reliable measures of Critical Care Medicine decision-making skills. Despite a wide range of performance, those with more ICU training and experience performed better, providing evidence to support the validity of the scores. Simulation-based assessments may ultimately prove useful to determine readiness to assume decision-making roles in the ICU.
Collapse
|
14
|
Simulation Training for Residents Focused on Mechanical Ventilation: A Randomized Trial Using Mannequin-Based Versus Computer-Based Simulation. Simul Healthc 2018; 12:349-355. [PMID: 28825930 PMCID: PMC5768222 DOI: 10.1097/sih.0000000000000249] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Supplemental digital content is available in the text. Introduction Advances in knowledge regarding mechanical ventilation (MV), in particular lung-protective ventilation strategies, have been shown to reduce mortality. However, the translation of these advances in knowledge into better therapeutic performance in real-life clinical settings continues to lag. High-fidelity simulation with a mannequin allows students to interact in lifelike situations; this may be a valuable addition to traditional didactic teaching. The purpose of this study is to compare computer-based and mannequin-based approaches for training residents on MV. Methods This prospective randomized single-blind trial involved 50 residents. All participants attended the same didactic lecture on respiratory pathophysiology and were subsequently randomized into two groups: the mannequin group (n = 25) and the computer screen–based simulator group (n = 25). One week later, each underwent a training assessment using five different scenarios of acute respiratory failure of different etiologies. Later, both groups underwent further testing of patient management, using in situ high-fidelity simulation of a patient with acute respiratory distress syndrome. Results Baseline knowledge was not significantly different between the two groups (P = 0.72). Regarding the training assessment, no significant differences were detected between the groups. In the final assessment, the scores of only the mannequin group significantly improved between the training and final session in terms of either global rating score [3.0 (2.5–4.0) vs. 2.0 (2.0–3.0), P = 0.005] or percentage of key score (82% vs. 71%, P = 0.001). Conclusions Mannequin-based simulation has the potential to improve skills in managing MV.
Collapse
|
15
|
Butler L, Whitfill T, Wong AH, Gawel M, Crispino L, Auerbach M. The Impact of Telemedicine on Teamwork and Workload in Pediatric Resuscitation: A Simulation-Based, Randomized Controlled Study. Telemed J E Health 2018; 25:205-212. [PMID: 29957150 DOI: 10.1089/tmj.2018.0017] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND Telemedicine provides access to specialty care to critically ill patients from a geographic distance. The effects of using telemedicine on (1) teamwork and communication (TC), (2) task workload during resuscitation, and (3) the processes of critical care have not been well described. OBJECTIVES To evaluate the impact of telemedicine on (1) TC, (2) task workload during a resuscitation, and (3) the processes of critical care during a simulated pediatric resuscitation. METHODS Prospective single-center randomized trial. Teams of two physicians (senior and junior resident) and two standardized confederate nurses were randomized to either telemedicine (telepresent senior physician team leader) or usual care (both physicians in the room) during a simulated infant resuscitation. Simulations were video recorded and assessed for teamwork, workload, and processes of care using the Simulated Team Assessment Tool (STAT), the NASA Task Load Index (NASA-TLX) tool, and time between onset of ventricular fibrillation and defibrillation, respectively. RESULTS Twenty teams participated. There was no difference in teamwork between the groups (mean STAT score 72% vs. 69%; p = 0.383); however, there was a significantly greater workload in the telemedicine group (mean TLX score 56% vs. 48%, p = 0.020). Using linear regression, no difference was found in time-to-defibrillation between groups (p = 0.671), but higher teamwork scores predicted faster time to defibrillation (p = 0.020). CONCLUSIONS In this simulation-based study, a telepresent team leader was associated with increased team workload compared to usual care. However, no differences were noted in teamwork and processes of care metrics.
Collapse
Affiliation(s)
- Lucas Butler
- 1 Department of Emergency Medicine, University of Washington, Seattle, Washington.,2 Department of Pediatrics, Section of Emergency Medicine, Yale University, New Haven, Connecticut
| | - Travis Whitfill
- 2 Department of Pediatrics, Section of Emergency Medicine, Yale University, New Haven, Connecticut
| | - Ambrose H Wong
- 3 Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut
| | - Marcie Gawel
- 4 Department of Community Outreach, Yale-New Haven Hospital, New Haven, Connecticut
| | - Lauren Crispino
- 2 Department of Pediatrics, Section of Emergency Medicine, Yale University, New Haven, Connecticut
| | - Marc Auerbach
- 2 Department of Pediatrics, Section of Emergency Medicine, Yale University, New Haven, Connecticut.,3 Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut
| |
Collapse
|
16
|
Mejía V, Gonzalez C, Delfino AE, Altermatt FR, Corvetto MA. Acquiring skills in malignant hyperthermia crisis management: comparison of high-fidelity simulation versus computer-based case study. BRAZILIAN JOURNAL OF ANESTHESIOLOGY (ENGLISH EDITION) 2018. [PMID: 29631880 PMCID: PMC9391818 DOI: 10.1016/j.bjane.2018.01.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Introduction The primary purpose of this study was to compare the effect of high fidelity simulation versus a computer-based case solving self-study, in skills acquisition about malignant hyperthermia on first year anesthesiology residents. Methods After institutional ethical committee approval, 31 first year anesthesiology residents were enrolled in this prospective randomized single-blinded study. Participants were randomized to either a High Fidelity Simulation Scenario or a computer-based Case Study about malignant hyperthermia. After the intervention, all subjects’ performance in was assessed through a high fidelity simulation scenario using a previously validated assessment rubric. Additionally, knowledge tests and a satisfaction survey were applied. Finally, a semi-structured interview was done to assess self-perception of reasoning process and decision-making. Results 28 first year residents finished successfully the study. Resident's management skill scores were globally higher in High Fidelity Simulation versus Case Study, however they were significant in 4 of the 8 performance rubric elements: recognize signs and symptoms (p = 0.025), prioritization of initial actions of management (p = 0.003), recognize complications (p = 0.025) and communication (p = 0.025). Average scores from pre- and post-test knowledge questionnaires improved from 74% to 85% in the High Fidelity Simulation group, and decreased from 78% to 75% in the Case Study group (p = 0.032). Regarding the qualitative analysis, there was no difference in factors influencing the student's process of reasoning and decision-making with both teaching strategies. Conclusion Simulation-based training with a malignant hyperthermia high-fidelity scenario was superior to computer-based case study, improving knowledge and skills in malignant hyperthermia crisis management, with a very good satisfaction level in anesthesia residents.
Collapse
Affiliation(s)
- Vilma Mejía
- Universidad de Chile, Facultad de Medicina, Departamento de Educación en Ciencias de la Salud, Santiago, Chile
| | - Carlos Gonzalez
- Pontificia Universidad Católica de Chile, Facultad de Educación, Santiago, Chile
| | - Alejandro E Delfino
- Pontificia Universidad Católica de Chile, Facultad de Medicina, Departamento de Anestesiología, Santiago, Chile
| | - Fernando R Altermatt
- Pontificia Universidad Católica de Chile, Facultad de Medicina, Departamento de Anestesiología, Santiago, Chile
| | - Marcia A Corvetto
- Pontificia Universidad Católica de Chile, Facultad de Medicina, Departamento de Anestesiología, Santiago, Chile.
| |
Collapse
|
17
|
Mejía V, Gonzalez C, Delfino AE, Altermatt FR, Corvetto MA. Adquirir habilidades no manejo de crises de hipertermia maligna: comparação de simulação de alta‐fidelidade versus estudo de caso em computador. Braz J Anesthesiol 2018; 68:292-298. [DOI: 10.1016/j.bjan.2018.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2016] [Accepted: 01/06/2018] [Indexed: 10/17/2022] Open
|
18
|
Evaluation of a Simpler Tool to Assess Nontechnical Skills During Simulated Critical Events. Simul Healthc 2018; 12:69-75. [PMID: 28704284 DOI: 10.1097/sih.0000000000000199] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
INTRODUCTION Management of critical events requires teams to employ nontechnical skills (NTS), such as teamwork, communication, decision making, and vigilance. We sought to estimate the reliability and provide evidence for the validity of the ratings gathered using a new tool for assessing the NTS of anesthesia providers, the behaviorally anchored rating scale (BARS), and compare its scores with those of an established NTS tool, the Anaesthetists' Nontechnical Skills (ANTS) scale. METHODS Six previously trained raters (4 novices and 2 experts) reviewed and scored 18 recorded simulated pediatric crisis management scenarios using a modified ANTS and a BARS tool. Pearson correlation coefficients were calculated separately for the novice and expert raters, by scenario, and overall. RESULTS The intrarater reliability of the ANTS total score was 0.73 (expert, 0.57; novice, 0.84); for the BARS tool, it was 0.80 (expert, 0.79; novice, 0.81). The average interrater reliability of BARS scores (0.58) was better than ANTS scores (0.37), and the interrater reliabilities of scores from novices (0.69 BARS and 0.52 ANTS) were better than those obtained from experts (0.47 BARS and 0.21 ANTS) for both scoring instruments. The Pearson correlation between the ANTS and BARS total scores was 0.74. CONCLUSIONS Overall, reliability estimates were better for the BARS scores than the ANTS scores. For both measures, the intrarater and interrater reliability was better for novices compared with domain experts, suggesting that properly trained novices can reliably assess the NTS of anesthesia providers managing a simulated critical event. There was substantial correlation between the 2 scoring instruments, suggesting that the tools measured similar constructs. The BARS tool can be an alternative to the ANTS scale for the formative assessment of NTS of anesthesia providers.
Collapse
|
19
|
Blum RH, Muret-Wagstaff SL, Boulet JR, Cooper JB, Petrusa ER. Simulation-based Assessment to Reliably Identify Key Resident Performance Attributes. Anesthesiology 2018; 128:821-831. [DOI: 10.1097/aln.0000000000002091] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Abstract
Background
Obtaining reliable and valid information on resident performance is critical to patient safety and training program improvement. The goals were to characterize important anesthesia resident performance gaps that are not typically evaluated, and to further validate scores from a multiscenario simulation-based assessment.
Methods
Seven high-fidelity scenarios reflecting core anesthesiology skills were administered to 51 first-year residents (CA-1s) and 16 third-year residents (CA-3s) from three residency programs. Twenty trained attending anesthesiologists rated resident performances using a seven-point behaviorally anchored rating scale for five domains: (1) formulate a clear plan, (2) modify the plan under changing conditions, (3) communicate effectively, (4) identify performance improvement opportunities, and (5) recognize limits. A second rater assessed 10% of encounters. Scores and variances for each domain, each scenario, and the total were compared. Low domain ratings (1, 2) were examined in detail.
Results
Interrater agreement was 0.76; reliability of the seven-scenario assessment was r = 0.70. CA-3s had a significantly higher average total score (4.9 ± 1.1 vs. 4.6 ± 1.1, P = 0.01, effect size = 0.33). CA-3s significantly outscored CA-1s for five of seven scenarios and domains 1, 2, and 3. CA-1s had a significantly higher proportion of worrisome ratings than CA-3s (chi-square = 24.1, P < 0.01, effect size = 1.50). Ninety-eight percent of residents rated the simulations more educational than an average day in the operating room.
Conclusions
Sensitivity of the assessment to CA-1 versus CA-3 performance differences for most scenarios and domains supports validity. No differences, by experience level, were detected for two domains associated with reflective practice. Smaller score variances for CA-3s likely reflect a training effect; however, worrisome performance scores for both CA-1s and CA-3s suggest room for improvement.
Collapse
Affiliation(s)
- Richard H. Blum
- Department of Anesthesiology, Perioperative and Pain Medicine, Boston Children’s Hospital and Harvard Medical School, Boston, Massachusetts
- Center for Medical Simulation, Charlestown, Massachusetts
| | | | - John R. Boulet
- Foundation for Advancement of International Medical Education and Research, Philadelphia, Pennsylvania
| | - Jeffrey B. Cooper
- Center for Medical Simulation, Charlestown, Massachusetts
- Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Emil R. Petrusa
- Department of Surgery and Massachusetts General Hospital Learning Laboratory, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
20
|
Chauvin A, Truchot J, Bafeta A, Pateron D, Plaisance P, Yordanov Y. Randomized controlled trials of simulation-based interventions in Emergency Medicine: a methodological review. Intern Emerg Med 2018; 13:433-444. [PMID: 29147942 DOI: 10.1007/s11739-017-1770-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/31/2017] [Accepted: 11/10/2017] [Indexed: 11/27/2022]
Abstract
The number of trials assessing Simulation-Based Medical Education (SBME) interventions has rapidly expanded. Many studies show that potential flaws in design, conduct and reporting of randomized controlled trials (RCTs) can bias their results. We conducted a methodological review of RCTs assessing a SBME in Emergency Medicine (EM) and examined their methodological characteristics. We searched MEDLINE via PubMed for RCT that assessed a simulation intervention in EM, published in 6 general and internal medicine and in the top 10 EM journals. The Cochrane Collaboration risk of Bias tool was used to assess risk of bias, intervention reporting was evaluated based on the "template for intervention description and replication" checklist, and methodological quality was evaluated by the Medical Education Research Study Quality Instrument. Reports selection and data extraction was done by 2 independents researchers. From 1394 RCTs screened, 68 trials assessed a SBME intervention. They represent one quarter of our sample. Cardiopulmonary resuscitation (CPR) is the most frequent topic (81%). Random sequence generation and allocation concealment were performed correctly in 66 and 49% of trials. Blinding of participants and assessors was performed correctly in 19 and 68%. Risk of attrition bias was low in three-quarters of the studies (n = 51). Risk of selective reporting bias was unclear in nearly all studies. The mean MERQSI score was of 13.4/18.4% of the reports provided a description allowing the intervention replication. Trials assessing simulation represent one quarter of RCTs in EM. Their quality remains unclear, and reproducing the interventions appears challenging due to reporting issues.
Collapse
Affiliation(s)
- Anthony Chauvin
- Service d'Accueil des Urgences, Emergency Département, Hôpital Lariboisière, Assistance Publique-Hôpitaux de Paris, 2 Rue Ambroise Paré, 75010, Paris, France.
- Faculté de Médecine, Université Diderot, Paris, France.
- INSERM U1153, Statistic and Epidemiologic Research Center Sorbonne Paris Cité (CRESS), METHODS Team, Hotel-Dieu Hospital, Paris, France.
| | - Jennifer Truchot
- Service d'Accueil des Urgences, Emergency Département, Hôpital Lariboisière, Assistance Publique-Hôpitaux de Paris, 2 Rue Ambroise Paré, 75010, Paris, France
- Faculté de Médecine, Université Diderot, Paris, France
- Ilumens Simulation Department, Paris Descartes University, 45 rue des Saint Pères, 75006, Paris, France
| | - Aida Bafeta
- INSERM U1153, Statistic and Epidemiologic Research Center Sorbonne Paris Cité (CRESS), METHODS Team, Hotel-Dieu Hospital, Paris, France
| | - Dominique Pateron
- Sorbonne Universités, UPMC Paris Univ-06, Paris, France
- Service des Urgences-Hôpital Saint Antoine, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France
| | - Patrick Plaisance
- Service d'Accueil des Urgences, Emergency Département, Hôpital Lariboisière, Assistance Publique-Hôpitaux de Paris, 2 Rue Ambroise Paré, 75010, Paris, France
- Faculté de Médecine, Université Diderot, Paris, France
| | - Youri Yordanov
- INSERM U1153, Statistic and Epidemiologic Research Center Sorbonne Paris Cité (CRESS), METHODS Team, Hotel-Dieu Hospital, Paris, France
- Sorbonne Universités, UPMC Paris Univ-06, Paris, France
- Service des Urgences-Hôpital Saint Antoine, Assistance Publique-Hôpitaux de Paris (APHP), Paris, France
| |
Collapse
|
21
|
McEvoy MD, Thies KC, Einav S, Ruetzler K, Moitra VK, Nunnally ME, Banerjee A, Weinberg G, Gabrielli A, Maccioli GA, Dobson G, O’Connor MF. Cardiac Arrest in the Operating Room. Anesth Analg 2018; 126:889-903. [DOI: 10.1213/ane.0000000000002595] [Citation(s) in RCA: 29] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
|
22
|
Hart D, Bond W, Siegelman JN, Miller D, Cassara M, Barker L, Anders S, Ahn J, Huang H, Strother C, Hui J. Simulation for Assessment of Milestones in Emergency Medicine Residents. Acad Emerg Med 2018; 25:205-220. [PMID: 28833892 DOI: 10.1111/acem.13296] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2017] [Revised: 08/01/2017] [Accepted: 08/16/2017] [Indexed: 11/29/2022]
Abstract
OBJECTIVES All residency programs in the United States are required to report their residents' progress on the milestones to the Accreditation Council for Graduate Medical Education (ACGME) biannually. Since the development and institution of this competency-based assessment framework, residency programs have been attempting to ascertain the best ways to assess resident performance on these metrics. Simulation was recommended by the ACGME as one method of assessment for many of the milestone subcompetencies. We developed three simulation scenarios with scenario-specific milestone-based assessment tools. We aimed to gather validity evidence for this tool. METHODS We conducted a prospective observational study to investigate the validity evidence for three mannequin-based simulation scenarios for assessing individual residents on emergency medicine (EM) milestones. The subcompetencies (i.e., patient care [PC]1, PC2, PC3) included were identified via a modified Delphi technique using a group of experienced EM simulationists. The scenario-specific checklist (CL) items were designed based on the individual milestone items within each EM subcompetency chosen for assessment and reviewed by experienced EM simulationists. Two independent live raters who were EM faculty at the respective study sites scored each scenario following brief rater training. The inter-rater reliability (IRR) of the assessment tool was determined by measuring intraclass correlation coefficient (ICC) for the sum of the CL items as well as the global rating scales (GRSs) for each scenario. Comparing GRS and CL scores between various postgraduate year (PGY) levels was performed with analysis of variance. RESULTS Eight subcompetencies were chosen to assess with three simulation cases, using 118 subjects. Evidence of test content, internal structure, response process, and relations with other variables were found. The ICCs for the sum of the CL items and the GRSs were >0.8 for all cases, with one exception (clinical management GRS = 0.74 in sepsis case). The sum of CL items and GRSs (p < 0.05) discriminated between PGY levels on all cases. However, when the specific CL items were mapped back to milestones in various proficiency levels, the milestones in the higher proficiency levels (level 3 [L3] and 4 [L4]) did not often discriminate between various PGY levels. L3 milestone items discriminated between PGY levels on five of 12 occasions they were assessed, and L4 items discriminated only two of 12 times they were assessed. CONCLUSION Three simulation cases with scenario-specific assessment tools allowed evaluation of EM residents on proficiency L1 to L4 within eight of the EM milestone subcompetencies. Evidence of test content, internal structure, response process, and relations with other variables were found. Good to excellent IRR and the ability to discriminate between various PGY levels was found for both the sum of CL items and the GRSs. However, there was a lack of a positive relationship between advancing PGY level and the completion of higher-level milestone items (L3 and L4).
Collapse
Affiliation(s)
- Danielle Hart
- Emergency Medicine; Hennepin County Medical Center; University of Minnesota Medical School; Minneapolis MN
| | - William Bond
- Department of Emergency Medicine; Lehigh Valley Health Network; Allentown PA
| | | | - Daniel Miller
- Department of Emergency Medicine; University of Iowa; Iowa City IA
| | - Michael Cassara
- Department of Emergency Medicine; Hofstra University North Shore Long Island Jewish SOM; Northwell Health Center; Lake Success NY
| | - Lisa Barker
- Department of Emergency Medicine; University of Illinois College of Medicine at Peoria; Peoria IL
| | - Shilo Anders
- Department of Anesthesiology; Vanderbilt University; Nashville TN
| | - James Ahn
- Department of Emergency Medicine; University of Chicago; Chicago IL
| | - Hubert Huang
- Division of Education; Lehigh Valley Health Network; Allentown PA
| | | | - Joshua Hui
- Department of Emergency Medicine; Kaiser Permanente; Los Angeles Medical Center; Los Angeles CA
| |
Collapse
|
23
|
Subat A, Goldberg A, Demaria S, Katz D. The Utility of Simulation in the Management of Patients With Congenital Heart Disease: Past, Present, and Future. Semin Cardiothorac Vasc Anesth 2017; 22:81-90. [PMID: 29231093 DOI: 10.1177/1089253217746243] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Significant advancements have been made in the diagnosis and management of congenital heart disease (CHD). As a result, a higher percentage of these patients are surviving to adulthood. Despite this improvement in management, these patients remain at higher risk of morbidity and mortality, particularly in the perioperative setting. One new area of interest in these patients is the implementation of simulation-based medical education. Simulation has demonstrated various benefits across high-acuity scenarios encountered in the hospital. In CHD, simulation has been used in the training of pediatrics residents, assessment of intraoperative complications, echocardiography, and anatomic modeling with 3-dimensional printing. Here, we describe the current state of simulation in CHD, its role in training care providers for the management of this population, and future directions of CHD simulation.
Collapse
Affiliation(s)
- Ali Subat
- 1 Icahn School of Medicine at Mt Sinai, New York, NY, USA
| | | | - Samuel Demaria
- 1 Icahn School of Medicine at Mt Sinai, New York, NY, USA
| | - Daniel Katz
- 1 Icahn School of Medicine at Mt Sinai, New York, NY, USA
| |
Collapse
|
24
|
Juriga LL, Murray DJ, Boulet JR, Fehr JJ. Simulation and the diagnostic process: a pilot study of trauma and rapid response teams. Diagnosis (Berl) 2017. [DOI: 10.1515/dx-2017-0010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
AbstractBackground:Simulation is frequently used to recreate many of the crises encountered in patient care settings. Teams learn to manage these crises in an environment that maximizes their learning experiences and eliminates the potential for patient harm. By designing simulation scenarios that include conditions associated with diagnostic errors, teams can experience how their decisions can lead to errors. The purpose of this study was to assess how trauma teams (TrT) and pediatric rapid response teams (RRT) managed scenarios that included a diagnostic error.Methods:We developed four scenarios that would require TrT and pediatric RRT to manage an error in diagnosis. The two trauma scenarios (spinal cord injury and tracheobronchial tear) were designed to not respond to the heuristic management approach frequently used in trauma settings. The two pediatric scenarios (foreign body aspiration and coarctation of the aorta) had an incorrect diagnosis on admission. Two raters independently scored the scenarios using a rating system based on how teams managed the diagnostic process (search, establish and confirm a new diagnosis and initiate therapy based on the new diagnosis).Results:Twenty-one TrT and 17 pediatric rapid response managed 51 scenarios. All of the teams questioned the initial diagnosis. The teams were able to establish and confirm a new diagnosis in 49% of the scenarios (25 of 51). Only 23 (45%) teams changed their management of the patient based on the new diagnosis.Conclusions:Simulation can be used to recreate conditions that engage teams in the diagnostic process. In contrast to most instruction about diagnostic error, teams learn through realistic experiences and receive timely feedback about their decision-making skills. Based on the findings in this pilot study, the majority of teams would benefit from an education intervention designed to improve their diagnostic skills.
Collapse
|
25
|
Rescuing the Clinical Breast Examination: Advances in Classifying Technique and Assessing Physician Competency. Ann Surg 2017; 266:1069-1074. [PMID: 27655241 DOI: 10.1097/sla.0000000000002024] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
OBJECTIVE Develop new performance evaluation standards for the clinical breast examination (CBE). SUMMARY BACKGROUND DATA There are several, technical aspects of a proper CBE. Our recent work discovered a significant, linear relationship between palpation force and CBE accuracy. This article investigates the relationship between other technical aspects of the CBE and accuracy. METHODS This performance assessment study involved data collection from physicians (n = 553) attending 3 different clinical meetings between 2013 and 2014: American Society of Breast Surgeons, American Academy of Family Physicians, and American College of Obstetricians and Gynecologists. Four, previously validated, sensor-enabled breast models were used for clinical skills assessment. Models A and B had solitary, superficial, 2 cm and 1 cm soft masses, respectively. Models C and D had solitary, deep, 2 cm hard and moderately firm masses, respectively. Finger movements (search technique) from 1137 CBE video recordings were independently classified by 2 observers. Final classifications were compared with CBE accuracy. RESULTS Accuracy rates were model A = 99.6%, model B = 89.7%, model C = 75%, and model D = 60%. Final classification categories for search technique included rubbing movement, vertical movement, piano fingers, and other. Interrater reliability was (k = 0.79). Rubbing movement was 4 times more likely to yield an accurate assessment (odds ratio 3.81, P < 0.001) compared with vertical movement and piano fingers. Piano fingers had the highest failure rate (36.5%). Regression analysis of search pattern, search technique, palpation force, examination time, and 6 demographic variables, revealed that search technique independently and significantly affected CBE accuracy (P < 0.001). CONCLUSIONS Our results support measurement and classification of CBE techniques and provide the foundation for a new paradigm in teaching and assessing hands-on clinical skills. The newly described piano fingers palpation technique was noted to have unusually high failure rates. Medical educators should be aware of the potential differences in effectiveness for various CBE techniques.
Collapse
|
26
|
Krupat E, Wormwood J, Schwartzstein RM, Richards JB. Avoiding premature closure and reaching diagnostic accuracy: some key predictive factors. MEDICAL EDUCATION 2017; 51:1127-1137. [PMID: 28857266 DOI: 10.1111/medu.13382] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/21/2016] [Revised: 03/03/2017] [Accepted: 05/24/2017] [Indexed: 06/07/2023]
Abstract
CONTEXT Early studies suggested that experienced clinicians simply generate more accurate diagnoses than those less experienced. However, more recent studies indicate that experienced clinicians may be subject to biases in formulating and confirming hypotheses that lead to inaccuracy. OBJECTIVES The goal of this study was to identify factors associated with the ability to process information in ways that overcome premature closure and result in accurate diagnosis using a set of vignettes in which inconsistent information was introduced midway. METHODS Seventy-five participants (25 Year 3 medical students, 25 internal medicine residents in their second year of residency and 25 internal medicine faculty) were recruited to solve each of four complex clinical vignettes. In each vignette, the first four rounds of information pointed toward a narrowing range of diagnostic possibilities, but patient information presented in and after the fifth round was inconsistent with prior findings. In addition to accuracy, outcome measures were length of differential diagnosis, certainty of diagnosis, persistence in data collection and tendency to switch diagnoses. RESULTS There were no significant differences in diagnostic accuracy across the three groups, each of which differed in level of training. However, across experience levels, diagnostic accuracy was associated with the mean number of items in the differential, tendency to persist (e.g. to request a greater number of rounds of information), and openness to switch (e.g. to change the most likely diagnosis on receipt of disconfirming information). CONCLUSIONS Level of training (i.e. clinical experience) was not associated with accuracy on this task. As faculty clinicians certainly have more knowledge than their junior counterparts, it is important to identify ways in which cognitive factors can lead to more or less persistence and openness, and to teach clinicians how to overcome tendencies associated with error.
Collapse
Affiliation(s)
- Edward Krupat
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Jolie Wormwood
- Department of Psychology, Northeastern University, Boston, Massachusetts, USA
| | | | - Jeremy B Richards
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| |
Collapse
|
27
|
Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists. Anesthesiology 2017; 127:475-489. [DOI: 10.1097/aln.0000000000001739] [Citation(s) in RCA: 48] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Abstract
Background
We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods.
Methods
A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant’s technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist.
Results
Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance.
Conclusions
Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.
Collapse
|
28
|
Geeraerts T, Roulleau P, Cheisson G, Marhar F, Aidan K, Lallali K, Leguen M, Schnell D, Trabold F, Fauquet-Alekhine P, Duranteau J, Benhamou D. Physiological and self-assessed psychological stress induced by a high fidelity simulation course among third year anesthesia and critical care residents: An observational study. Anaesth Crit Care Pain Med 2017. [PMID: 28648752 DOI: 10.1016/j.accpm.2017.06.002] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
INTRODUCTION The use of high fidelity simulators in Medicine can improve knowledge, behaviour and practice but may be associated with significant stress. Our objective was to measure physiological and psychological self-assessed intensity of stress before and after a planned simulation training session among third year anaesthesia and critical care residents. METHODS A convenience sample of 27 residents participating in a simulation training course was studied. Stress was evaluated by self-assessment using a numerical scale and by salivary amylase concentration before and after the session. Technical and non-technical (using the Aberdeen Anaesthetists' Non Technical Skills scale) performances were assessed through videotapes analysis. RESULTS The median stress score was 5 (2-8) before and 7 (2-10) after the simulation session (P<0.001). For 48% of residents studied, the stress score after the session was superior or equal to 8/10. Salivary amylase concentration increased significantly after the session compared to before the session, respectively (1,250,440±1,216,667 vs. 727,260±603,787IU/L, P=0.008). There was no significant correlation between stress parameters and non-technical performance. DISCUSSION Simulation-induced stress, as measured by self-assessment and biological parameter, is high before the session and increases significantly during the course. While this stress did not seem to impact performance negatively, it should be taken into account.
Collapse
Affiliation(s)
- Thomas Geeraerts
- Département d'anesthésie-réanimation, Institut Toulousain de Simulation en Santé (ItSims), CHU de Toulouse, Université Paul-Sabatier, 31059 Toulouse, France.
| | - Philippe Roulleau
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Gaëlle Cheisson
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Fouad Marhar
- Département d'anesthésie-réanimation, Institut Toulousain de Simulation en Santé (ItSims), CHU de Toulouse, Université Paul-Sabatier, 31059 Toulouse, France
| | - Karl Aidan
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Karim Lallali
- Service de biochimie, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Morgan Leguen
- Département d'anesthésie-réanimation, hôpital Foch, université Paris Île-de-France Ouest, 92150 Suresnes, France
| | - David Schnell
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Fabien Trabold
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | | | - Jacques Duranteau
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| | - Dan Benhamou
- Département d'anesthésie-réanimation, hôpital Bicêtre, université Paris-Sud, AP-HP, 94275 Le Kremlin-Bicêtre, France
| |
Collapse
|
29
|
Karam VY, Barakat H, Aouad M, Harris I, Park YS, Youssef N, Boulet JJ, Tekian A. Effect of a simulation-based workshop on breaking bad news for anesthesiology residents: an intervention study. BMC Anesthesiol 2017; 17:77. [PMID: 28615002 PMCID: PMC5471713 DOI: 10.1186/s12871-017-0374-7] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2017] [Accepted: 06/07/2017] [Indexed: 11/17/2022] Open
Abstract
Background Breaking bad news (BBN) to patients and their relatives is a complex and stressful task. The ideal structure, training methods and assessment instruments best used to teach and assess BBN for anesthesiology residents remain unclear. The purpose of this study is to evaluate the effectiveness of an education intervention for BBN based on immersive experiences with a high fidelity simulator and role-play with standardized patients (SPs). A secondary purpose is to gather validity evidence to support the use of a GRIEV_ING instrument to assess BBN skills. Methods The communication skills for BBN of 16 residents were assessed via videotaped SP encounters at baseline and immediately post-intervention. Residents’ perceptions about their ability and comfort for BBN were collected using pre and post workshop surveys. Results Posttest scores were significantly higher than the pretest scores for the GRIEV_ING checklist, as well as on the communication global rating. The GRIEV_ING checklist had acceptable inter-rater and internal-consistency reliabilities. Performance was not related to years of training, or previous BBN experience. Conclusion Anesthesiology residents’ communication skills when BBN in relation to a critical incident may be improved with educational interventions based on immersive experiences with a high fidelity simulator and role-play with SPs. Electronic supplementary material The online version of this article (doi:10.1186/s12871-017-0374-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Vanda Yazbeck Karam
- Lebanese American University School of Medicine, P.O. Box: 36, Beirut, Lebanon.
| | - Hanane Barakat
- Lebanese American University School of Medicine, P.O. Box: 36, Beirut, Lebanon
| | - Marie Aouad
- American University of Beirut Medical Center, Beirut, Lebanon
| | - Ilene Harris
- Department of Medical Education, University of Illinois at Chicago, Illinois, IL, USA
| | - Yoon Soo Park
- Department of Medical Education, University of Illinois at Chicago, Illinois, IL, USA
| | - Nazih Youssef
- Lebanese American University School of Medicine, P.O. Box: 36, Beirut, Lebanon
| | - John Jack Boulet
- Foundation for Advancement of International Medical Education and Research, Philadelphia, PA, USA
| | - Ara Tekian
- Department of Medical Education, University of Illinois at Chicago, Illinois, IL, USA
| |
Collapse
|
30
|
DeMaria S, Levine A, Petrou P, Feldman D, Kischak P, Burden A, Goldberg A. Performance gaps and improvement plans from a 5-hospital simulation programme for anaesthesiology providers: a retrospective study. BMJ SIMULATION & TECHNOLOGY ENHANCED LEARNING 2017; 3:37-42. [DOI: 10.1136/bmjstel-2016-000163] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 01/21/2017] [Indexed: 01/08/2023]
Abstract
BackgroundSimulation is increasingly employed in healthcare provider education, but usage as a means of identifying system-wide practitioner gaps has been limited. We sought to determine whether practice gaps could be identified, and if meaningful improvement plans could result from a simulation course for anaesthesiology providers.MethodsOver a 2-year cycle, 288 anaesthesiologists and 67 certified registered nurse anaesthetists (CRNAs) participated in a 3.5 hour, malpractice insurer-mandated simulation course, encountering 4 scenarios. 5 anaesthesiology departments within 3 urban academic healthcare systems were represented. A real-time rater scored each individual on 12 critical performance items (CPIs) representing learning objectives for a given scenario. Participants completed a course satisfaction survey, a 1-month postcourse practice improvement plan (PIP) and a 6-month follow-up survey.ResultsAll recorded course data were retrospectively reviewed. Course satisfaction was generally positive (88–97% positive rating by item). 4231 individual CPIs were recorded (of a possible 4260 rateable), with a majority of participants demonstrating remediable gaps in medical/technical and non-technical skills (97% of groups had at least one instance of a remediable gap in communication/non-technical skills during at least one of the scenarios). 6 months following the course, 91% of respondents reported successfully implementing 1 or more of their PIPs. Improvements in equipment/environmental resources or personal knowledge domains were most often successful, and several individual reports demonstrated a positive impact on actual practice.ConclusionsThis professional liability insurer-initiated simulation course for 5 anaesthesiology departments was feasible to deliver and well received. Practice gaps were identified during the course and remediation of gaps, and/or application of new knowledge, skills and resources was reported by participants.
Collapse
|
31
|
Ahmed A, Khan FA, Ismail S. Reliability and validity of a tool to assess airway management skills in anesthesia trainees. J Anaesthesiol Clin Pharmacol 2016; 32:333-8. [PMID: 27625481 PMCID: PMC5009839 DOI: 10.4103/0970-9185.168171] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
BACKGROUND AND AIMS Gaining expertise in procedural skills is essential for achieving clinical competence during anesthesia training. Supervisors have the important responsibility of deciding when the trainee can be allowed to perform various procedures without direct supervision while ensuring patient safety. This requires robust and reliable assessment techniques. Airway management with bag-mask ventilation and tracheal intubation are routinely performed by anesthesia trainees at induction of anesthesia and to save lives during a cardiorespiratory arrest. The purpose of this study was to evaluate the construct validity, and inter-rater and test-retest reliability of a tool designed to assess competence in bag-mask ventilation followed by tracheal intubation in anesthesia trainees. MATERIAL AND METHODS Informed consent was obtained from all participants. Tracheal intubation and bag-mask ventilation skills in 10 junior and 10 senior anesthesia trainees were assessed by two investigators on two occasions at a 3-4 weeks interval, using a procedure-specific assessment tool. RESULTS Average kappa value for inter-rater reliability was 0.91 and 0.99 for the first and second assessments, respectively, with an average agreement of 95%. The average agreement for test-retest reliability was 82% with a kappa value of 0.39. Senior trainees obtained higher scores compared to junior trainees in all areas of assessment, with a significant difference for patient positioning, preoxygenation, and laryngoscopy technique, depicting good construct validity. CONCLUSION The tool designed to assess bag-mask ventilation and tracheal intubation skills in anesthesia trainees demonstrated excellent inter-rater reliability, fair test-retest reliability, and good construct validity. The authors recommend its use for formative and summative assessment of junior anesthesia trainees.
Collapse
Affiliation(s)
- Aliya Ahmed
- Department of Anesthesiology, Aga Khan University, Karachi, Pakistan
| | - Fauzia Anis Khan
- Department of Anesthesiology, Aga Khan University, Karachi, Pakistan
| | - Samina Ismail
- Department of Anesthesiology, Aga Khan University, Karachi, Pakistan
| |
Collapse
|
32
|
Oermann MH, Kardong-Edgren S, Rizzolo MA. Summative Simulated-Based Assessment in Nursing Programs. J Nurs Educ 2016; 55:323-8. [DOI: 10.3928/01484834-20160516-04] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2015] [Accepted: 04/05/2016] [Indexed: 11/20/2022]
|
33
|
A Smartphone-based Decision Support Tool Improves Test Performance Concerning Application of the Guidelines for Managing Regional Anesthesia in the Patient Receiving Antithrombotic or Thrombolytic Therapy. Anesthesiology 2016; 124:186-98. [PMID: 26513023 DOI: 10.1097/aln.0000000000000885] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND The American Society of Regional Anesthesia and Pain Medicine (ASRA) consensus statement on regional anesthesia in the patient receiving antithrombotic or thrombolytic therapy is the standard for evaluation and management of these patients. The authors hypothesized that an electronic decision support tool (eDST) would improve test performance compared with native physician behavior concerning the application of this guideline. METHODS Anesthesiology trainees and faculty at 8 institutions participated in a prospective, randomized trial in which they completed a 20-question test involving clinical scenarios related to the ASRA guidelines. The eDST group completed the test using an iOS app programmed to contain decision logic and content of the ASRA guidelines. The control group completed the test by using any resource in addition to the app. A generalized linear mixed-effects model was used to examine the effect of the intervention. RESULTS After obtaining institutional review board's approval and informed consent, 259 participants were enrolled and randomized (eDST = 122; control = 137). The mean score was 92.4 ± 6.6% in the eDST group and 68.0 ± 15.8% in the control group (P < 0.001). eDST use increased the odds of selecting correct answers (7.8; 95% CI, 5.7 to 10.7). Most control group participants (63%) used some cognitive aid during the test, and they scored higher than those who tested from memory alone (76 ± 15% vs. 57 ± 18%, P < 0.001). There was no difference in time to completion of the test (P = 0.15) and no effect of training level (P = 0.56). CONCLUSIONS eDST use improved application of the ASRA guidelines compared with the native clinician behavior in a testing environment.
Collapse
|
34
|
Ryall T, Judd BK, Gordon CJ. Simulation-based assessments in health professional education: a systematic review. J Multidiscip Healthc 2016; 9:69-82. [PMID: 26955280 PMCID: PMC4768888 DOI: 10.2147/jmdh.s92695] [Citation(s) in RCA: 51] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/30/2022] Open
Abstract
INTRODUCTION The use of simulation in health professional education has increased rapidly over the past 2 decades. While simulation has predominantly been used to train health professionals and students for a variety of clinically related situations, there is an increasing trend to use simulation as an assessment tool, especially for the development of technical-based skills required during clinical practice. However, there is a lack of evidence about the effectiveness of using simulation for the assessment of competency. Therefore, the aim of this systematic review was to examine simulation as an assessment tool of technical skills across health professional education. METHODS A systematic review of Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), Medical Literature Analysis and Retrieval System Online (Medline), and Web of Science databases was used to identify research studies published in English between 2000 and 2015 reporting on measures of validity, reliability, or feasibility of simulation as an assessment tool. The McMasters Critical Review for quantitative studies was used to determine methodological value on all full-text reviewed articles. Simulation techniques using human patient simulators, standardized patients, task trainers, and virtual reality were included. RESULTS A total of 1,064 articles were identified using search criteria, and 67 full-text articles were screened for eligibility. Twenty-one articles were included in the final review. The findings indicated that simulation was more robust when used as an assessment in combination with other assessment tools and when more than one simulation scenario was used. Limitations of the research papers included small participant numbers, poor methodological quality, and predominance of studies from medicine, which preclude any definite conclusions. CONCLUSION Simulation has now been embedded across a range of health professional education and it appears that simulation-based assessments can be used effectively. However, the effectiveness as a stand-alone assessment tool requires further research.
Collapse
Affiliation(s)
- Tayne Ryall
- Physiotherapy Department, Canberra Hospital, ACT Health, Canberra, ACT, Australia
| | - Belinda K Judd
- Faculty of Health Sciences, The University of Sydney, Sydney, NSW, Australia; Sydney Nursing School, The University of Sydney, Sydney, NSW, Australia
| | | |
Collapse
|
35
|
Hilton G, Daniels K, Goldhaber-Fiebert SN, Lipman S, Carvalho B, Butwick A. Checklists and multidisciplinary team performance during simulated obstetric hemorrhage. Int J Obstet Anesth 2016; 25:9-16. [PMID: 26421705 PMCID: PMC4727983 DOI: 10.1016/j.ijoa.2015.08.011] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2015] [Revised: 08/11/2015] [Accepted: 08/14/2015] [Indexed: 10/23/2022]
Abstract
BACKGROUND Checklists can optimize team performance during medical crises. However, there has been limited examination of checklist use during obstetric crises. In this simulation study we exposed multidisciplinary teams to checklist training to evaluate checklist use and team performance during a severe postpartum hemorrhage. METHODS Fourteen multidisciplinary teams participated in a postpartum hemorrhage simulation occurring after vaginal delivery. Before participating, each team received checklist training. The primary study outcome was whether each team used the checklist during the simulation. Secondary outcomes were the times taken to activate our institution-specific massive transfusion protocol and commence red blood cell transfusion, and whether a designated checklist reader was used. RESULTS The majority of teams (12/14 (86%)) used the checklist. Red blood cell transfusion was administered by all teams. The median [IQR] times taken to activate the massive transfusion protocol and transfuse red blood cells were 5min 14s [3:23-6:43] and 14min 40s [12:56-17:28], respectively. A designated checklist reader was used by 7/12 (58%) teams that used the checklist. Among teams that used a checklist with versus without a designated reader, we observed no differences in the times to activate the massive transfusion protocol or to commence red blood cell transfusion (P>0.05). CONCLUSIONS Although checklist training was effective in promoting checklist use, multidisciplinary teams varied in their scope of checklist use during a postpartum hemorrhage simulation. Future studies are required to determine whether structured checklist training can result in more standardized checklist use during a postpartum hemorrhage.
Collapse
Affiliation(s)
- G Hilton
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, USA.
| | - K Daniels
- Department of Obstetrics and Gynecology, Stanford University School of Medicine, Stanford, CA, USA
| | - S N Goldhaber-Fiebert
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | - S Lipman
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | - B Carvalho
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | - A Butwick
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, CA, USA
| |
Collapse
|
36
|
Dennis DM, Sainsbury DA, Redwood TM, Ng L, Furness A. Introducing Simulation Based Learning Activities to Physiotherapy Course Curricula. ACTA ACUST UNITED AC 2016. [DOI: 10.4236/ce.2016.76092] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
37
|
Practice improvements based on participation in simulation for the maintenance of certification in anesthesiology program. Anesthesiology 2015; 122:1154-69. [PMID: 25985025 DOI: 10.1097/aln.0000000000000613] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
BACKGROUND This study describes anesthesiologists' practice improvements undertaken during the first 3 yr of simulation activities for the Maintenance of Certification in Anesthesiology Program. METHODS A stratified sampling of 3 yr (2010-2012) of participants' practice improvement plans was coded, categorized, and analyzed. RESULTS Using the sampling scheme, 634 of 1,275 participants in Maintenance of Certification in Anesthesiology Program simulation courses were evaluated from the following practice settings: 41% (262) academic, 54% (339) community, and 5% (33) military/other. A total of 1,982 plans were analyzed for completion, target audience, and topic. On follow-up, 79% (1,558) were fully completed, 16% (310) were partially completed, and 6% (114) were not completed within the 90-day reporting period. Plans targeted the reporting individual (89% of plans) and others (78% of plans): anesthesia providers (50%), non-anesthesia physicians (16%), and non-anesthesia non-physician providers (26%). From the plans, 2,453 improvements were categorized as work environment or systems changes (33% of improvements), teamwork skills (30%), personal knowledge (29%), handoff (4%), procedural skills (3%), or patient communication (1%). The median word count was 63 (interquartile range, 30 to 126) for each participant's combined plans and 147 (interquartile range, 52 to 257) for improvement follow-up reports. CONCLUSIONS After making a commitment to change, 94% of anesthesiologists participating in a Maintenance of Certification in Anesthesiology Program simulation course successfully implemented some or all of their planned practice improvements. This compares favorably to rates in other studies. Simulation experiences stimulate active learning and motivate personal and collaborative practice improvement changes. Further evaluation will assess the impact of the improvements and further refine the program.
Collapse
|
38
|
Aho JM, Thiels CA, AlJamal YN, Ruparel RK, Rowse PG, Heller SF, Farley DR. Every surgical resident should know how to perform a cricothyrotomy: an inexpensive cricothyrotomy task trainer for teaching and assessing surgical trainees. JOURNAL OF SURGICAL EDUCATION 2015; 72:658-661. [PMID: 25703738 DOI: 10.1016/j.jsurg.2014.12.012] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/26/2014] [Accepted: 12/26/2014] [Indexed: 06/04/2023]
Abstract
OBJECTIVE Emergency cricothyrotomy is a rare but potentially lifesaving procedure. Training opportunities for surgical residents to learn this skill are limited, and many graduating residents have never performed one during their training. We aimed to develop and validate a novel and inexpensive cricothyrotomy task trainer that can be constructed from household items. DESIGN A model was constructed using a toilet paper roll (trachea and larynx), Styrofoam (soft tissue), cardboard (thyroid cartilage), zip tie (cricoid), and fabric (skin). Participants were asked to complete a simulated cricothyrotomy procedure using the model. They were then evaluated using a 10-point checklist (5 points total) devised by 6 general surgeons. Participants were also asked to complete an anonymous survey rating the educational value and the degree of enjoyment regarding the model. SETTING A tertiary care teaching hospital. PARTICIPANTS A total of 54 students and general surgery residents (11 medical students, 32 interns, and 11 postgraduate year 3 residents). RESULTS All 54 participants completed the training and assessment. The scores ranged from 0 to 5. The mean (range) scores were 1.8 (1-4) for medical students, 3.5 (1-5) for junior residents, and 4.9 (4-5) for senior-level residents. Medical students were significantly outperformed by junior- and senior-level residents (p < 0.001). Trainees felt that the model was educational (4.5) and enjoyable (4.0). CONCLUSIONS A low-fidelity, low-cost cricothyrotomy simulator distinguished the performance of emergency cricothyrotomy between medical students and junior- and senior-level general surgery residents. This task trainer may be ideally suited to providing basic skills to all physicians in training, especially in settings with limited resources and clinical opportunities.
Collapse
Affiliation(s)
| | | | | | - Raaj K Ruparel
- Department of Surgery, Mayo Clinic, Rochester, Minnesota
| | | | | | - David R Farley
- Department of Surgery, Mayo Clinic, Rochester, Minnesota.
| |
Collapse
|
39
|
Kurrek MM, Morgan P, Howard S, Kranke P, Calhoun A, Hui J, Kiss A. Simulation as a new tool to establish benchmark outcome measures in obstetrics. PLoS One 2015; 10:e0131064. [PMID: 26107661 PMCID: PMC4480859 DOI: 10.1371/journal.pone.0131064] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2015] [Accepted: 05/28/2015] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND There are not enough clinical data from rare critical events to calculate statistics to decide if the management of actual events might be below what could reasonably be expected (i.e. was an outlier). OBJECTIVES In this project we used simulation to describe the distribution of management times as an approach to decide if the management of a simulated obstetrical crisis scenario could be considered an outlier. DESIGN Twelve obstetrical teams managed 4 scenarios that were previously developed. Relevant outcome variables were defined by expert consensus. The distribution of the response times from the teams who performed the respective intervention was graphically displayed and median and quartiles calculated using rank order statistics. RESULTS Only 7 of the 12 teams performed chest compressions during the arrest following the 'cannot intubate/cannot ventilate' scenario. All other outcome measures were performed by at least 11 of the 12 teams. Calculation of medians and quartiles with 95% CI was possible for all outcomes. Confidence intervals, given the small sample size, were large. CONCLUSION We demonstrated the use of simulation to calculate quantiles for management times of critical event. This approach could assist in deciding if a given performance could be considered normal and also point to aspects of care that seem to pose particular challenges as evidenced by a large number of teams not performing the expected maneuver. However sufficiently large sample sizes (i.e. from a national data base) will be required to calculate acceptable confidence intervals and to establish actual tolerance limits.
Collapse
Affiliation(s)
- Matt M. Kurrek
- Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada
- * E-mail:
| | - Pamela Morgan
- Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada
| | - Steven Howard
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Palo Alto, California, United States of America
| | - Peter Kranke
- Department of Anesthesia and Critical Care, University of Wuerzburg, Wuerzburg, Germany
| | - Aaron Calhoun
- Division of Critical Care, Department of Pediatrics, University of Louisville, Louisville, Kentucky, United States of America
| | - Joshua Hui
- Department of Emergency Medicine, University of California Los Angeles and Olive View-UCLA Medical Center, Los Angeles, California, United States of America
| | - Alex Kiss
- Department of Research Design and Biostatistics, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
40
|
Validity and reliability assessment of detailed scoring checklists for use during perioperative emergency simulation training. Simul Healthc 2015; 9:295-303. [PMID: 25188486 DOI: 10.1097/sih.0000000000000048] [Citation(s) in RCA: 21] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
INTRODUCTION Few valid and reliable grading checklists have been published for the evaluation of performance during simulated high-stakes perioperative event management. As such, the purposes of this study were to construct valid scoring checklists for a variety of perioperative emergencies and to determine the reliability of scores produced by these checklists during continuous video review. METHODS A group of anesthesiologists, intensivists, and educators created a set of simulation grading checklists for the assessment of the following scenarios: severe anaphylaxis, cerebrovascular accident, hyperkalemic arrest, malignant hyperthermia, and acute coronary syndrome. Checklist items were coded as critical or noncritical. Nonexpert raters evaluated 10 simulation videos in a random order, with each video being graded 4 times. A group of faculty experts also graded the videos to create a reference standard to which nonexpert ratings were compared. P < 0.05 was considered significant. RESULTS Team leaders in the simulation videos were scored by the expert panel as having performed 56.5% of all items on the checklist (range, 43.8%-84.0%), and 67.2% of the critical items (range, 30.0%-100%). Nonexpert raters agreed with the expert assessment 89.6% of the time (95% confidence interval, 87.2%-91.6%). No learning curve development was found with repetitive video assessment or checklist use. The κ values comparing nonexpert rater assessments to the reference standard averaged 0.76 (95% confidence interval, 0.71-0.81). CONCLUSIONS The findings indicate that the grading checklists described are valid, are reliable, and could be used in perioperative crisis management assessment.
Collapse
|
41
|
|
42
|
Hall AK, Pickett W, Dagnone JD. Development and evaluation of a simulation-based resuscitation scenario assessment tool for emergency medicine residents. CAN J EMERG MED 2015; 14:139-46. [DOI: 10.2310/8000.2012.110385] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
ABSTRACT
Objective:
We sought to develop and validate a three-station simulation-based Objective Structured Clinical Examination (OSCE) tool to assess emergency medicine resident competency in resuscitation scenarios.
Methods:
An expert panel of emergency physicians developed three scenarios for use with high-fidelity mannequins. For each scenario, a corresponding assessment tool was developed with an essential actions (EA) checklist and a global assessment score (GAS). The scenarios were (1) unstable ventricular tachycardia, (2) respiratory failure, and (3) ST elevation myocardial infarction. Emergency medicine residents were videotaped completing the OSCE, and three clinician experts independently evaluated the videotapes using the assessment tool.
Results:
Twenty-one residents completed the OSCE (nine residents in the College of Family Physicians of Canada– Emergency Medicine [CCFP-EM] program, six junior residents in the Fellow of the Royal College of Physicians of Canada–Emergency Medicine [FRCP-EM] program, six senior residents in the FRCP-EM). Interrater reliability for the EA scores was good but varied between scenarios (Spearman rho 5 [1] 0.68, [2] 0.81, [3] 0.41). Interrater reliability for the GAS was also good, with less variability (rho 5 [1] 0.64, [2] 0.56, [3] 0.62). When comparing GAS scores, senior FRCP residents outperformed CCFP-EM residents in all scenarios and junior residents in two of three scenarios (p , 0.001 to 0.01). Based on EA scores, senior FRCP residents outperformed CCFP-EM residents, but junior residents outperformed senior FRCP residents in scenario 1 and CCFPEM residents in all scenarios (p 5 0.006 to 0.04).
Conclusions:
This study outlines the creation of a high-fidelity simulation assessment tool for trainees in emergency medicine. A single-point GAS demonstrated stronger relational validity and more consistent reliability in comparison with an EA checklist. This preliminary work will provide a foundation for ongoing future development of simulationbased assessment tools.
Collapse
|
43
|
The role of simulation in continuing medical education for acute care physicians: a systematic review. Crit Care Med 2015; 43:186-93. [PMID: 25343571 DOI: 10.1097/ccm.0000000000000672] [Citation(s) in RCA: 52] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
OBJECTIVES We systematically reviewed the effectiveness of simulation-based education, targeting independently practicing qualified physicians in acute care specialties. We also describe how simulation is used for performance assessment in this population. DATA SOURCES Data source included: DataMEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane CENTRAL Database of Controlled Trials, and National Health Service Economic Evaluation Database. The last date of search was January 31, 2013. STUDY SELECTION All original research describing simulation-based education for independently practicing physicians in anesthesiology, critical care, and emergency medicine was reviewed. DATA EXTRACTION Data analysis was performed in duplicate with further review by a third author in cases of disagreement until consensus was reached. Data extraction was focused on effectiveness according to Kirkpatrick's model. For simulation-based performance assessment, tool characteristics and sources of validity evidence were also collated. DATA SYNTHESIS Of 39 studies identified, 30 studies focused on the effectiveness of simulation-based education and nine studies evaluated the validity of simulation-based assessment. Thirteen studies (30%) targeted the lower levels of Kirkpatrick's hierarchy with reliance on self-reporting. Simulation was unanimously described as a positive learning experience with perceived impact on clinical practice. Of the 17 remaining studies, 10 used a single group or "no intervention comparison group" design. The majority (n = 17; 44%) were able to demonstrate both immediate and sustained improvements in educational outcomes. Nine studies reported the psychometric properties of simulation-based performance assessment as their sole objective. These predominantly recruited independent practitioners as a convenience sample to establish whether the tool could discriminate between experienced and inexperienced operators and concentrated on a single aspect of validity evidence. CONCLUSIONS Simulation is perceived as a positive learning experience with limited evidence to support improved learning. Future research should focus on the optimal modality and frequency of exposure, quality of assessment tools and on the impact of simulation-based education beyond the individuals toward improved patient care.
Collapse
|
44
|
|
45
|
Nguyen D, Gurvitz-Gambrel S, Sloan PA, Dority JS, DiLorenzo A, Hassan ZU, Rebel A. The impact of exposure to liver transplantation anesthesia on the ability to treat intraoperative hyperkalemia: a simulation experience. Int Surg 2015; 100:672-7. [PMID: 25875549 PMCID: PMC4400937 DOI: 10.9738/intsurg-d-14-00279.1] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
The objective of this study was to assess whether resident exposure to liver transplantation anesthesia results in improved patient care during a simulated critical care scenario. Our hypothesis was that anesthesia residents exposed to liver transplantation anesthesia care would be able to identify and treat a simulated hyperkalemic crisis after reperfusion more appropriately than residents who have not been involved in liver transplantation anesthesia care. Participation in liver transplantation anesthesia is not a mandatory component of the curriculum of anesthesiology training programs in the United States. It is unclear whether exposure to liver transplantation anesthesia is beneficial for skill set development. A high-fidelity human patient simulation scenario was developed. Times for administration of epinephrine, calcium chloride, and secondary hyperkalemia treatment were recorded. A total of 25 residents with similar training levels participated: 13 residents had previous liver transplantation experience (OLT), whereas 12 residents had not been previously exposed to liver transplantations (non-OLT). The OLT group performed better in recognizing and treating the hyperkalemic crisis than the non-OLT group. Pharmacologic therapy for hyperkalemia was given earlier (OLT 53.3 ± 27.0 seconds versus non-OLT 148 ± 104.1 seconds; P < 0.01) and hemodynamics restored quicker (OLT 87.9 ± 24.9 seconds versus non-OLT 219.9 ± 87.1 seconds; P < 0.01). Simulation-based assessment of clinical skills is a useful tool for evaluating anesthesia resident performance during an intraoperative crisis situation related to liver transplantations. Previous liver transplantation experience improves the anesthesia resident's ability to recognize and treat hyperkalemic cardiac arrest.
Collapse
Affiliation(s)
- Dung Nguyen
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| | | | - Paul A. Sloan
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| | - Jeremy S. Dority
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| | - Amy DiLorenzo
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| | - Zaki-Udin Hassan
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| | - Annette Rebel
- Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, USA
| |
Collapse
|
46
|
|
47
|
Adherence to guidelines for the management of local anesthetic systemic toxicity is improved by an electronic decision support tool and designated "Reader". Reg Anesth Pain Med 2015; 39:299-305. [PMID: 24956454 DOI: 10.1097/aap.0000000000000097] [Citation(s) in RCA: 40] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
BACKGROUND AND OBJECTIVES A hardcopy or paper cognitive aid has been shown to improve performance during the management of simulated local anesthetic systemic toxicity (LAST) when given to the team leader. However, there remains room for improvement to ensure a system that can achieve perfect adherence to the published guidelines for LAST management. Recent research has shown that implementing a checklist via a designated reader may be of benefit. Accordingly, we sought to investigate the effect of an electronic decision support tool (DST) and designated "Reader" role on team performance during an in situ simulation of LAST. METHODS Participants were randomized to Reader + DST (n = 16, rDST) and Control (n = 15, memory alone). The rDST group received the assistance of a dedicated Reader on the response team who was equipped with an electronic DST. The primary outcome measure was adherence to guidelines. RESULTS For overall and critical percent correct scores, the rDST group scored higher than Control (99.3% vs 72.2%, P < 0.0001; 99.5% vs 70%, P < 0.0001, respectively). In the LAST scenario, 0 (0%) of 15 in the control group performed 100% of critical management steps, whereas 15 (93.8%) of 16 in the rDST group did so (P < 0.0001). CONCLUSIONS In a prospective, randomized single-blinded study, a designated Reader with an electronic DST improved adherence to guidelines in the management of an in situ simulation of LAST. Such tools are promising in the future of medicine, but further research is needed to ensure the best methods for implementing them in the clinical arena.
Collapse
|
48
|
Hastings RH, Rickard TC. Deliberate Practice for Achieving and Maintaining Expertise in Anesthesiology. Anesth Analg 2015; 120:449-59. [DOI: 10.1213/ane.0000000000000526] [Citation(s) in RCA: 31] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
49
|
Gosai J, Purva M, Gunn J. Simulation in cardiology: state of the art. Eur Heart J 2015; 36:777-83. [DOI: 10.1093/eurheartj/ehu527] [Citation(s) in RCA: 26] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/27/2014] [Accepted: 12/27/2014] [Indexed: 01/01/2023] Open
|
50
|
Ortner C, Richebé P, Bollag L, Ross B, Landau R. Repeated simulation-based training for performing general anesthesia for emergency cesarean delivery: long-term retention and recurring mistakes. Int J Obstet Anesth 2014; 23:341-7. [DOI: 10.1016/j.ijoa.2014.04.008] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/05/2013] [Revised: 04/23/2014] [Accepted: 04/26/2014] [Indexed: 10/25/2022]
|