Simulation for maintenance of certification in anesthesiology: the first two years

Redesigning Continuous Professional Development: Aligning Learning Needs With Clinical Practice

Continuous professional development (CPD) in health care refers to the process of lifelong learning including the acquisition of new competencies, knowledge, and professional growth throughout the career of a health care professional. Since implementation, there has seen little change or innovation in CPD. This perspective will review the current state of CPD, including the challenges in traditional CPD models, foundations and strategies for redesign to meet the needs of current and future physicians, and suggestions for changes to modernize CPD. Precision education and the use of technology, including artificial intelligence, and their application to CPD will be discussed.

Continuing professional development (CPD) for anesthetists: A systematic review

BACKGROUND

In accordance with the focus on patient safety and quality in healthcare, continuing professional development (CPD) has received increasing levels of attention as a means to ensure physicians maintain their clinical competencies and are fit to practice. There is some evidence of a beneficial effect of CPD, though few studies have evaluated its effect within anesthesia. The primary aim of this systematic review was to establish which CPD activities anesthetists are engaged in and their effectiveness. The secondary aim was to explore which methods are employed to evaluate anesthetists' clinical performance.

METHODS

Databases searched: Medline, Embase and Web of Science, in May 2023. Additional papers were identified through searching the references of included studies. Eligible studies included anesthetists, either exclusively or combined with other healthcare professionals, who underwent a learning activity or assessment method as part of a formalized CPD program or a stand-alone activity. Non-English language studies, non-peer reviewed studies and studies published prior to 2000 were excluded. Eligible studies were quality assessed and narratively synthesized, with results presented as descriptive summaries.

RESULTS

A total of 2112 studies were identified, of which 63 were eligible for inclusion, encompassing more than 137,518 participants. Studies were primarily of quantitative design and medium quality. Forty-one studies reported outcomes of single learning activities, whilst 12 studies investigated different roles of assessment methods in CPD and ten studies evaluated CPD programs or combined CPD activities. A 36 of the 41 studies reported positive effects of single learning activities. Investigations of assessment methods revealed evidence of inadequate performance amongst anesthetists and a mixed effect of feedback. Positive attitudes and high levels of engagement were identified for CPD programs, with some evidence of a positive impact on patient/organizational outcomes.

DISCUSSION

Anesthetists are engaged in a variety of CPD activities, with evidence of high levels of satisfaction and a positive learning effect. However, the impact on clinical practice and patient outcomes remains unclear and the role of assessment is less well-defined. There is a need for further, high-quality studies, evaluating a broader range of outcomes, in order to identify which methods are most effective to train and assess specialists in anesthesia.

Impact and effectiveness of a mandatory competency-based simulation program for pediatric emergency medicine faculty

Introduction

Pediatric emergency medicine physicians struggle to maintain their critical procedural and resuscitation skills. Continuing professional development programs incorporating simulation and competency-based standards may help ensure skill maintenance. Using a logic model framework, we sought to evaluate the effectiveness of a mandatory annual competency-based medical education (CBME) simulation program.

Methods

The CBME program, evaluated from 2016 to 2018, targeted procedural, point-of-care ultrasound (POCUS) and resuscitation skills. Delivery of educational content included a flipped-classroom website, deliberate practice, mastery-based learning, and stop-pause debriefing. Participants' competence was assessed using a 5-point global rating scale (GRS; 3 = competent, 5 = mastery). Statistical process control charts were used to measure the effect of the CBME program on team performance during in situ simulations (ISS), measured using the Team Emergency Assessment Measure (TEAM) scale. Faculty completed an online program evaluation survey.

Results

Forty physicians and 48 registered nurses completed at least one course over 3 years (physician mean ± SD 2.2 ± 0.92). Physicians achieved competence on 430 of 442 stations (97.3%). Mean ± SD GRS scores for procedural, POCUS, and resuscitation stations were 4.34 ± 0.43, 3.96 ± 0.35, and 4.17 ± 0.27, respectively. ISS TEAM scores for "followed standards and guidelines" improved significantly. No signals of special cause variation emerged for the other 11 TEAM items, indicating skills maintenance. Physicians rated CBME training as highly valuable (mean question scores 4.15-4.85/5). Time commitment and scheduling were identified as barriers to participation.

Conclusions

Our mandatory simulation-based CBME program had high completion rates and very low station failures. The program was highly rated and faculty improved or maintained their ISS performance across TEAM scale domains.

Innovative growth and development of a neurological surgery residency cadaveric skull base simulation training program: A single institution experience

OBJECTIVE

Neurosurgical cadaveric and simulation training is a valuable opportunity for residents and fellows to develop as neurosurgeons, further neuroanatomy knowledge, and develop decision-making and technical expertise. The authors describe the growth and development of Oregon Health & Science University (OHSU) Department of Neurological Surgery (NSG) resident hands-on simulation skull base course and provide details of course layout and setup.

METHODS

A three-part surgical simulation series was created to provide training in cadaveric skull base procedures. Course objectives were outlined for participants. Residents participated in NSG hands-on simulation courses (years 2015-2020) and completed annual course curriculum and anonymous course evaluations, which included free text reviews. Courses were evaluated by Likert scale analysis within Python, and free text was quantified using Valence Aware Dictionary for sEntiment Reasoning (VADER). Descriptive statistics were calculated and plotted using Python's Seaborn and Matplotlib library modules.

RESULTS

Analysis included 162 skull base (anterior fossa, middle fossa and lateral, and endoscopic endonasal-based) simulation course evaluations. Resident responses were overwhelmingly positive. Likert responses demonstrated high average responses for each question (4.62 ± 0.56 and above). A positive attitude about simulation courses is supported by an average compound sentiment value of 0.558 ± 0.285.

CONCLUSION

This is the first time Likert responses and sentiment analysis have been used to demonstrate how neurosurgical residents view a comprehensive, multi-year hands-on simulation training program. We hope the information presented serves as a guide for other institutions to develop their own residency educational curriculum in cadaveric skull base procedures.

The Laryngeal Mask

BACKGROUND

Since 2005, the Neonatal Resuscitation Program (NRP) has recommended the laryngeal mask (LM) as an alternative airway when positive-pressure ventilation (PPV) is unsuccessful and intubation is unsuccessful or unfeasible. There is a paucity of literature regarding LM use in high-resource countries; however, anecdotal information and recent studies suggest low implementation.

PURPOSE

This article increases awareness of LM safety and efficacy, provides guidance for determining nursing scope of practice, and includes suggestions for developing a training program.

METHODS

Methods include considering NRP customers' questions, recent systematic reviews of the literature, how NRP recommendations are developed, how to determine nursing scope of practice, and how to develop a training program.

FINDINGS/RESULTS

Recent studies support the NRP recommendation to use the LM as a rescue airway when PPV fails and intubation is not feasible. The LM provides more effective PPV than face-mask ventilation. The LM may be effective as the initial PPV device for infants of at least 34 weeks' gestation. To use the LM, nurses may require training, which may include a simulation-based curriculum.

IMPLICATIONS FOR PRACTICE

Nurses can increase their awareness of the advantages of LM use, determine their scope of practice, and develop a training program. When face-mask ventilation is unsuccessful and intubation is unfeasible, LM insertion will likely increase the newborn's heart rate and avoid intubation.

IMPLICATIONS FOR RESEARCH

Future research should determine training requirements, efficacy of the LM as the initial PPV device in high-resource settings, and LM use during complex resuscitation and in preterm newborns.

Doctors' attitudes to maintenance of professional competence: A scoping review

CONTEXT

Recent decades have seen the international implementation of programmes aimed at assuring the continuing competence of doctors. Maintenance of Professional Competence (MPC) programmes aim to encourage doctors' lifelong learning and ensure high-quality, safe patient care; however, programme requirements can be perceived as bureaucratic and irrelevant to practice, leading to disengagement. Doctors' attitudes and beliefs about MPC are critical to translating regulatory requirements into committed and effective lifelong learning. We aimed to summarise knowledge about doctors' attitudes to MPC to inform the development of MPC programmes and identify under-researched areas.

METHODS

We undertook a scoping review following Arksey and O'Malley, including sources of evidence about doctors' attitudes to MPC in the United States, the United Kingdom, Canada, Australia, New Zealand and Ireland, and using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) as a guide.

RESULTS

One hundred and twenty-five sources of evidence were included in the review. One hundred and two were peer-reviewed publications, and 23 were reports. Most were from the United Kingdom or the United States and used quantitative or mixed methods. There was agreement across jurisdictions that MPC is a good idea in theory but doubt that it achieves its objectives in practice. Attitudes to the processes of MPC, and their impact on learning and practice were mixed. The lack of connection between MPC and practice was a recurrent theme. Barriers to participation were lack of time and resources, complexity of the requirements and a lack of flexibility in addressing doctors' personal and professional circumstances.

CONCLUSIONS

Overall, the picture that emerged is that doctors are supportive of the concept of MPC but have mixed views on its processes. We highlight implications for research and practice arising from these findings.

Development and implementation of a novel, mandatory competency-based medical education simulation program for pediatric emergency medicine faculty

Background

Maintaining acute care physician competence is critically important. Current maintenance of certification (MOC) programs has started to incorporate simulation-based education (SBE). However, competency expectations have not been defined. This article describes the development of a mandatory annual SBE, competency-based simulation program for technical and resuscitation skills for pediatric emergency medicine (PEM) physicians.

Methods

The competency-based medical education (CBME) program was introduced in 2016. Procedural skill requirements were based on a needs assessment derived from Royal College PEM training guidelines. Resuscitation scenarios were modified versions of pre-existing in-situ mock codes or critical incident cases. All full-time faculty were required to participate annually in both sessions. Delivery of educational content included a flipped classroom website, deliberate practice, and stop-pause debriefing. All stations required competency checklists and global rating scales.

Results

Between 2016 and 2018, 40 physicians and 48 registered nurses attended these courses. Overall course evaluations in 2018 were 4.92/5 and 4.93/5. Barriers to implementation include the need for many simulation education experts, time commitment, and clinical scheduling during course events.

Conclusion

We have developed a mandatory simulation-based, technical, and resuscitation CBME program for PEM faculty. This simulation-based CBME program could be adapted to other acute care disciplines. Further research is required to determine if these skills are enhanced both in a simulated and real environment and if there is an impact on patient outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41077-021-00170-4.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific

INTRODUCTION

Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established.

METHODS

Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters.

RESULTS

The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability.

CONCLUSIONS

The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

High-Fidelity Simulation Education and Crisis Resource Management

This article explores high-fidelity simulation in anesthesiology education and provides strategies for its use to improve management of critical events. Educational theories that underlie the use of simulation are described. High-fidelity simulation is useful in teaching technical (diagnostic and procedural) and nontechnical (communication and professionalism) skills, including crisis resource management (CRM) skills. The practice of CRM is fundamental to ensuring patient safety during critical events and to the safe practice of anesthesiology, and its critical elements are presented. A discussion of the use of high-fidelity simulation to learn to combine highly complex procedural skills and CRM is also provided.

Data and debriefing observations on healthcare simulation to prepare for the COVID-19 pandemic

We report on data and debriefing observations in the context of an immersive simulation conducted to (a) train clinicians and (b) test new protocols and kits, developed in table-top exercises without prior clinical experience to fit anticipated clinical encounters in the setting of the rapidly expanding COVID-19 pandemic. We simulated scenarios with particular relevance for anesthesiology, perioperative and critical care, including (1) cardiac arrest, (2) emergency airway management, (3) tele-instruction for remote guidance and supervision, and (4) transporting an intubated patient. Using a grounded theory approach, three authors (MHA, DLR, EHS) developed emergent themes. First alone and then together, we sought consensus in uncovering overarching themes and constructs from the debriefings. We thus performed an informal qualitative thematic analysis based in a critical realist epistemological position - the understanding that our findings, while real, are affected by situational variables and the observer's perspective[1,2]. We compared data from videos and triangulated the data by member checking. All participants and course instructors volunteered to participate in this educational project and contributed as co-authors to this manuscript. During debriefing, we applied crisis resource management concepts including situation awareness, prioritization of tasks, and clear communication practices, conducting the debriefing with emphasis on current TeamStepps 2.0 terminology and concepts. [3,4] In addition, we re-evaluated formerly familiar processes, as shortcomings of protocols, kits, and interdisciplinary cooperation became apparent. The data provide detailed observations on how immersive simulation and debriefing among peers mitigated the unfamiliarity of individual clinicians and the organization at large with the demands of an unprecedented healthcare crisis. We also observed and report on the anxiety caused by resource constraints, risk to clinicians in the face of limited personal equipment, and the overall uncertainty surrounding COVID-19. We began to summarize, interpret, critique, and discuss our data and debriefing observations in a rapid co-publication in the Journal of Clinical Anesthesia. [Healthcare Simulation to Prepare for the COVID-19 Pandemic][5].

Roads less traveled: understanding the "why" in simulation as an integrated continuing professional development activity

Background

The simulation community has experienced significant advances, strengthening the case for the use of simulation in medical education toward improving patient outcomes. However, an underlying assumption particularly regarding utilization of simulation by those who are in practice, is that simulation will be selected as a continuing professional development (CPD) strategy. Exploring reasons for choices of educational formats, particularly regarding simulation, is poorly integrated into CPD research.

Discussion

Despite significant advances the scientific simulation community has been slow to produce evidence regarding why practitioners may be reserved in engaging in simulation or not. Using examples from related education contexts the author attempts to bridge simulation science, CPD and less commonly used theoretical frameworks to address this issue. The author argues that theoretical perspectives that recognize the use of simulation for CPD as a socio-personal process and/or a personal or group issue (e.g., theories of intelligence, self-determination theory, theory of planned behavior, social identity theory) and that are conceptually distinct from educational mechanisms/ provision are necessary to advance simulation use in CPD contexts.

Conclusion

Given the close relationship practicing clinicians have to patient outcomes a new imperative may be to focus on the theoretical and practical links informing simulation use for CPD at the level of the individual and individual-among-professional groups. The simulation community may therefore need to engage in research that attempts to further uncover and address underlying issues of “why” clinicians integrate simulation as CPD activities or not.

Can Maintenance of Certification Work? Associations of MOC and Improvements in Physicians' Knowledge and Practice

PURPOSE

To summarize the findings of studies, conducted by individuals both internal and external to the American Board of Medical Specialties (ABMS) Member Boards, of the associations of Maintenance of Certification (MOC) and improvements in physicians' knowledge and patient care processes or outcomes.

METHOD

The authors conducted a narrative review of studies identified by searching PubMed and Web of Science for English-language articles from the United States published between 2000 and May 2017. To be included, articles had to examine the relationship of MOC to physician knowledge, clinical practice processes, or patient care outcomes. The initial search yielded 811 articles. After two rounds of review and excluding those articles that did not fit the study criteria, 39 articles were included for analysis.

RESULTS

The 39 included studies were conducted by or included diplomates of 12 ABMS Member Boards. Twenty-two studies examined MOC processes that were developed by an ABMS Board; 17 examined interventions that were developed by nonboard entities but accepted for MOC credit by an ABMS Board. Thirty-eight studies examined a single component of MOC; 24 studied the improvement in medical practice component. Thirty-seven studies reported at least one positive outcome.

CONCLUSIONS

Most of the studies included in this review highlighted circumstances in which MOC was associated with positive impacts on physician knowledge and patient care processes or outcomes. Future collaborative research is needed to improve the relevance, helpfulness, and generalizability of continuing certification to different physicians across specialties and practice settings.

Use of Simulation in Performance Improvement

Human error and system failures continue to play a substantial role in preventable errors that lead to adverse patient outcomes or death. Many of these deaths are not the result of inadequate medical knowledge and skill, but occur because of problems involving communication and team management. Anesthesiologists pioneered the use of simulation for medical education in an effort to improve physician performance and patient safety. This article explores the use of simulation for performance improvement. Educational theories that underlie effective simulation programs are described as driving forces behind the advancement of simulation in performance improvement.

Toward an Optimal Pedagogy for Teamwork

Teamwork and collaboration are increasingly listed as core competencies for undergraduate health professions education. Despite the clear mandate for teamwork training, the optimal method for providing that training is much less certain. In this Perspective, the authors propose a three-level classification of pedagogical approaches to teamwork training based on the presence of two key learning factors: interdependent work and explicit training in teamwork. In this classification framework, level 1-minimal team learning-is where learners work in small groups but neither of the key learning factors is present. Level 2-implicit team learning-engages learners in interdependent learning activities but does not include an explicit focus on teamwork. Level 3-explicit team learning-creates environments where teams work interdependently toward common goals and are given explicit instruction and practice in teamwork. The authors provide examples that demonstrate each level. They then propose that the third level of team learning, explicit team learning, represents a best practice approach in teaching teamwork, highlighting their experience with an explicit team learning course at the University of Colorado Anschutz Medical Campus. Finally, they discuss several challenges to implementing explicit team-learning-based curricula: the lack of a common teamwork model on which to anchor such a curriculum; the question of whether the knowledge, skills, and attitudes acquired during training would be transferable to the authentic clinical environment; and effectively evaluating the impact of explicit team learning.

Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists

Background

We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods.

Methods

A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant’s technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist.

Results

Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance.

Conclusions

Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.

A Taxonomy of Delivery and Documentation Deviations During Delivery of High-Fidelity Simulations

INTRODUCTION

We developed a taxonomy of simulation delivery and documentation deviations noted during a multicenter, high-fidelity simulation trial that was conducted to assess practicing physicians' performance. Eight simulation centers sought to implement standardized scenarios over 2 years. Rules, guidelines, and detailed scenario scripts were established to facilitate reproducible scenario delivery; however, pilot trials revealed deviations from those rubrics. A taxonomy with hierarchically arranged terms that define a lack of standardization of simulation scenario delivery was then created to aid educators and researchers in assessing and describing their ability to reproducibly conduct simulations.

METHODS

Thirty-six types of delivery or documentation deviations were identified from the scenario scripts and study rules. Using a Delphi technique and open card sorting, simulation experts formulated a taxonomy of high-fidelity simulation execution and documentation deviations. The taxonomy was iteratively refined and then tested by 2 investigators not involved with its development.

RESULTS

The taxonomy has 2 main classes, simulation center deviation and participant deviation, which are further subdivided into as many as 6 subclasses. Inter-rater classification agreement using the taxonomy was 74% or greater for each of the 7 levels of its hierarchy. Cohen kappa calculations confirmed substantial agreement beyond that expected by chance. All deviations were classified within the taxonomy.

CONCLUSIONS

This is a useful taxonomy that standardizes terms for simulation delivery and documentation deviations, facilitates quality assurance in scenario delivery, and enables quantification of the impact of deviations upon simulation-based performance assessment.

Performance gaps and improvement plans from a 5-hospital simulation programme for anaesthesiology providers: a retrospective study

Background

Simulation is increasingly employed in healthcare provider education, but usage as a means of identifying system-wide practitioner gaps has been limited. We sought to determine whether practice gaps could be identified, and if meaningful improvement plans could result from a simulation course for anaesthesiology providers.

Methods

Over a 2-year cycle, 288 anaesthesiologists and 67 certified registered nurse anaesthetists (CRNAs) participated in a 3.5 hour, malpractice insurer-mandated simulation course, encountering 4 scenarios. 5 anaesthesiology departments within 3 urban academic healthcare systems were represented. A real-time rater scored each individual on 12 critical performance items (CPIs) representing learning objectives for a given scenario. Participants completed a course satisfaction survey, a 1-month postcourse practice improvement plan (PIP) and a 6-month follow-up survey.

Results

All recorded course data were retrospectively reviewed. Course satisfaction was generally positive (88–97% positive rating by item). 4231 individual CPIs were recorded (of a possible 4260 rateable), with a majority of participants demonstrating remediable gaps in medical/technical and non-technical skills (97% of groups had at least one instance of a remediable gap in communication/non-technical skills during at least one of the scenarios). 6 months following the course, 91% of respondents reported successfully implementing 1 or more of their PIPs. Improvements in equipment/environmental resources or personal knowledge domains were most often successful, and several individual reports demonstrated a positive impact on actual practice.

Conclusions

This professional liability insurer-initiated simulation course for 5 anaesthesiology departments was feasible to deliver and well received. Practice gaps were identified during the course and remediation of gaps, and/or application of new knowledge, skills and resources was reported by participants.

Neonatology faculty development using simulation

The goal of faculty development activities is to supply the public with knowledgeable, skilled, and competent physicians who are prepared for high performance in the dynamic and complex healthcare environment. Current faculty development programs lack evidence-based support and are not sufficient to meet the professional needs of practicing physicians. Simulation activities for faculty development offer an alternative to traditional, teacher-centric educational offerings. Grounded in adult learning theory, simulation is a learner-centric, interactive, efficient, and effective method to train busy professionals. Many of the faculty development needs of clinical neonatologists can be met by participating in simulation-based activities that focus on technical skills, teamwork, leadership, communication, and patient safety.

Evaluation of a simulation-based curriculum for implementing a new clinical protocol

OBJECTIVE

To evaluate the implementation of a new clinical protocol utilizing on-unit simulation for team training.

METHODS

A prospective observational study was performed at the obstetrics unit of Von Voightlander Women's Hospital, Michigan, USA, between October 1, 2012 to April 30, 2013. All members of the labor and delivery team were eligible for participation. Traditional education methods and in-situ multi-disciplinary simulations were used to educate labor and delivery staff. Following each simulation, participants responded to a survey regarding their experience. To evaluate the effect of the interventions, paging content was analyzed for mandated elements and adherence to operating room entry-time tracking was examined.

RESULTS

In total, 51 unique individuals participated in 12 simulations during a 6-month period. Simulation was perceived as a valuable activity and paging content improved. Following the intervention, the inclusion of a goal time for reaching the operation room increased from 7% to 61% of pages and the proportion of patients entering to operating room within 10 minutes of the stated goal increased from 67% to 85%.

CONCLUSION

The training program was well received, and the accuracy of the communication and the goal set for reaching the operating room improved.

Adapting Simulation Curricula to the Future Needs of Anesthesiology

We have discussed some examples of the types of program development strategies that are in common use and have presented examples of the type of performance gaps that can occur when a coordinated curriculum development process is not applied (or is applied in an uncoordinated fashion). We have outlined one method to develop a simulation-based curriculum focusing more on ways to identify how to "simulate what is needed" rather than using the "simulate what we know" style. We believe that curricula must be designed to continually evolve rather being conceived as a single finished program. We have attempted to illustrate what a designed simulation curriculum for training anesthesiology residents and faculty in the PSH might look like, and we have provided a sample scenario to illustrate how this process could be presented (Supplemental Digital Content 2, http://links.lww.com/AIA/A23, Supplemental Digital Content 3, http://links.lww.com/AIA/A24). Our hope is that this model may be applied to create simulation education curricula in a wide variety of areas. We suggest that it be a part of any attempt to create a standardized, longitudinal simulation-based assessment for residents or practitioners. A cohesive, strategic approach to simulation curriculum design and implementation will be required as we seek to create the same type of effective safety training in medicine that has been present in other high-risk professions.

Physician Satisfaction With and Practice Changes Resulting From American Board of Family Medicine Maintenance of Certification Performance in Practice Modules

INTRODUCTION

Physician payment in the United States will be increasingly tied to quality measurement and performance. Whether participation in quality improvement (QI) through Maintenance of Certification for Family Physicians Performance in Practice Modules (PPMs) is useful and results in practice change remains unknown.

METHODS

All PPM feedback data from inception to April 2014 were analyzed using descriptive statistics by year, topic, and number of PPMs completed. Qualitative content analysis was applied to analyze responses to open-ended questions on practice changes.

RESULTS

Of note, 29,755 diplomates completed 38,201 PPMs; median 1 interquartile range (1, 1). Nearly two-thirds (65.8%, n = 25,150) of PPMs had completed feedback surveys. Of note, 78.7% of respondents indicated that they would change patient care and 90.2% indicated that they would continue QI activities after completing the PPM. Respondents endorsed high relevance to practice (90.5%), high currency of clinical information (86.4%), and high usefulness of clinical information (80.5%). When feedback was analyzed by the number of PPMs completed, respondents were less likely to change care but reported increased usefulness to practice and stable intention to continue QI efforts with more PPMs completed. Of note, 86.0% of respondents who said that they would change care provided examples: these varied by PPM topic but "doing more," focusing on patients, and education were common.

DISCUSSION

These findings suggest that QI completed through the PPMs may assist family physicians in improving the care they provide. Furthermore, ratings by the number of PPMs completed suggest that repeated exposure to QI efforts produce continued relevance and usefulness, even when changes in practice decline.

Correlation of Simulation Examination to Written Test Scores for Advanced Cardiac Life Support Testing: Prospective Cohort Study

Introduction

Traditional Advanced Cardiac Life Support (ACLS) courses are evaluated using written multiple-choice tests. High-fidelity simulation is a widely used adjunct to didactic content, and has been used in many specialties as a training resource as well as an evaluative tool. There are no data to our knowledge that compare simulation examination scores with written test scores for ACLS courses.

Objective

To compare and correlate a novel high-fidelity simulation-based evaluation with traditional written testing for senior medical students in an ACLS course.

Methods

We performed a prospective cohort study to determine the correlation between simulation-based evaluation and traditional written testing in a medical school simulation center. Students were tested on a standard acute coronary syndrome/ventricular fibrillation cardiac arrest scenario. Our primary outcome measure was correlation of exam results for 19 volunteer fourth-year medical students after a 32-hour ACLS-based Resuscitation Boot Camp course. Our secondary outcome was comparison of simulation-based vs. written outcome scores.

Results

The composite average score on the written evaluation was substantially higher (93.6%) than the simulation performance score (81.3%, absolute difference 12.3%, 95% CI [10.6–14.0%], p<0.00005). We found a statistically significant moderate correlation between simulation scenario test performance and traditional written testing (Pearson r=0.48, p=0.04), validating the new evaluation method.

Conclusion

Simulation-based ACLS evaluation methods correlate with traditional written testing and demonstrate resuscitation knowledge and skills. Simulation may be a more discriminating and challenging testing method, as students scored higher on written evaluation methods compared to simulation.

Revalidation: implications for Australian anaesthetists

In early 2015, the Medical Board of Australia commissioned research into international revalidation models and what might be applicable for Australia. This review examines the implications for Australian anaesthetists. What problem is revalidation seeking to address? What is happening in similar countries? Is there an issue with Australian anaesthetists' performance? Isn't continuing professional development enough? Could the Medical Board target known high-risk doctors? What is the evidence for the benefit of revalidation? How is and how should the profession be involved? Revalidation has been introduced in other developed countries. It commonly involves continuing professional development, feedback from colleagues, co-workers and patients, clinical audit and peer review. Although its evidence base is limited, the General Medical Council in the United Kingdom is evaluating its revalidation system, which should provide useful guidance for other countries. Australian anaesthetists and their professional organisations must remain informed about, and engaged in, the national debate about revalidation, to ensure that any new process is workable for Australian anaesthesia practice.

Practice improvements based on participation in simulation for the maintenance of certification in anesthesiology program

BACKGROUND

This study describes anesthesiologists' practice improvements undertaken during the first 3 yr of simulation activities for the Maintenance of Certification in Anesthesiology Program.

METHODS

A stratified sampling of 3 yr (2010-2012) of participants' practice improvement plans was coded, categorized, and analyzed.

RESULTS

Using the sampling scheme, 634 of 1,275 participants in Maintenance of Certification in Anesthesiology Program simulation courses were evaluated from the following practice settings: 41% (262) academic, 54% (339) community, and 5% (33) military/other. A total of 1,982 plans were analyzed for completion, target audience, and topic. On follow-up, 79% (1,558) were fully completed, 16% (310) were partially completed, and 6% (114) were not completed within the 90-day reporting period. Plans targeted the reporting individual (89% of plans) and others (78% of plans): anesthesia providers (50%), non-anesthesia physicians (16%), and non-anesthesia non-physician providers (26%). From the plans, 2,453 improvements were categorized as work environment or systems changes (33% of improvements), teamwork skills (30%), personal knowledge (29%), handoff (4%), procedural skills (3%), or patient communication (1%). The median word count was 63 (interquartile range, 30 to 126) for each participant's combined plans and 147 (interquartile range, 52 to 257) for improvement follow-up reports.

CONCLUSIONS

After making a commitment to change, 94% of anesthesiologists participating in a Maintenance of Certification in Anesthesiology Program simulation course successfully implemented some or all of their planned practice improvements. This compares favorably to rates in other studies. Simulation experiences stimulate active learning and motivate personal and collaborative practice improvement changes. Further evaluation will assess the impact of the improvements and further refine the program.

Simulation for Maintenance of Certification

Maintenance of certification (MOC) is a process through which practitioners are able to show continuing competence in their areas of expertise. Simulation plays an increasingly important role in the assessment of students and residents, as well as in the initial practice certification for health care professionals. The use of simulation as an assessment tool in MOC has been sluggish to be universally accepted. This article discusses the role of simulation in health care education, how simulation might be effectively applied in the MOC process, and the future role of simulation in the MOC process.

[Simulation training in pulmonary medicine: Rationale, review of the literature and perspectives]

Training in pulmonary medicine requires the acquisition of a great deal of knowledge, but also technical know-how and interpersonal skills. The prevailing teaching pattern is mentorship. It implies a direct transmission of knowledge, but also entails some drawbacks such as disparity in learning opportunities, subjective evaluation of the trainee and potential risks for patients. There is growing interest in simulation training as a teaching technique, where students practice their skills in a secure environment, then analyse their performance in a debriefing session. It is complementary to other learning methods (abstraction, observation or mentorship) and forms part of an ethical approach: 'never practice on a real patient for the first time'. We have reviewed the literature related to simulation training in pulmonary medicine and in particular for physical examination, technical skills, pathologies, communication with patients and therapeutic education. In most of the studies, simulation training is a way of speeding up students' training - without necessarily yielding better results - and of respecting the procedures. We then present the French regulations and official guidelines regarding the use of this training method in the teaching of medicine. Finally, we shall consider some prospects of this approach for the community of pulmonologists.

This Is Not a Test!

Simulation-based training courses for maintenance of certification in anesthesiology have been very well received and have led to widespread reports of meaningful practice improvement.

Building the Evidence on Simulation Validity

Background:

Effective teamwork is important for patient safety, and verbal communication underpins many dimensions of teamwork. The validity of the simulated environment would be supported if it elicited similar verbal communications to the real setting. The authors hypothesized that anesthesiologists would exhibit similar verbal communication patterns in routine operating room (OR) cases and routine simulated cases. The authors further hypothesized that anesthesiologists would exhibit different communication patterns in routine cases (real or simulated) and simulated cases involving a crisis.

Methods:

Key communications relevant to teamwork were coded from video recordings of anesthesiologists in the OR, routine simulation and crisis simulation and percentages were compared.

Results:

The authors recorded comparable videos of 20 anesthesiologists in the two simulations, and 17 of these anesthesiologists in the OR, generating 400 coded events in the OR, 683 in the routine simulation, and 1,419 in the crisis simulation. The authors found no significant differences in communication patterns in the OR and the routine simulations. The authors did find significant differences in communication patterns between the crisis simulation and both the OR and the routine simulations. Participants rated team communication as realistic and considered their communications occurred with a similar frequency in the simulations as in comparable cases in the OR.

Conclusion:

The similarity of teamwork-related communications elicited from anesthesiologists in simulated cases and the real setting lends support for the ecological validity of the simulation environment and its value in teamwork training. Different communication patterns and frequencies under the challenge of a crisis support the use of simulation to assess crisis management skills.

Ensuring high-quality patient care: the role of accreditation, licensure, specialty certification and revalidation in medicine

CONTEXT

The accreditation of medical school programmes and the licensing and revalidation (or recertification) of doctors are thought to be important for ensuring the quality of health care. Whereas regulation of the medical profession is mandated in most jurisdictions around the world, the processes by which doctors become licensed, and maintain their licences, are quite varied. With respect to educational programmes, there has been a recent push to expand accreditation activities. Here too, the quality standards on which medical schools are judged can vary from one region to another.

OBJECTIVES

Given the perceived importance placed by the public and other stakeholders on oversight in medicine, both at the medical school and individual practitioner levels, it is important to document and discuss the regulatory practices employed throughout the world.

METHODS

This paper describes current issues in regulation, provides a brief summary of research in the field, and discusses the need for further investigations to better quantify relationships among regulatory activities and improved patient outcomes.

DISCUSSION

Although there is some evidence to support the value of medical school accreditation, the direct impact of this quality assurance initiative on patient care is not yet known. For both licensure and revalidation, some investigations have linked specific processes to quality indicators; however, additional evaluations should be conducted across the medical education and practice continuum to better elucidate the relationships among regulatory activities and patient outcomes. More importantly, the value of accreditation, licensure and revalidation programmes around the world, including the effectiveness of specific protocols employed in these diverse systems, needs to be better quantified and disseminated.