Simulation-based Assessment and Retraining for the Anesthesiologist Seeking Reentry to Clinical Practice

Development and applications of the Anaesthetists' Non-Technical Skills behavioural marker system: a systematic review

OBJECTIVES

To comprehensively synthesise evidence regarding the validity and reliability of the Anaesthetists' Non-Technical Skills (ANTS) behavioural marker system and its application as a tool for the training and assessment of non-technical skills to improve patient safety.

DESIGN

Systematic review.

DATA SOURCES

We employed a citation search strategy. The Scopus and Web of Science databases were searched for articles published from 2002 to May 2022.

ELIGIBILITY CRITERIA

English-language publications that applied the ANTS system in a meaningful way, including its use to guide data collection, analysis and reporting.

DATA EXTRACTION AND SYNTHESIS

Study screening, data extraction and quality assessment were performed by two independent reviewers. We appraised the quality of included studies using the Joanna Briggs Institute Critical Appraisal Checklists. A framework analysis approach was used to summarise and synthesise the included articles.

RESULTS

54 studies were identified. The ANTS system was applied across a wide variety of study objectives, settings and units of analysis. The methods used in these studies varied and included quantitative (n=42), mixed (n=8) and qualitative (n=4) approaches. Most studies (n=47) used the ANTS system to guide data collection. The most commonly reported reliability statistic was inter-rater reliability (n=35). Validity evidence was reported in 51 (94%) studies. The qualitative application outcomes of the ANTS system provided a reference for the analysis and generation of new theories across disciplines.

CONCLUSION

Our results suggest that the ANTS system has been used in a wide range of studies. It is an effective tool for assessing non-technical skills. Investigating the methods by which the ANTS system can be evaluated and implemented for training within clinical environments is anticipated to significantly enhance ongoing enhancements in staff performance and patient safety.

PROSPERO REGISTRATION NUMBER

CRD42022297773.

Remediation practices for health profession students and clinicians: An integrative review

BACKGROUND

The current state of practices in health care remediation is not well known. The purpose of this review is to characterize, assess, and present synthesized results of current student and professional remediation practices described in the literature.

METHODS

This study used an integrative review process including article extraction and review, descriptive characterization and statistics, classification of levels of evidence, assessment of risk of bias, and examination of relationships between factors and types of remediation. Articles were located in a search of PubMed (MEDLINE) and EBSCO (CINAHL Complete) last accessed in May 2022.

INCLUSION CRITERIA

Full text journal articles and Briefs published between January 2001 and May 2022, English language, focus on remediation in health science education programs and professionals, identified key words in title, abstract, or article.

EXCLUSION CRITERIA

Published outside the date range; focus of study or article outside health sciences; main focus not on remediation process or program (defined above), books, presentations and abstracts.

RESULTS

97 articles were included. Design rigor clustered around Level 6 (case-controlled studies, case series, case reports). All programs and activities were reported as successful. There was a statistically significant relationship (p < 0.01) between healthcare discipline and type of remediation.

CONCLUSIONS

A variety of remediation methods for health care students and professionals are reported to be successful. Higher level studies are needed to help define best practices for remediation activities in health care professional knowledge and skill.

Eight years of CRASH: A bi-national initiative helping critical care doctors return to work

Returning to work in critical care after a break in clinical practice can be a daunting process. This article describes development and evolution of the Critical Care, Resuscitation, Airway Skills: Helping you return to work (CRASH) course, including the supporting literature. CRASH is the first bi-national course assisting return to work (RTW) for critical care practitioners. It evolved as a collaborative effort across Australia and New Zealand, involving anaesthetists, emergency physicians and intensivists. The course is based around tailored sessions practising skills and clinical decision-making using simulation and case discussions, incorporating practical tips on returning to work. Participants receive resources to assist RTW including questionnaires and checklists developed by the faculty, which have been used to aid RTW in more than 30 hospitals in Australia and New Zealand. Attendance is open to all critical care practitioners who are registered with the relevant medical board and returning to work, who have taken leave from work for any reason. Since 2014, 197 participants have attended CRASH in-person or virtually, one of the largest RTW groups described in the literature. The aim of this article is to outline the literature around inactivity and RTW, and describe the development and adaptation of the CRASH course.

Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

Physician reentry-A timely topic for emergency medicine

As physician workforce shortages persist, physician reentry is an important and timely issue for the specialty of emergency medicine. Physician reentry is defined as a return to clinical practice following an extended period of clinical inactivity not resulting from discipline or impairment. This review provides a general overview of the physician reentry published literature with a focus on the specialty of emergency medicine. Transition into a non-clinical position, personal health, family issues, and career dissatisfaction all contribute to physicians leaving the workforce voluntarily. Previously, the majority of reentry physicians did not pursue additional training prior to returning to the workforce; however, regulatory agencies are now increasingly requiring additional training, standardized testing, and fitness to practice evaluations prior to restarting clinical work. The burden of proof is on the reentry physician to meet the appropriate requirements for licensure, certification, and credentialing prior to returning to clinical work.

Effectiveness of a new sensorized videolaryngoscope for retraining on neonatal intubation in simulation environment

Background

In recent years, medical training has significantly increased the use of simulation for teaching and evaluation. The retraining of medical personnel in Italy is entrusted to the program of Continuous Education in Medicine, mainly based on theoretical training. The aim of this study is to assess whether the use of a new sensorized platform for the execution of the neonatal intubation procedure in simulation environment can complement theoretical retraining of experienced health professionals.

Methods

Neonatal intubation tests were performed using a commercial manikin and a modified video-laryngoscope by the addition of force and position sensors, which provide the user with feedback when the threshold is exceeded. Two categories carried out the simulation tests: anesthesiologists and pediatricians. The categories were divided into three groups each, and various configurations were tested: the first group of both specialists carried out the tests without feedback (i.e. control groups, gr. A and A1), the second groups received sound and visual feedback from the instrument (gr. B and B1) and the third ones had also the support of a physician expert in the use of the instrument (gr. C and C1). The instrumentation used by pediatricians was provided in a playful form, including a game with increasing difficulty levels.

Results

Both in the case with feedback only and in the case with humans support, anesthesiologists did not show a specific trend of improvement. Pediatricians, in comparison with anesthesiologists, showed a positive reaction to both the presence of feedback and that of experienced personnel. Comparing the performance of the two control groups, the two categories of experienced doctors perform similar forces. Pediatricians enjoyed the “Level Game”, through which they were able to test and confront themselves, trying to improve their own performance.

Conclusions

Our instrument is more effective when is playful and competitive, introducing something more than just a sound feedback, and allowing training by increasing levels. It is more effective if the users can adapt their own technique to the instrument by themselves, without any external help.

Success and Challenge When Returning to Clinical Practice: A Case Series in Anesthesiologist Re-Entry

Introduction

Anesthesiologists returning to clinical practice pose unique challenges for licensing and credentialing boards. Few institutions provide re-education. We describe the physician refresher/re-entry program at our College of Medicine.

Methods

We launched the physician re-entry program in 2006. This individualized program re-educates physicians who left clinical practice for any reason and are seeking to return. We report results achieved for 12 anesthesiologists who successfully completed the course between August 2012 and February 2018.

Results

Seven men and five women left their practices for various reasons, which included relocation, family or medical reasons, substance use, and burnout. None left practice for medical negligence. Range away from clinical activity was 0–10 years. Five had active licenses. Seven were US graduates and five were international. Nine of 12 achieved their goals. Of the 3 others, 1 did not pursue her goal, another did not obtain a residency, and the other just finished the program. Seven out of 9 (78%) achieved their goal within 1 year of course completion.

Discussion

Despite our small sample size, our experience to successfully return inactive physicians to the workforce adds to the scant literature and experience in refreshing inactive physicians. Our trainees return to practice serving communities across the country and are now a pivotal part of the anesthesiology workforce. Thus, this program not only services individual physicians, but the whole community affected by their absence.

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.

Simulation-based Assessment of the Management of Critical Events by Board-certified Anesthesiologists

Background

We sought to determine whether mannequin-based simulation can reliably characterize how board-certified anesthesiologists manage simulated medical emergencies. Our primary focus was to identify gaps in performance and to establish psychometric properties of the assessment methods.

Methods

A total of 263 consenting board-certified anesthesiologists participating in existing simulation-based maintenance of certification courses at one of eight simulation centers were video recorded performing simulated emergency scenarios. Each participated in two 20-min, standardized, high-fidelity simulated medical crisis scenarios, once each as primary anesthesiologist and first responder. Via a Delphi technique, an independent panel of expert anesthesiologists identified critical performance elements for each scenario. Trained, blinded anesthesiologists rated video recordings using standardized rating tools. Measures included the percentage of critical performance elements observed and holistic (one to nine ordinal scale) ratings of participant’s technical and nontechnical performance. Raters also judged whether the performance was at a level expected of a board-certified anesthesiologist.

Results

Rater reliability for most measures was good. In 284 simulated emergencies, participants were rated as successfully completing 81% (interquartile range, 75 to 90%) of the critical performance elements. The median rating of both technical and nontechnical holistic performance was five, distributed across the nine-point scale. Approximately one-quarter of participants received low holistic ratings (i.e., three or less). Higher-rated performances were associated with younger age but not with previous simulation experience or other individual characteristics. Calling for help was associated with better individual and team performance.

Conclusions

Standardized simulation-based assessment identified performance gaps informing opportunities for improvement. If a substantial proportion of experienced anesthesiologists struggle with managing medical emergencies, continuing medical education activities should be reevaluated.

Performance gaps and improvement plans from a 5-hospital simulation programme for anaesthesiology providers: a retrospective study

Background

Simulation is increasingly employed in healthcare provider education, but usage as a means of identifying system-wide practitioner gaps has been limited. We sought to determine whether practice gaps could be identified, and if meaningful improvement plans could result from a simulation course for anaesthesiology providers.

Methods

Over a 2-year cycle, 288 anaesthesiologists and 67 certified registered nurse anaesthetists (CRNAs) participated in a 3.5 hour, malpractice insurer-mandated simulation course, encountering 4 scenarios. 5 anaesthesiology departments within 3 urban academic healthcare systems were represented. A real-time rater scored each individual on 12 critical performance items (CPIs) representing learning objectives for a given scenario. Participants completed a course satisfaction survey, a 1-month postcourse practice improvement plan (PIP) and a 6-month follow-up survey.

Results

All recorded course data were retrospectively reviewed. Course satisfaction was generally positive (88–97% positive rating by item). 4231 individual CPIs were recorded (of a possible 4260 rateable), with a majority of participants demonstrating remediable gaps in medical/technical and non-technical skills (97% of groups had at least one instance of a remediable gap in communication/non-technical skills during at least one of the scenarios). 6 months following the course, 91% of respondents reported successfully implementing 1 or more of their PIPs. Improvements in equipment/environmental resources or personal knowledge domains were most often successful, and several individual reports demonstrated a positive impact on actual practice.

Conclusions

This professional liability insurer-initiated simulation course for 5 anaesthesiology departments was feasible to deliver and well received. Practice gaps were identified during the course and remediation of gaps, and/or application of new knowledge, skills and resources was reported by participants.

Procedural Competence Among Faculty in Academic Health Centers: Challenges and Future Directions

Increasingly, faculty are taking on more direct responsibilities in patient care because of reductions in resident work hours, increasing admissions, and an endless push for efficiency. Furthermore, the rise of different career tracks in academia (i.e., patient care, research, education, or administration) and a drive for efficiency and subspecialization have placed additional strains on academic health centers. Combined, these factors have led to faculty increasingly being placed in the position of supervising bedside procedures that they may have not performed in years or with tools they have never trained with at all. Despite these challenges, procedural retraining for faculty remains nonstandardized across most academic health centers. The resulting lack of procedural competence among faculty creates a number of challenges for the different parties involved.In this Perspective, the authors discuss the nature of the current problem of faculty procedural competence and the challenges it poses for faculty and academic health centers, medicolegal ramifications, and the challenges it poses to the faculty-trainee relationship. The authors then suggest several strategies to delineate and resolve this problem. To delineate the problem, they suggest single-center surveys to address the current paucity of data. To resolve the problem, they suggest the consideration of some modest, low-cost interventions such as having backup systems in place for procedure supervision (e.g., procedural service teams or interventional radiologists) and providing faculty with opportunities to retrain.

The impaired and/or disabled anesthesiologist

PURPOSE OF REVIEW

Impairment and/or disability resulting from any of a number of etiologies will afflict a significant number of anesthesiologists at some point during their career. The impaired anesthesiologist can be difficult to identify and challenging to manage. Questions will arise as to if, how, and when colleagues, family members, or friends should intercede if significant impairment is suspected.This review will examine the common sources of impairment among anesthesiologists and the professional implications of these conditions. We will discuss the obligations of an anesthesiologist and his/her colleagues when there is sufficient suspicion that he/she might be impaired.

RECENT FINDINGS

Substance use disorder remains one of the commonest sources of impairment among both resident and attending anesthesiologists. Other common etiologies of impairment include various physical ailments, major psychiatric disorders, especially depression and burnout, and age related dementia. Many regulatory organizations, healthcare systems, and state licensing agencies have developed programmes and protocols with which to identify and direct into treatment those suspected of significant impairment.

SUMMARY

Some degree of impairment will occur to one-third of anesthesiologists during the course of their career. It is important to understand how such impairments might impact the safe practice of anesthesiology.

Working memory is limited: improving knowledge transfer by optimising simulation through cognitive load theory

This analysis explores how to optimise knowledge transfer in healthcare simulation by applying cognitive load theory to curriculum design and delivery for both novice and expert learners. This is particularly relevant for interprofessional learning which is team-based, as each participant comes to the simulation experience with different levels of expertise. Healthcare simulation can offer opportunities to create complex and dynamic experiences that replicate real clinical situations. Understanding Cognitive Load Theory can foster the acquisition of complex knowledge, skills and abilities required to deliver excellence in patient care without overwhelming a learner's ability to handle new materials due to working memory limitations. The 2 aspects of working memory that will be explored in this paper are intrinsic load and extrinsic load. These will be addressed in terms of the learner's level of expertise and how to consider these elements to enhance the learning environment in simulation scenario development and delivery. By applying the concepts of Cognitive Load Theory, this paper offers educators a method to tailor their curricula to navigate working memory and optimise the opportunity for knowledge transfer.

Progress in simulation education: developing an anesthesia curriculum

PURPOSE OF REVIEW

Simulation's role in anesthesia education is expanding to include more advanced skills and training for subspecialty practice. This review will provide an overview of many recent studies that expand the simulation curriculum for anesthesia education.

RECENT FINDINGS

Recent studies describe a curriculum that uses a range of simulation modalities, including part-task trainers, mannequin-based simulation, virtual reality, in-situ techniques, screen-based simulations as well as encounters with 'standardized' patients, nurses or physician colleagues. A variety of studies describe the use of task-training devises to more effectively acquire skills, such as fibre-optic intubation, ultrasound-guided regional anesthesia and transthoracic echocardiography as well as expand on a variety of teamwork skills particularly in subspecialty anesthesia practice.

SUMMARY

A curriculum is emerging that utilizes a variety of simulation modalities as part of a more comprehensive educational strategy for anesthesia specialty training.

The role of simulation in continuing medical education for acute care physicians: a systematic review

OBJECTIVES

We systematically reviewed the effectiveness of simulation-based education, targeting independently practicing qualified physicians in acute care specialties. We also describe how simulation is used for performance assessment in this population.

DATA SOURCES

Data source included: DataMEDLINE, Embase, Cochrane Database of Systematic Reviews, Cochrane CENTRAL Database of Controlled Trials, and National Health Service Economic Evaluation Database. The last date of search was January 31, 2013.

STUDY SELECTION

All original research describing simulation-based education for independently practicing physicians in anesthesiology, critical care, and emergency medicine was reviewed.

DATA EXTRACTION

Data analysis was performed in duplicate with further review by a third author in cases of disagreement until consensus was reached. Data extraction was focused on effectiveness according to Kirkpatrick's model. For simulation-based performance assessment, tool characteristics and sources of validity evidence were also collated.

DATA SYNTHESIS

Of 39 studies identified, 30 studies focused on the effectiveness of simulation-based education and nine studies evaluated the validity of simulation-based assessment. Thirteen studies (30%) targeted the lower levels of Kirkpatrick's hierarchy with reliance on self-reporting. Simulation was unanimously described as a positive learning experience with perceived impact on clinical practice. Of the 17 remaining studies, 10 used a single group or "no intervention comparison group" design. The majority (n = 17; 44%) were able to demonstrate both immediate and sustained improvements in educational outcomes. Nine studies reported the psychometric properties of simulation-based performance assessment as their sole objective. These predominantly recruited independent practitioners as a convenience sample to establish whether the tool could discriminate between experienced and inexperienced operators and concentrated on a single aspect of validity evidence.

CONCLUSIONS

Simulation is perceived as a positive learning experience with limited evidence to support improved learning. Future research should focus on the optimal modality and frequency of exposure, quality of assessment tools and on the impact of simulation-based education beyond the individuals toward improved patient care.

Simulation in the operating room

Simulation has become a significant training tool in the operating room (OR). It can be used in both simple task training and complex scenarios. The challenge for simulation in the OR is how to translate that which is learned, and perceived to beneficial, into behavioral change and improved patient outcomes. Simulation in the developing world is progressing, but is still hampered by a shortage of material, personnel funding.

The aging anesthesiologist: a narrative review and suggested strategies

PURPOSE

To address an aging anesthesia workforce, we review the relevant changes and implications associated with age in order to stimulate discussion at the individual, local, and national levels regarding appropriate changes in practice aimed at protecting patient safety.

PRINCIPAL FINDINGS

In a 2013 survey of Canadian Anesthesiologists, 22% were aged 55-64 yr, 7% were aged 65-74 yr, and 3% were older than 74 yr. Clinical abilities decline with age, making older anesthesiologists more likely than their younger colleagues to be associated with adverse patient events. Anesthesiologists older than 65 yr in Ontario, Quebec, and British Columbia had 50% more cases involving litigation and almost twice the number of cases involving severe patient injury compared with anesthesiologists younger than 51 yr of age. In the absence of overt deterioration in skills, decisions about reducing activities and retirement are left largely to individuals despite their limited ability to self-assess competence. This state of affairs may contribute to the increased incidence of adverse events and poor patient outcomes.

CONCLUSIONS

Provincial regulatory bodies have peer assessment programs to evaluate physicians at random, following a complaint, and at certain ages, but all have limitations. Simulation has been used widely for training and assessment in the aviation industry as well as in automobile driving exams. Simulation can assess crisis recognition and management, which is crucial in anesthesiology and not well assessed by other methods, and could assist elderly anesthesiologists during the pre-retirement phase of their careers. A standardized schedule for winding down would have advantages for physicians, their department, and their patients. A suggested schedule might include no further on-call duties for those aged 60 yr and older, no further high-acuity cases for those aged 65 yr and older, and retirement from operating room (OR) clinical practice (with possible continuation of non-OR clinical or other non-clinical activities, if desired) at age 70 yr. These timelines could be extended with satisfactory performance in annual simulation sessions involving assessment and practice in crisis management.