Detailed Findings From the CLER National Report of Findings 2018

Quality Improvement Projects and Anesthesiology Graduate Medical Education: A Systematic Review

Quality improvement (QI) projects are essential components of graduate medical education and healthcare organizations to improve patient outcomes. We systematically reviewed the literature on QI projects in anesthesiology graduate medical education programs to assess whether these projects are leading to publications. A literature search was conducted in July 2023, using PubMed, Embase, and the Central Register of Controlled Trials (CENTRAL) for articles describing QI initiatives originating within the United States and applicable to anesthesiology residency training programs. The following data were collected: intervention(s), sample size (number of participants or events), outcome metric(s), result(s), and conclusion(s). One hundred and fifty publications were identified, and 31 articles met the inclusion criteria. A total of 2,259 residents and 72,889 events were included in this review. Educational modalities, such as simulation, training sessions, or online curricula, were the most prevalent interventions in the included studies. Pre-intervention and post-intervention assessments were the most common outcome metrics reported. Our review of the literature demonstrates that few QI projects performed within anesthesiology training programs lead to published manuscripts. Further research should aim at increasing the impact of required QI projects within the sponsoring institution and specialty.

Prevalence of and Barriers to Health Disparities Education Among Otolaryngology Residency Curricula

Objective

To assess the prevalence of health disparities curricula in otolaryngology residency programs and identify implementation barriers.

Study Design

Cross-sectional survey.

Setting

National otolaryngology residency programs.

Methods

A survey based on published literature discussing the incorporation of health disparities curricula, educational design, quality, barriers to implementation, and patient demographics was sent to US otolaryngology residency program directors (PDs). Otolaryngology programs excluded from consideration included those of osteopathic recognition, programs outside of the United States, and military programs. In excluding osteopathic, international, and military-based residency programs from our survey, we aimed to maintain sample homogeneity and focus our analysis on allopathic programs due to potential variations in demographic compositions and practice settings. This decision was made to ensure a more targeted examination of health disparities within a specific context, aligning with our research objectives and resource constraints. Anonymous survey results were collected and analyzed to determine the prevalence of health disparities curricula as well as their effectiveness and standardization across residency programs.

Results

A total of 24 PDs (response rate, 23%) responded to the survey. Half of the PDs reported having a health disparities curriculum, among whom only 25% felt the quality of their curriculum was very good or excellent. All institutions with an explicit health disparities educational program reported having developed their own curriculum, 75% of which changed annually. However, 92% of these programs reported not measuring outcomes to assess their curriculum's utility. The most reported barriers to curriculum development for all programs included insufficient time (63%), limited teaching ability specific to health disparities education (54%), and faculty disinterest in teaching (33%).

Conclusion

Very few of the surveyed otolaryngology residency programs have implemented a health disparities curriculum. A comprehensive and standardized health disparities curriculum would be beneficial to ensure that residents can confidently develop competency in health disparities, aligning with the Clinical Learning Environment Review mandate and Accreditation Council for Graduate Medical Education expectations.

How Many Lives Will You Save? A Mixed Methods Evaluation of a Novel, Online Game for Patient Safety and Quality Improvement Education

Medical trainees have limited knowledge of quality improvement and patient safety concepts. The authors developed a free quality improvement/patient safety educational game entitled Safety Quest (SQ). However, 1803 undergraduate medical trainees, graduate medical trainees, and continuing medical education learners globally completed at least 1 level of SQ. Pre- and post-SQ knowledge and satisfaction were assessed among continuing medical education learners. Thematic analysis of feedback given by trainees was conducted. Among graduate medical trainees, SQ outranked other learning modalities. Three content areas emerged from feedback: engagement, ease of use, and effectiveness; 87% of comments addressing engagement were positive. After completing SQ, 98.6% of learners passed the post-test, versus 59.2% for the pretest ( P < 0.0001). Ninety-three percent of learners agreed that SQ was engaging and interactive, and 92% believed it contributed to their professional growth. With an increased need for educational curricula to be delivered virtually, gamification emerges as a unique strategy that learners praise as engaging and effective.

Endured and prevailed: a phenomenological study of doctors' first year of clinical practice

CONTEXT

The challenging nature of the transition from medical student to doctor is highlighted by the associated negative consequences to new doctors' mental health and wellbeing. Enhanced understanding of the lived experience of recent medical graduates as they move through the stages of transition over the first year of practice can inform interventions to ease the difficulties encountered.

METHODS

Using interpretative phenomenological analysis (IPA), a novel approach to this topic, we explored the lived experience of transition from student to doctor over the first year of practice after graduation. Twelve new graduates were purposively recruited. We conducted semi-structured interviews at the end of their first year of practice with respect to their experience over the first year.

RESULTS

The experience of transition was characterised by overlapping temporal stages. Participants' initial adjustment period was characterised by shock, coping and stabilisation. A phase of development followed, with growth in confidence and a focus on self-care. Adversity was experienced in the form of interprofessional tensions, overwork, isolation and mistreatment. Finally, a period of reflection and rationalisation marked the end of the first year.

DISCUSSION

Following initial anxiety regarding competence and performance, participants' experience of transition was predominantly influenced by cultural, relational and contextual aspects of clinical practice. Solutions to ease this challenging time include stage-specific transitional interventions, curricular change at both undergraduate and postgraduate levels and a re-evaluation of the clinical learning environment to mitigate the difficulties endured.

Vax the Max, a Gamification Intervention for COVID-19 Vaccination Task Engagement in the Inpatient Setting

The development of vaccines for SARS-CoV-2 has offered game-changing protection from severe disease and death from COVID-19. Despite efforts to vaccinate individuals in the ambulatory setting, a sizable minority of the US population remains unvaccinated for COVID-19. For unvaccinated patients, hospitalization for non-COVID-19 illness offers another opportunity for vaccination. In the summer of 2021, the authors noted that COVID-19 vaccination rate for medicine inpatients at their hospital had fallen to 5.3 vaccine doses administered per 4-week block. In response, they created Vax the Max, a gamification program of COVID-19 vaccination tasks where internal medicine resident teams were awarded points for completing these tasks. Residents were anonymously surveyed after participation. The hospital demonstrated higher rates of administering the initial COVID-19 vaccine dose and completing the vaccine series in the inpatient setting per 4-week plan-do-study-act cycle after implementation of Vax the Max (5.3 versus 8.8 doses per plan-do-study-act cycle). Among residents, 76.8% reported that Vax the Max spurred their COVID-19 task engagement, and 66% reported that a similar gamification model could be utilized for a different clinical task in the future. An increase was observed in the COVID-19 vaccination rate for medicine inpatients after launching the Vax the Max competition. This occurred in the setting of resident turnover every 4 weeks, which normally makes practice sustainment more challenging. Despite this, a high degree of engagement was produced by itinerant residents. There is potential to explore similar gamification approaches involving resident physicians in areas of quality improvement and patient safety.

The Surgical Safety Checklist from the resident's perspective. Observational study

Introduction: The Surgical Safety Checklist implemented by the World Health Organization has proven to decrease perioperative morbidity and mortality; however, the barriers and limitations to its implementation are consistently reported in the literature. Objective: To establish the level of appropriation of the surgical safety checklist in the training of human resources in anesthesiology, in addition to identifying the perception and the level of implementation of such checklist at the national scale.   Methods: Descriptive cross-sectional study conducted through a survey administered to the residents of anesthesiology in Colombia. Likert-type questions were included, distributed into three domains: appropriation, perception and implementation. Results: 215 answers corresponding to 54.5 % of the population were analyzed, comprising participants from all of the anesthesiology programs in the country. 20% of the residents have never been subject to formal academic reviews about checklists, and this trend did not change throughout the residency; 97.2 % considers that the implementation of the lists improves the safety of surgical procedures and 40 % have seen rejection or indifference by surgeons. 80.5 % of the residents have seen the frequent use of the checklist, while only 13.5% have seen the use of the checklist during the three surgical moments – before the induction of anesthesia, before the surgical incision, and before the patient leaves the operating room -; 88 % have observed that the form is completed without actually doing the verification. Conclusions: There is limited exposure to education about the surgical safety checklist in anesthesiology postgraduate programs in the country. The residents have a favorable perception about the value of the list, however, there are some shortcomings in its administration.

The Mystery Dinner RCA: Using Gamification and Simulation to Teach Root Cause Analysis

Introduction

Root cause analysis (RCA) is a widely utilized tool for investigating systems issues that lead to patient safety events and near misses, yet only 38% of learners participate in an interdisciplinary patient safety investigation during training. Common barriers to RCA education and participation include faculty time and materials, trainee time constraints, and learner engagement.

Methods

We developed a simulated RCA workshop to be taught to a mix of medical and surgical specialties from over 11 GME programs and to third-year medical students. The workshop was a single 90-minute session formatted as a gamified mystery dinner including characters and sequentially revealed clues to promote engagement. Participant satisfaction and subjective knowledge, skills, and attitudes were assessed with a pre/post survey.

Results

The workshop was completed by 134 learners between October 2018 and October 2019. The short workshop duration and premade simulation allowed a small number of faculty to train a wide variety of learners in various educational settings. Participants' presurvey (124 out of 134, 92%) versus postsurvey (113 out of 134, 84%) responses showed that attitudes about RCA were statistically improved across all domains queried, with an average effect size of 0.6 (moderate effect); 91% of participants would recommend this course to a colleague.

Discussion

A 90-minute, gamified, simulated RCA workshop was taught to medical students and multiple GME specialties with subjective improvements in patient safety attitudes and knowledge while alleviating faculty time constraints in case development.

Developing Leaders and Scholars in Health Care Improvement: The VA Quality Scholars Program Competencies

Despite the need for leaders in health care improvement across health professions, there are no standards for the knowledge and skills that should be achieved through advanced interprofessional health care improvement training. Existing health care improvement training competencies focus on foundational knowledge expected of all trainees or for specific career pathways. Health care improvement leaders fill multiple roles within organizations and promote interprofessional improvement practice. The diverse skill set required of modern health care improvement leaders necessitates the development of training competencies specifically for fellowships in applied health care improvement. The authors describe the development of the revised national Veterans Affairs Quality Scholars (VAQS) Program competencies. The VAQS Program is an interprofessional, postdoctoral training program whose mission is to develop leaders and scholars to improve health care. An interprofessional committee of VAQS faculty reviewed and revised the competencies over 4 months beginning in fall 2018. The first draft was developed using 111 competencies submitted by 11 VAQS training sites and a review of published competencies. The final version included 22 competencies spanning 5 domains: interprofessional collaboration and teamwork, improvement and implementation science, organization and system leadership, methodological skills and analytic techniques for improvement and research, and teaching and coaching. Once attained, the VAQS competencies will guide the skill development that interprofessional health care improvement leaders need to participate in and lead health care improvement scholarship and implementation. These broad competencies are relevant to advanced training programs that develop health care improvement leaders and scholars and may be used by employers to understand the knowledge and skills expected of individuals who complete advanced fellowships in applied health care improvement.

Using a Direct Observation Tool (TOC-CEX) to Standardize Transitions of Care by Residents at a Community Hospital

Background: High-quality transitions of care are crucial for patient safety in hospitals, yet few undergraduate curricula include transition-of-care training. In 2012, the Wayne State University Office of Graduate Medical Education (WSUGME) required its residency programs to use the SAIF-IR mnemonic (summary, active issues, if-then contingency planning, follow-up activities, interactive questioning, readback) to ensure accurate and uniform handoffs. Subsequent program evaluations indicated that resident awareness and adoption of the mnemonic at our primary clinical site, Ascension Providence Rochester Hospital (APRH), could be improved. According to our institution's 2016 Clinical Learning Environment Review (CLER), 88% of residents reported following a standardized transition of care handoff, and 53% reported that faculty rarely supervised their handoffs. A 2016 WSUGME internal survey also revealed low rates of awareness (7% to 10%) of the mandated mnemonic. WSUGME then created a direct observation tool, the Transitions of Care-Clinical Evaluation Exercise (TOC-CEX), for faculty to monitor resident skill in using the mnemonic and thus standardize transitions of care as a practice habit at APRH. Methods: Since 2014, WSUGME had relied on 2 methods for training residents in the required handoff mnemonic: (1) introduction to the SAIF-IR mnemonic during the WSUGME orientation for all interns and (2) simulations during an objective simulated handoff evaluation activity for all postgraduate year (PGY) 1s and PGY 2s. In 2017, WSUGME innovated a direct observation tool, the TOC-CEX, for adoption by faculty at APRH to assess resident knowledge of and monitor their skill in using the SAIF-IR mnemonic in 3 primary care programs. The total number of possible participants was 138, and the actual number of individuals in the sample was 95. A majority (86%) of the observations during the study period were of PGY 1 residents, and thus the analysis reflects the ratings of 99% of all interns but only 69% of all possible residents. Results: WSUGME found that faculty use of a direct observation instrument in the clinical learning environment during 2017-2019 increased awareness and adoption of the SAIF-IR mnemonic among residents. Using a z-test of equal proportions on resident responses on an internal WSUGME survey, we found a significant rise in the percentage reporting yes to the question "Does your program have a mechanism for monitoring handoffs?" (χ2 [3]=23.6, P<0.0001) and in the percentage identifying SAIF-IR in response to the question "Does your program endorse a specific mnemonic for organizing the contents of a verbal handoff?" (χ2 [3]=45.0, P<0.0001). The increase from 2016 to 2017 is the result of the implementation of the TOC-CEX in the interim (question 1: χ2 [1]=12.4, P<0.0005; question 2: χ2 [1]=10.1, P<0.0025). Conclusion: Our research found that use of the TOC-CEX to monitor resident handoffs resulted in improved awareness and adoption of the SAIF-IR mnemonic in the clinical learning environment. Program leadership reported that the practice was both feasible and well accepted by residents, faculty, and the APRH chief medical officer as the TOC-CEX became a customary component of APRH organizational culture and was perceived as central to quality patient care.

Health Systems Science in Medical Education: Unifying the Components to Catalyze Transformation

Medical education exists in the service of patients and communities and must continually calibrate its focus to ensure the achievement of these goals. To close gaps in U.S. health outcomes, medical education is steadily evolving to better prepare providers with the knowledge and skills to lead patient- and systems-level improvements. Systems-related competencies, including high-value care, quality improvement, population health, informatics, and systems thinking, are needed to achieve this but are often curricular islands in medical education, dependent on local context, and have lacked a unifying framework. The third pillar of medical education-health systems science (HSS)-complements the basic and clinical sciences and integrates the full range of systems-related competencies. Despite the movement toward HSS, there remains uncertainty and significant inconsistency in the application of HSS concepts and nomenclature within health care and medical education. In this Article, the authors (1) explore the historical context of several key systems-related competency areas; (2) describe HSS and highlight a schema crosswalk between HSS and systems-related national competency recommendations, accreditation standards, national and local curricula, educator recommendations, and textbooks; and (3) articulate 6 rationales for the use and integration of a broad HSS framework within medical education. These rationales include: (1) ensuring core competencies are not marginalized, (2) accounting for related and integrated competencies in curricular design, (3) providing the foundation for comprehensive assessments and evaluations, (4) providing a clear learning pathway for the undergraduate-graduate-workforce continuum, (5) facilitating a shift toward a national standard, and (6) catalyzing a new professional identity as systems citizens. Continued movement toward a cohesive framework will better align the clinical and educational missions by cultivating the next generation of systems-minded health care professionals.

Assessment of Training in Health Disparities in US Internal Medicine Residency Programs

IMPORTANCE

Health disparities continue to exist despite the call to increase education of health care practitioners. An assessment of health disparities education has not been previously studied in a national cohort.

OBJECTIVE

To describe and compare the curriculum on health disparities from the perspective of program directors and perceptions of training among internal medicine residents.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional survey study used a survey of US internal medicine program directors, the 2015 Association of Program Directors in Internal Medicine annual survey, which included questions about health disparities curriculum, and a 1-time survey of US internal medicine residents that asked questions related to their training in health disparities on the American College of Physicians 2015 Internal Medicine In-Training Examination. All internal medicine program directors who were members of Association of Program Directors in Internal Medicine (368 of 396 accredited programs), and internal medicine residents who took the Internal Medicine In-Training Examination were eligible. Final analysis of the merged data set was completed in 2018.

EXPOSURES

Questions were included on the annual Association of Program Directors in Internal Medicine survey and the Internal Medicine In-Training Examination.

MAIN OUTCOMES AND MEASURES

Program directors reported on presence of health disparities curriculum, educational methods, quality, barriers and challenges to curriculum, and Clinical Learning Environment Review. Residents reported whether they received training and quality of the training in health disparities.

RESULTS

A total of 227 program directors (response rate, 61.7%) and 22 723 residents (response rate, 87.2%) responded to the surveys. A total of 90 program directors (39.6%) reported a curriculum in health disparities, but among these, only 16 program directors (17.8%) felt quality of their education was very good or excellent. In more than half of the programs (52 programs [55.9%]), outcomes of the curriculum were not measured. After merging, the combined data set included 18 883 residents from 366 APDIM member programs with 225 program director responses. Among these, 13 251 residents (70.2%) reported some training in caring for patients at risk for health disparities. Of residents who reported receiving training, 10 494 (79.2%) rated the quality as very good or excellent.

CONCLUSIONS AND RELEVANCE

These findings suggest that that despite the Clinical Learning Environment Review mandate and Accreditation Council for Graduate Medical Education required competencies for training in health disparities, relatively few internal medicine programs in the US provided educational curriculum. Additionally, the existence of health disparities curricula in internal medicine training programs was not associated with resident's perception of training or its quality.

General Internists as Change Agents: Opportunities and Barriers to Leadership in Health Systems and Medical Education Transformation

Health systems are increasingly engaging in mission development around the quadruple aim of patient experience of care, population health, cost of care, and work-life balance of clinicians. This integrated approach is closely aligned with the education principles and competencies of health systems science (HSS), which includes population health, high-value care, leadership, teamwork, collaboration, and systems thinking. Influenced by health outcomes research, the systems-based practice competency, and the Clinical Learning Environment Review, many medical schools and residency programs are taking on the challenge of comprehensively incorporating these HSS competencies into the education agenda. General internal medicine physicians, inclusive of hospitalists, geriatricians, and palliative and primary care physicians, are at the frontlines of this transformation and uniquely positioned to contribute to and lead health system transformation, role model HSS competencies for trainees, and facilitate the education of a new workforce equipped with HSS skills to accelerate change in healthcare. Although GIM faculty are positioned to be early adopters and leaders in evolving systems of care and education, professional development and changes with academic health systems are required. This Perspective article explores the conceptualization and opportunities to effectively link GIM with healthcare and medical education transformation.

Trainee and Program Director Perspectives on Meaningful Patient Attribution and Clinical Outcomes Data

BACKGROUND

The Accreditation Council for Graduate Medical Education specifies that trainees must receive clinical outcomes and quality benchmark data at specific levels related to institutional patient populations. Program directors (PDs) are challenged to identify meaningful data and provide them in formats acceptable to trainees.

OBJECTIVE

We sought to understand what types of patients, data/metrics, and data delivery systems trainees and PDs prefer for supplying trainees with clinical outcomes data.

METHODS

Trainees (n = 21) and PDs (n = 12) from multiple specialties participated in focus groups during academic year 2017-2018. They described key themes for providing clinical outcomes data to trainees.

RESULTS

Trainees and PDs differed in how they identified patients for clinical outcomes data for trainees. Trainees were interested in encounters where they felt a sense of responsibility or had autonomy/independent decision-making opportunities, continuity, or learned something new; PDs used broader criteria including all patients cared for by their trainees. Both groups thought trainees should be given trainee-level metrics and consistently highlighted the importance of comparison to peers and/or benchmarks. Both groups found value in "push" and "pull" data systems, although trainees wanted both, while PDs wanted one or the other. Both groups agreed that trainees should review data with specific faculty. Trainees expressed concern about being judged based on their patients' clinical outcomes.

CONCLUSIONS

Trainee and PD perspectives on which patients they would like outcomes data for differed, but they overlapped for types of metrics, formats, and review processes for the data.

A Novel, Web-Based Quality Improvement Platform to Address ACGME CLER Requirements

BACKGROUND

In 2014, the Accreditation Council for Graduate Medical Education (ACGME) formally mandated trainee (resident and fellow) participation in health care quality improvement (QI) projects as one of the Clinical Learning Environment Review (CLER) Pathways to Excellence. Subsequent national reviews showed large variations in how QI education is conducted, as well as a significant mismatch between educational and organizational goals.

OBJECTIVE

We developed a web-based platform to engage trainees in QI that better aligned with best practice methodology and matched identified institutional priorities.

METHODS

A needs assessment survey was distributed to trainees to understand the obstacles to compliance with ACGME QI requirements. Based on the results, a web-based clearinghouse, called the QI Platform, was developed and launched in July 2016, and utilization was analyzed in February 2019.

RESULTS

A total of 196 of 440 needs assessment surveys (45%) were completed. Themes extracted from surveys to identify barriers in QI participation included difficulties designing projects, lack of mentorship or expert support, and difficulty engaging an interprofessional team. Over 2.5 years, 151 projects were registered on the platform. Of these, 17 (11%) were collaborative entries. At the time of analysis, 166 of 437 trainees (38%) were listed as participants in active QI projects. A total of 22 projects were archived as complete, and 68 incomplete projects were reassigned to the "Ideas" section as works in progress after lead trainee graduation.

CONCLUSIONS

An institutional QI Platform clearinghouse for GME QI projects was feasible to develop and maintain, and it appeared acceptable to most GME programs and trainees for recording and tracking QI projects, and linking these to hospital QI priorities.

Assessing the Clinical Learning Environment in an Institution in the United Arab Emirates: The Resident Perspective

BACKGROUND

A new accreditation model in the United States has increased focus on the clinical learning environment (CLE). There is limited research on trainee perceptions of the CLE in international settings.

OBJECTIVE

We surveyed residents to obtain their perspective on the CLE at 1 sponsoring institution in the United Arab Emirates (UAE).

METHODS

We surveyed residents at Tawam Hospital, UAE, a sponsoring institution with 142 trainees, on their perspectives in the 6 focal areas of the US Clinical Learning Environment Review (CLER) to gather baseline information. We administered a 26-item questionnaire to residents through an audience response system in November 2018.

RESULTS

Of 100 residents in postgraduate year 2 and above, 72 (72%) responded. The perspective of the majority of respondents was favorable in the areas of reporting patient safety incidents, engaging in quality improvement activities, using a standardized form for care transition, and using professional guidelines for electronic health record documentation. In contrast, only half of the respondents perceived there is honesty in the reporting of duty hours, and only 36% felt the organization supported fatigue management. Other areas for improvement included residents' understanding of the concept of health disparities and activities to address health disparities.

CONCLUSIONS

Our findings suggest that in key focal areas related to patient safety, health care quality, care transitions, and professionalism, UAE residents have similar perceptions of their CLE as US trainees. Opportunities for improvement include duty hour reporting, fatigue mitigation, and addressing health disparities.