A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education

The systems-based practice competency: an innovative model to achieving class-wide proficiency in a pain medicine fellowship

One of the ACGME's six core competencies, systems-based practice (SBP), is difficult to interpret and developing proficiency over a one-year fellowship poses a challenge. Given the implications that successful SBP can have on pain medicine, it is especially important for fellows to focus on this competency during their training. Here, we propose a way to implement effective SBP into a pain medicine fellowship and the impact it may have within the larger health care system.

Which lecturers' characteristics facilitate the learning process? A qualitative study on students' perceptions in the rehabilitation sciences

BACKGROUND

In education, lecturers play a crucial role in facilitating students' learning process. However, only a few studies explored which lecturers' characteristics can facilitate this process in higher education for rehabilitation healthcare professionals. Starting from students' perspectives, our qualitative study investigated the lecturers' characteristics that facilitate students' learning process in the rehabilitation sciences.

METHODS

A qualitative interview study. We enrolled students attending the 2nd year of the Master of Science (MSc) degree in 'Rehabilitation Sciences of Healthcare Professions'. Different themes were generated following a 'Reflexive Thematic Analysis'.

RESULTS

Thirteen students completed the interviews. From their analysis, we generated five themes. Specifically, a lecturer that facilitates students' learning process should be: 1) 'A Performer who Interacts with the Classroom', 2) A Flexible Planner who Adopts Innovative Teaching Skills', 3) 'A Motivator who Embraces Transformational Leadership', 4) 'A Facilitator Who Encourages a Constructive Learning Context' and 5) 'A Coach who Devises Strategies to Reach Shared Learning Goals'.

CONCLUSIONS

The results of this study underscore the importance for lecturers in rehabilitation to cultivate a diverse set of skills drawn from the arts and performance, education, team building and leadership to facilitate students' learning process. By developing these skills, lecturers can design lessons that are worth attending not only for their relevant content but also for their value in human experience.

Targeted Public Health Training for Neurosurgeons: An Essential Task for the Prioritization of Neurosurgery in the Evolving Global Health Landscape

The gap between the tremendous burden of neurological disease requiring surgical management and the limited capacity for neurosurgical care has fueled the growth of the global neurosurgical movement. It is estimated that an additional 23 300 neurosurgeons are needed to meet the burden posed by essential cases across the globe. Initiatives to increase neurosurgical capacity through systems strengthening and workforce development are key elements in correcting this deficit. Building on the growing interest in global health among neurosurgical trainees, we propose the integration of targeted public health education into neurosurgical training, in both high-income countries and low- and middle-income countries. This effort will ensure that graduates possess the fundamental skillsets and experience necessary to participate in and lead capacity-building efforts in the developing countries. This additional public health training can also help neurosurgical residents to achieve the core competencies outlined by accreditation boards, such as the Accreditation Committee on Graduate Medical Education in the United States. In this narrative review, we describe the global burden of neurosurgical disease, establish the need and role for the global neurosurgeon, and discuss pathways for implementing targeted global public health education in the field of neurosurgery.

Perceptions of orthopaedic medicine students and their supervisors about practice-based learning: an exploratory qualitative study

BACKGROUND

Practice-based learning is crucial in forming appropriate strategies for improving learning among the medical students that support the country's understaffed health sector. Unsatisfactory learning consequently results in poor performance of students and poor quality of health care workforce in the long run. Exploring the perceptions about the current practice-based learning system and how to improve is thus vital. This study set out to explore the perceptions of Orthopaedic medicine students and their supervisors about practice-based learning at a tertiary training hospital.

METHODS

This was an exploratory phenomenological qualitative study that involved in-depth interviews among 10 Orthopedic students during their rotation in the emergency ward of Mulago hospital and 6 of their supervisors. Interviews were audio-recorded, transcribed, and then imported into Atlas ti 8.3 for analysis. The data were coded and grouped into themes relating to perceptions of practice-based learning, general inductive analysis was used. The general inductive approach involved condensing the raw textual data into a brief and summary format. The summarized format was then analyzed to establish clear links between the perceptions of practice-based learning and the summary findings derived from the raw data.

RESULTS

The mean age of the students was 23 ± 1.5 years. Four out of the six supervisors were Orthopaedic officers while the remaining two were principal Orthopaedic officers, four out of the six had a university degree while the other two were diploma holders. The main themes arising were hands-on skills, an unconducive learning environment, the best form of learning, and having an undefined training structure. Particularly, the perceptions included the presence of too many students on the wards during the rotation, frequent stock-outs of supplies for learning, and supervisors being overwhelmed caring for a large number of patients.

CONCLUSION

Barriers to satisfactory practice-based learning were overcrowding on the wards and insufficient training materials. To improve practice-based learning, adequate learning materials are required and the number of students enrolled needs to be appropriate for the student - supervisor ratio.

Essentials of neonatal-perinatal medicine fellowship: innovations in medical education

Due to the changing complex healthcare environment, educational innovation is essential to meet the needs of current and future neonatal-perinatal medicine (NPM) leaders. Greater clinical demands, decreased academic funding, and expanded graduate medical education program requirements have negatively impacted time for teaching and educational scholarship potentially limiting innovation in the field. By focusing on adult learning principles, embracing technology, and promoting collaboration, today's educators are preparing the next generation of neonatologists. Current innovations include regionalizing simulation boot camps, leveraging virtual learning to increase accessibility, developing niche training opportunities, and incorporating population health principles within existing quality initiatives. Areas in need of additional innovation include faculty and fellow development for teaching skills, expansion of educational networks, and dissemination and financial support of educational scholarship. These efforts and future innovations will require medical institutions and national NPM organizations to further invest in the medical educator as part of their missions.

Improving Medical Education: A Narrative Review

With the advancement in basic and clinical sciences, medical education is also constantly evolving. The Accreditation Council for Graduate Medical Education (ACGME) has endorsed six core competencies to improve teaching and learning. This narrative review was conducted after searching the article databases (PubMed, PubMed Central, Embase, and Scopus) about the core competencies such as medical knowledge (problem-based learning), interpersonal communication, patient care, professionalism, practice-based learning and improvement, and system-based care endorsed by ACGME. We included randomized and quasi-experimental trials, cohorts, and case-control studies in this narrative review. In a problem-based learning modality, a real-life scenario is allocated to a group of students. Studies have shown that it is more effectively demonstrated by a better post-test score, improved concentration, and application of knowledge. Interpersonal communication skills promote collaboration with interdisciplinary teams, work quality, and patient adherence to treatment. Professionalism is a human attribute that creates a pleasant work environment and is an essential trait that improves patients' adherence to treatment. In system-based care, patients are benefitted through a well-structured plan of care. Finally, in practice-based learning, medical trainees learn to systematically evaluate the pattern of care and practice the best modality to improve the overall patient care and physician satisfaction. These core competencies need to be incorporated into all levels of medical training.

Tracking Scan to Incision Time in Patients with Emergent Operative Traumatic Brain Injuries as a Measure for Systems-Based Practice in Neurosurgical Trainees

BACKGROUND

Evaluation of trainee performance remains a challenge in resident education, particularly for systems-based practice (SysBP) metrics including care coordination and interdisciplinary teamwork. Time to intervention is an important modifiable outcome variable in severe traumatic brain injury (TBI) and may serve as a trackable metric for SysBP evaluation.

METHODS

We retrospectively studied time from computed tomography head scan to surgical incision (CTH-INC, minutes) among neurosurgical trainees treating patients with emergently operative TBI as a proxy SysBP measure. Our institutional operative database was utilized to identify all emergent TBI cases between July 2015 and June 2020. Patients evaluated by program year (PGY)-2 residents proceeding directly to the operating room from the emergency department were included. Statistical analysis was performed using linear regression.

RESULTS

One hundred sixty-six cases triaged by 14 PGY-2 neurosurgical trainees were analyzed. Median CTH-INC was 104 minutes (interquartile range, 82-136 minutes). CTH-INC improved 20.1% over the academic year (95% confidence interval, 4.3%-33.2%, P = 0.015). Between the first and second 6-month periods, the rate of CTH-INC within 90 minutes (29% vs. 46%, P = 0.04) improved. On a per-individual PGY-2 basis, median CTH-INC ranged from 83-127 minutes, 25th percentile CTH-INC ranged from 62-108 minutes, and fastest CTH-INC ranged from 45-92 minutes.

CONCLUSIONS

CTH-INC is an objective and trackable proxy measure for evaluating SysBP during neurosurgical training. Use of CTH-INC or other time metrics in resident evaluations should not supersede the safe and effective delivery of patient care.

A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts?

BACKGROUND

With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes.

METHODS

A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts.

RESULTS

18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes.

CONCLUSION

This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.

Clinical Reasoning as a Core Competency

Diagnostic error is a challenging problem; addressing it effectively will require innovation across multiple domains of health care, including medical education. Diagnostic errors often relate to problems with clinical reasoning, which involves the cognitive and relational steps up to and including establishing a diagnostic and therapeutic plan with a patient. However, despite a call from the National Academies of Sciences for medical educators to improve the teaching and assessment of clinical reasoning, the creation of explicit, theory-informed clinical reasoning curricula, faculty development resources, and assessment tools has proceeded slowly in both undergraduate and graduate medical education. To accelerate the development of this critical element of health professions education and to promote needed research and innovation in clinical reasoning education, the Accreditation Council for Graduate Medical Education (ACGME) should revise its core competencies to include clinical reasoning. The core competencies have proven to be an effective means of expanding educational innovation across the United States and ensuring buy-in across a diverse array of institutions and disciplines. Reformulating the ACGME core competencies to include clinical reasoning would spark much-needed educational innovation and scholarship in graduate medical education, as well as collaboration across institutions in this vital aspect of physicianship, and ultimately, could contribute to a reduction of patient suffering by better preparing trainees to build individual, team-based, and system-based tools to monitor for and avoid diagnostic error.

Keystone: Exploring Pediatric Residents' Experiences in a Longitudinal Integrated Block

PROBLEM

Residency training in many specialties has traditionally been divided into short, discrete, single-specialty rotations. Although providing the learner with in-depth exposure to a specific discipline, educators have challenged this rotational model, citing problems with patient and team continuity and maladaptive coping. Longitudinal integrated clerkships, adopted by many medical schools, offer an alternative model and have demonstrated improved outcomes for students related to patient-centeredness, advocacy, and integration with teams. Despite this, longitudinal integrated training in residency is rare.

INTERVENTION

We developed a novel 3-month longitudinal integrated block for residents, called Keystone. The block combined 3 previously discrete, shorter rotations in developmental-behavioral pediatrics, advocacy, and emergency medicine into a longer and integrated experience. Within each week, the block utilized half-day sessions in the resident's primary care clinic, a new continuity Developmental Behavioral Pediatrics clinic where the resident worked with the same faculty preceptor and interprofessional team each week, shifts in the emergency department, and half-day sessions dedicated to clinic- and community-based advocacy activities.

CONTEXT

The context was a single, large pediatric urban residency program based at 2 university-affiliated hospitals, an academic freestanding children's hospital, and academic safety net hospital.

OUTCOME

Using a phenomenologic framework, we conducted interviews and a focus group discussion to explore residents' attitudes about the block; their perceptions regarding the block's impact on relationships with preceptors, peers, or patients; and the block's impact on learning and practice. Fourteen residents participated, 10 in interviews and 4 in the focus group discussion. Six themes emerged from thematic analysis: (a) the longitudinal nature of Keystone influenced professional relationships and as a result entrustment, (b) the longitudinal integrated design shaped engagement and learning, (c) flexibility promoted work-life balance and self-directed learning, (d) learners experienced time and space for professional identity development, (e) Keystone provided a unique opportunity to reclaim patient-centeredness, and (f) learners experienced important advantages and challenges related to the schedule.

LESSONS LEARNED

The longitudinal integrated nature of Keystone provided a novel structure for addressing important yet challenging educational goals in residency, including enhancing relationships, facilitating entrustment and engagement, encouraging patient-centeredness, and emphasizing the importance of self-directed learning.

Evaluation of early oral cavity cancer treatment quality at a single institution

OBJECTIVE

To evaluate the adherence to oral cavity quality guidelines endorsed by the American Head and Neck Society (AHNS) at a large tertiary care hospital.

METHODS

This retrospective study identified patients treated for early-stage oral tongue squamous cell carcinoma at a tertiary care hospital from 1992 to 2013. Patient charts were reviewed for 26 process quality measures and four key indicator process quality measures as endorsed by the AHNS. Patients were then grouped by diagnosis date either before (historical group, 1992-2007) or after (current treatment group, 2008-2013) the published process quality measures from the AHNS. Descriptive statistics were used to evaluate the rates of adherence for each process quality measure within the 2 groups.

RESULTS

Of the 57 patients identified, 29 were female (51%). The mean age was 62.3 years. A majority of the oral cavity cancers were stage I (59.6%), followed by stage II (35.1%) and stage III (5.3%). Compliance with the process quality measures was in the acceptable range in both cohorts. However, several areas demonstrated lower adherence in both cohorts. Statistically significant improvements were noted between the two cohorts, which showed a measurable improvement in adherence to process quality measures in several key areas over time.

CONCLUSION

Using the process quality measures proposed by the AHNS, adherence to the process quality measures for early-stage oral cavity cancer care at a tertiary care center was successfully evaluated. In general, good compliance with the proposed process quality measures was found and several areas for improvement were identified.

LEVEL OF EVIDENCE

2c Laryngoscope, 129:1816-1821, 2019.

Residents' Views on Research and Quality Improvement Training Can Guide Practice-Based Research Network Collaboration

Background and Objectives

This study was conducted by the West Virginia Practice-Based Research Network Learning Collaborative to assess research activity, confidence, and attitudes toward residency programs' research and quality improvement requirements and inform the integration of the state-wide practice-based research network (PBRN) as mentors to support practice transformation implementation initiatives across various resident training sites in West Virginia.

Methods

This pilot study assessed residents' attitudes regarding (1) research activity, (2) confidence and (3) requirements of their program in research and quality improvement training by using an anonymous survey, administered during regular residency meetings.

Results

Of the 68 residents,representing four DO and MD Family Medicine residency programs in West Virginia, 40 (58.8%) responded to the survey. About 64 percent of residents had worked on a quality improvement project, and more than half of residents (52.5%) submitted a research project for a competitive presentation within the most recent year. Sixty-five percent felt satisfied with the residency program's research and quality improvement curriculum. However, only 55 percent felt confident to perform a project and 52 percent submitted a project to a competitive forum.

Conclusion

Only half of the residents demonstrate activity and confidence in research and quality improvement. This shows an opportunity to assess current curriculums and provide new strategies to enhance their ability to conduct practice transformation initiatives.

Patient feedback for acupuncture practice improvement: A survey from Mayo Clinic

OBJECTIVE

To understand patient satisfaction with acupuncture appointment duration for the first and subsequent appointments in the plan of care, time spent with the acupuncture provider, willingness to pay for longer duration appointments, and lack of insurance coverage.

METHODS

Patients who were referred for an acupuncture consult to the Center for Integrative Medicine at Mayo Clinic were asked to participate in the survey in August 2014. Patients were given paper survey at time of appointment check-in and entered into Research Electronic Data Capture Survey database tool to reveal patient-reported outcomes.

RESULTS

In total, 104 of 329 patients receiving acupuncture treatment responded to the survey with a 32% response rate. Insight and recommended changes to the duration of the initial and subsequent appointments were obtained. Most patients reported that time spent with the provider (76 cases, 74%) and appointment length (74 cases, 68%) were "just right". Seventy cases (70%) of respondents indicated that they wanted longer treatment. Patients reported return appointments with duration of 30 min was not enough. Fifty-three patients (62%) were willing to pay for additional costs not covered by insurance.

CONCLUSIONS

We used patient feedback to assess the acupuncture practice in a complementary and integrative medicine program. Most patients referred for the acupuncture consult appeared to be satisfied with the current level of practice. These findings were used to establish a quality improvement plan that may be implemented to improve patient satisfaction with the acupuncture practice.

Flipping the Quality Improvement Classroom in Residency Education

PURPOSE

The flipped classroom (FC), in which instructional content is delivered before class with class time devoted to knowledge application, has the potential to engage residents. A Mayo Clinic Internal Medicine Residency Program study was conducted to validate an FC perception instrument (FCPI); determine whether participation improved FC perceptions; and determine associations between resident characteristics, change in quality improvement (QI) knowledge, and FC perception scores.

METHOD

All 143 internal medicine residents at Mayo Clinic, Rochester participated from 2014 to 2015; some experienced a flipped QI curriculum and others completed the traditional nonflipped course. The FCPI was developed, and factor analysis revealed an intuitive two-factor structure: preclass activity and in-class application. Residents were surveyed before and after the monthlong curriculum to measure changes in perception, and the QI Knowledge Assessment Tool was employed to measure knowledge improvement.

RESULTS

Postcourse FCPI scores significantly increased for three of the eight items. QI knowledge increased significantly among residents who experienced the FC compared with residents who completed the non-FC curriculum. Those without prior FC exposure demonstrated a significant increase in QI knowledge compared with those with previous FC experience. The FCPI had compelling validity evidence with improved scores after curriculum exposure and associations with greater engagement in online modules.

CONCLUSIONS

Residents who participated in the FC demonstrated improved QI knowledge compared with the control group. Residents valued the in-class application sessions more than the online component. These findings have important implications for graduate medical education as residency training programs increasingly use FC models.

Quality and Safety in Orthopaedics: Learning and Teaching at the Same Time: AOA Critical Issues

Increasing attention has been placed on providing higher quality and safer patient care. This requires the development of a new set of competencies to better understand and navigate the system and lead the orthopaedic team. While still trying to learn and develop these competencies, the academic orthopaedist is also expected to model and teach them.The orthopaedic surgeon must understand what is being measured and why, both for purposes of providing better care and to eliminate unnecessary expense in the system. Metrics currently include hospital-acquired conditions, "never events," and thirty-day readmission rates. More will undoubtedly follow.Although commitment and excellence at the individual level are essential, the orthopaedist must think at the systems level to provide the highest value of care. A work culture characterized by respect and trust is essential to improved communication, teamwork, and confidential peer review. An increasing number of resources, both in print and electronic format, are available for us to understand what we can do now to improve quality and safety.Resident education in quality and safety is a fundamental component of the systems-based practice competency, the Next Accreditation System, and the Clinical Learning Environment Review. This needs to be longitudinally integrated into the curriculum and applied parallel to the development of resident knowledge and skill, and will be best learned if resident learning is experiential and taught within a genuine culture of quality and safety.

GME Concentrations: A Collaborative Interdisciplinary Approach to Learner-Driven Education

BACKGROUND

Graduate medical education (GME) programs may struggle to provide the knowledge, skills, and experiences necessary to meet trainee career interests and goals beyond a clinical focus. Sponsoring institutions can partner with programs to deliver content not included in typical clinical experiences of GME programs.

OBJECTIVE

To perform a needs assessment and to develop, implement, and measure acceptability and feasibility of an institution-wide GME Concentrations curriculum.

METHODS

In response to the needs assessment, GME leadership developed 4 concentrations: (1) Resident-as-Teacher; (2) Patient Safety and Quality Improvement; (3) Law, Ethics, and Health Policy; and (4) Leaders in Medicine. We formed advisory committees that developed curricula for each concentration, including didactics, experiential learning, and individual project mentoring. Participants completed pre- and postassessments. We assessed feasibility and evaluated participant presentations and final projects.

RESULTS

Over the course of 3 years, 91 trainees (of approximately 1000 trainees each year) from 36 GME programs (of 82 accredited programs) have participated in the program. The number of participants has increased each year, and 22 participants have completed the program overall. Cost for each participant is estimated at $500. Participant projects addressed a variety of education and health care areas, including curriculum development, quality improvement, and national needs assessments. Participants reported that their GME Concentrations experience enhanced their training and fostered career interests.

CONCLUSIONS

The GME Concentrations program provides a feasible, institutionally based approach for educating trainees in additional interest areas. Institutional resources are leveraged to provide and customize content important to participants' career interests beyond their specialty.

Let's talk about integration: a study of students' understandings of integration

Today, the knowledge concerning clinical reasoning is advanced enough to translate into curriculum interventions such as an integrated curriculum, in which science theory and clinical practice can be interwoven effectively. However, the interpretations of what integration means differ and the purpose of this study was to elicit how students understand integration. This study was carried out using an interpretative perspective. Medical students, in their 2nd year of study, were asked to apply basic science knowledge from all previous courses to clinical cases in an examination. Subsequent to the examination, focus group interviews were conducted. The interviews were audio recorded, transcribed and analysed by the use of qualitative content analysis. The analysis revealed how students comprehended integration: as the creation of wholeness, as relating new knowledge to core concepts, as reasoning, as application and as collaboration between teachers. The five categories were linked to three dimensions: intra-personal, inter-personal and organizational, each of which resonates with different theories of how expertise is developed. The outcome of this study adds to our understanding of how students conceptualize integration. The categories of 'integration' drawn out by the study are helpful in promoting further discussion of how eliciting students' own reports of cognition and may help the ongoing design of curricula by putting students at the center of the curriculum design process.

A Novel Approach to Practice-Based Learning and Improvement Using a Web-Based Audit and Feedback Module

BACKGROUND

Systematically engaging residents in large programs in quality improvement (QI) is challenging.

OBJECTIVE

To coordinate a shared QI project in a large residency program using an online tool.

METHODS

A web-based QI tool guided residents through a 2-phase evaluation of performance of foot examinations in patients with diabetes. In phase 1, residents completed reviews of health records with online data entry. Residents were then presented with personal performance data relative to peers and were prompted to develop improvement plans. In phase 2, residents again reviewed personal performance. Rates of performance were compared at the program and clinic levels for each phase, with data presented for residents. Acceptability was measured by the number of residents completing each phase. Feasibility was measured by estimated faculty, programmer, and administrator time and costs.

RESULTS

Seventy-nine of 86 eligible residents (92%) completed improvement plans and reviewed 1471 patients in phase 1, whereas 68 residents (79%) reviewed 1054 patient charts in phase 2. Rates of performance of examination increased significantly between phases (from 52% to 73% for complete examination, P < .001). Development of the tool required 130 hours of programmer time. Project analysis and management required 6 hours of administrator and faculty time monthly.

CONCLUSIONS

An online tool developed and implemented for program-wide QI initiatives successfully engaged residents to participate in QI activities. Residents using this tool demonstrated improvement in a selected quality target. This tool could be adapted by other graduate medical education programs or for faculty development.

Assessing Effectiveness of a Geriatrics Rotation for Second-Year Internal Medicine Residents

BACKGROUND

Residents need to acquire an understanding of the biopsychosocial aspects of caring for older adults with chronic illness, along with effective use of interdisciplinary services inside and outside of the hospital.

OBJECTIVE

We expanded the geriatric medicine experience for second-year internal medicine residents and present the results of the first year's experience.

METHODS

We paired a mandatory rotation for postgraduate year-2 internal medicine residents (2 weeks of day and 1 week of night inpatient experience in the Acute Care for Elders Unit), and a 1-week outpatient systems-based practice experience with online modules and readings. Evaluation included a case presentation, an oral examination, a written questionnaire for all residents, and a global assessment of the residents' performance on the geriatrics portion of the 2012 In-Training Examination (ITE).

RESULTS

All residents passed their oral examination; there was little difference between classes in systems-based practice knowledge. More than 90% (21 of 23) of the residents who took the rotation reported that it left a lasting impression on how they would care for their patients. Mean ITE scores in geriatrics for all residents increased from 53% (versus 61% overall) in 2010 to 87% (versus 81%) in 2012, although they dropped to 69% (versus 82%) in 2013.

CONCLUSIONS

A rotation in geriatrics that is highly rated and covers both acute care and systems-based practice concepts is feasible for internal medicine residents. Residents did not learn detailed knowledge about specific programs for older adults, but clinical geriatrics knowledge improved.

The unmet need for a national surgical quality improvement curriculum: a systematic review

INTRODUCTION

The Accreditation Council for Graduate Medical Education Next Accreditation System will require general surgery training programs to demonstrate outstanding clinical outcomes and education in quality improvement (QI). The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative reports the results of a systematic review of the literature investigating the availability of a QI curriculum.

METHODS

Using defined search terms, a systematic review was conducted in Embase, PubMed, and Google Scholar (January 2000-March 2013) to identify a surgical QI curriculum. Bibliographies from selected articles and other relevant materials were also hand searched. Curriculum was defined as an organized program of learning complete with content, instruction, and assessment for use in general surgical residency programs. Two independent observers graded surgical articles on quality of curriculum presented.

RESULTS

Overall, 50 of 1155 references had information regarding QI in graduate medical education. Most (n = 24, 48%) described QI education efforts in nonsurgical fields. A total of 31 curricular blueprints were identified; 6 (19.4%) were specific to surgery. Targeted learners were most often post graduate year-2 residents (29.0%); only 6 curricula (19.4%) outlined a course for all residents within their respective programs. Plan, Do, Study, Act (n = 10, 32.3%), and Root Cause Analysis (n = 5, 16.1%) were the most common QI content presented, the majority of instruction was via lecture/didactics (n = 26, 83.9%), and only 7 (22.6%) curricula used validated tool kits for assessment.

CONCLUSION

Elements of QI curriculum for surgical education exist; however, comprehensive content is lacking. The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative will build on the high-quality components identified in our review and develop data-centered QI content to generate a comprehensive national QI curriculum for use in graduate surgical education.

Adopting a blended learning approach to teaching evidence based medicine: a mixed methods study

BACKGROUND

Evidence Based Medicine (EBM) is a core unit delivered across many medical schools. Few studies have investigated the most effective method of teaching a course in EBM to medical students. The objective of this study was to identify whether a blended-learning approach to teaching EBM is more effective a didactic-based approach at increasing medical student competency in EBM.

METHODS

A mixed-methods study was conducted consisting of a controlled trial and focus groups with second year graduate medical students. Students received the EBM course delivered using either a didactic approach (DID) to learning EBM or a blended-learning approach (BL). Student competency in EBM was assessed using the Berlin tool and a criterion-based assessment task, with student perceptions on the interventions assessed qualitatively.

RESULTS

A total of 61 students (85.9%) participated in the study. Competency in EBM did not differ between the groups when assessed using the Berlin tool (p = 0.29). Students using the BL approach performed significantly better in one of the criterion-based assessment tasks (p = 0.01) and reported significantly higher self-perceived competence in critical appraisal skills. Qualitative analysis identified that students had a preference for the EBM course to be delivered using the BL approach.

CONCLUSIONS

Implementing a blended-learning approach to EBM teaching promotes greater student appreciation of EBM principles within the clinical setting. Integrating a variety of teaching modalities and approaches can increase student self-confidence and assist in bridging the gap between the theory and practice of EBM.

Instituting systems-based practice and practice-based learning and improvement: a curriculum of inquiry

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) requires that training programs integrate system-based practice (SBP) and practice-based learning and improvement (PBLI) into internal medicine residency curricula. CONTEXT AND SETTING: We instituted a seminar series and year-long-mentored curriculum designed to engage internal medicine residents in these competencies.

METHODS

Residents participate in a seminar series that includes assigned reading and structured discussion with faculty who assist in the development of quality improvement or research projects. Residents pursue projects over the remainder of the year. Monthly works in progress meetings, protected time for inquiry, and continued faculty mentorship guide the residents in their project development. Trainees present their work at hospital-wide grand rounds at the end of the academic year. We performed a survey of residents to assess their self-reported knowledge, attitudes and skills in SBP and PBLI. In addition, blinded faculty scored projects for appropriateness, impact, and feasibility.

OUTCOMES

We measured resident self-reported knowledge, attitudes, and skills at the end of the academic year. We found evidence that participants improved their understanding of the context in which they were practicing, and that their ability to engage in quality improvement projects increased. Blinded faculty reviewers favorably ranked the projects' feasibility, impact, and appropriateness. The 'Curriculum of Inquiry' generated 11 quality improvement and research projects during the study period. Barriers to the ongoing work include a limited supply of mentors and delays due to Institutional Review Board approval. Hospital leadership recognizes the importance of the curriculum, and our accreditation manager now cites our ongoing work.

CONCLUSIONS

A structured residency-based curriculum facilitates resident demonstration of SBP and practice-based learning and improvement. Residents gain knowledge and skills though this enterprise and hospitals gain access to trainees who help to solve ongoing problems and meet accreditation requirements.

Perspectives on the changing healthcare system: teaching systems-based practice to medical residents

PURPOSE

The Accreditation Council for Graduate Medical Education restructured its accreditation system to be based on educational outcomes in six core competencies. Systems-based practice is one of the six core competencies. The purpose of this report is to describe Weill Cornell Medical College's Internal Medicine Residency program curriculum for systems-based practice (SBP) and its evaluation process.

METHODS

To examine potential outcomes of the POCHS curriculum, an evaluation was conducted, examining participants': (1) knowledge gain; (2) course ratings; and (3) qualitative feedback.

RESULTS

On average, there was a 19 percentage point increase in knowledge test scores for all three cohorts. The course was rated overall highly, receiving an average of 4.6 on a 1-5 scale. Lastly, the qualitative comments supported that the material is needed and valued.

CONCLUSION

The course, entitled Perspectives on the Changing Healthcare System (POCHS) and its evaluation process support that systems-based practice is crucial to residency education. The course is designed not only to educate residents about the current health care system but also to enable them to think critically about the risk and benefits of the changes. POCHS provides a framework for teaching and assessing this competency and can serve as a template for other residency programs looking to create or restructure their SBP curriculum.

A longitudinal, experiential quality improvement curriculum meeting ACGME competencies for geriatrics fellows: lessons learned

Quality improvement (QI) initiatives are critical in the care of older adults who are more vulnerable to substandard care. QI education meets aspects of core Accreditation Council of Graduate Medical Education competencies and prepares learners for the rising focus on performance measurement in health care. The authors developed, implemented, and evaluated a QI curriculum for geriatrics fellows. The evidence-based curriculum included didactics and a fellow-led QI intervention based on audit and feedback through the Practice Improvement Module in Care of the Vulnerable Elderly. QI knowledge, attitudes, and behaviors were assessed before and after the improvement project. Fellows' knowledge of QI improved (p = .0156), but behavior did not change significantly across a short-term improvement project. A structured focus group with fellows revealed themes of accountability and the importance of interprofessional teamwork in QI. QI education for geriatrics fellows can be feasible, well received, and prepare future physician leaders for patient-centered care, performance measurement, and effecting systems change.

ACGME core competency training, mentorship, and research in surgical subspecialty fellowship programs

OBJECTIVE

To determine the perceived effectiveness of surgical subspecialty training programs in teaching and assessing the 6 ACGME core competencies including research.

DESIGN

Cross-sectional survey.

SETTING

ACGME approved training programs in pediatric urology and colorectal surgery.

PARTICIPANTS

Program Directors and recent trainees (2007-2009).

RESULTS

A total of 39 program directors (60%) and 57 trainees (64%) responded. Both program directors and recent trainees reported a higher degree of training and mentorship (75%) in patient care and medical knowledge than the other core competencies (p<0.0001). Practice based learning and improvement, interpersonal and communication, and professionalism training were perceived effective to a lesser degree. Specifically, in the areas of teaching residents and medical students and team building, program directors, compared with recent trainees, perceived training to be more effective, (p = 0.004, p = 0.04). Responses to questions assessing training in systems based practice ubiquitously identified a lack of training, particularly in financial matters of running a practice. Although effective training in research was perceived as lacking by recent trainees, 81% reported mentorship in this area. According to program directors and recent trainees, the most effective method of teaching was faculty supervision and feedback. Only 50% or less of the recent trainees reported mentorship in career planning, work-life balance, and job satisfaction.

CONCLUSIONS

Not all 6 core competencies and research are effectively being taught in surgery subspecialty training programs and mentorship in areas outside of patient care and research is lacking. Emphasis should be placed on faculty supervision and feedback when designing methods to better incorporate all 6 core competencies, research, and mentorship.

Using lean methodology to teach quality improvement to internal medicine residents at a safety net hospital

The overall objective of this initiative was to develop a quality improvement (QI) curriculum using Lean methodology for internal medicine residents at Boston Medical Center, a safety net academic hospital. A total of 90 residents and 8 School of Public Health students participated in a series of four, 60- to 90-minute interactive and hands-on QI sessions. Seventeen QI project plans were created and conducted over a 4-month period. The curriculum facilitated internal medicine residents' learning about QI and development of positive attitudes toward QI (assessed using pre- and post-attitude surveys) and exposed them to an interprofessional team structure that duplicates future working relationships. This QI curriculum can be an educational model of how health care trainees can work collaboratively to improve health care quality.

Reshaping orthopaedic resident education in systems-based practice

BACKGROUND

Despite advances in understanding the "systems-based practice" competency in resident education, this topic has remained difficult to teach, assess, and document. The goal of this study was to perform a needs assessment and an analysis of the current state of systems-based practice education in orthopaedic residency programs across the U.S. and within our own institution.

METHODS

A sample of orthopaedic educators and residents from across the U.S. who were attending the 2010 American Orthopaedic Association (AOA) Effective Orthopaedic Educator Course, AOA Resident Leadership Forum, and AOA Council of Residency Directors meeting were surveyed to determine (1) which aspects of systems-based practice, if any, were being taught; (2) how systems-based practice is being taught; and (3) how residency programs are assessing systems-based practice. In addition, an in-depth case study of these issues was performed by means of seven semi-structured focus group sessions with diverse stakeholders who participated in the care of musculoskeletal patients at the authors' institution. A quantitative approach was used to analyze the survey data. The focus group data were analyzed with procedures associated with grounded theory, relying on a constant comparative method to develop salient themes arising from the discussion.

RESULTS

"Clinical observation" (33%) and "didactic case-based learning" (23%) were reported by the survey respondents as the most commonly used teaching methods, but specific topics were taught inconsistently. Competency assessment was reported to occur infrequently, and 36% of respondents reported that systems-based practice areas were not being assessed by any methods. The focus group discussions emphasized the need for standardized experiential learning that was closely linked to the patient's perspective. Orthopaedic faculty members were uncomfortable with their knowledge of this competency and their ability to teach and assess it.

CONCLUSIONS

Teaching the systems-based practice competency occurs inconsistently, and formal assessment occurs infrequently. In addition to formal teaching, learning systems-based practice will be best achieved experientially and from the patient's perspective.

Use of a structured template to facilitate practice-based learning and improvement projects

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to meet and demonstrate outcomes across 6 competencies. Measuring residents' competency in practice-based learning and improvement (PBLI) is particularly challenging.

PURPOSE

We developed an educational tool to meet ACGME requirements for PBLI. The PBLI template helped programs document quality improvement (QI) projects and supported increased scholarly activity surrounding PBLI learning.

METHODS

We reviewed program requirements for 43 residency and fellowship programs and identified specific PBLI requirements for QI activities. We also examined ACGME Program Information Form responses on PBLI core competency questions surrounding QI projects for program sites visited in 2008-2009. Data were integrated by a multidisciplinary committee to develop a peer-protected PBLI template guiding programs through process, documentation, and evaluation of QI projects. All steps were reviewed and approved through our GME Committee structure.

RESULTS

An electronic template, companion checklist, and evaluation form were developed using identified project characteristics to guide programs through the PBLI process and facilitate documentation and evaluation of the process. During a 24 month period, 27 programs have completed PBLI projects, and 15 have reviewed the template with their education committees, but have not initiated projects using the template.

DISCUSSION

The development of the tool generated program leaders' support because the tool enhanced the ability to meet program-specific objectives. The peer-protected status of this document for confidentiality and from discovery has been beneficial for program usage. The document aggregates data on PBLI and QI initiatives, offers opportunities to increase scholarship in QI, and meets the ACGME goal of linking measures to outcomes important to meeting accreditation requirements at the program and institutional level.

Educational Experiences Residents Perceive As Most Helpful for the Acquisition of the ACGME Competencies

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) requires physicians in training to be educated in 6 competencies considered important for independent medical practice. There is little information about the experiences that residents feel contribute most to the acquisition of the competencies.

OBJECTIVE

To understand how residents perceive their learning of the ACGME competencies and to determine which educational activities were most helpful in acquiring these competencies.

METHOD

A web-based survey created by the graduate medical education office for institutional program monitoring and evaluation was sent to all residents in ACGME-accredited programs at the David Geffen School of Medicine, University of California-Los Angeles, from 2007 to 2010. Residents responded to questions about the adequacy of their learning for each of the 6 competencies and which learning activities were most helpful in competency acquisition.

RESULTS

We analyzed 1378 responses collected from postgraduate year-1 (PGY-1) to PGY-3 residents in 12 different residency programs, surveyed between 2007 and 2010. The overall response rate varied by year (66%-82%). Most residents (80%-97%) stated that their learning of the 6 ACGME competencies was "adequate." Patient care activities and observation of attending physicians and peers were listed as the 2 most helpful learning activities for acquiring the 6 competencies.

CONCLUSION

Our findings reinforce the importance of learning from role models during patient care activities and the heterogeneity of learning activities needed for acquiring all 6 competencies.

Perspective: a road map for academic departments to promote scholarship in quality improvement and patient safety

The fields of quality improvement and patient safety (QI/PS) continue to grow with greater attention and awareness, increased mandates and incentives, and more research. Academic medical centers and their academic departments have a long-standing tradition for innovation and scholarship within a multifaceted mission to provide patient care, educate the next generation, and conduct research. Academic departments are well positioned to lead the science, education, and application of QI/PS efforts nationally. However, meaningful engagement of faculty and trainees to lead this work is a major barrier. Understanding and developing programs that foster QI/PS work while also promoting a scholarly focus can generate the incentives and acknowledgment to help elevate QI/PS into the academic mission. Academic departments should define and articulate a QI/PS strategy, develop individual and departmental capacity to lead scholarly QI/PS programs, streamline and support access to data, share information and improve collaboration, and recognize and elevate academic success in QI/PS. A commitment to these goals can also serve to cultivate important collaborations between academic departments and their respective medical centers, divisions, and training programs. Ultimately, the elevation of QI/PS into the academic mission can improve the quality and safety of our health care delivery systems.

A didactic and experiential quality improvement curriculum for psychiatry residents

BACKGROUND

Quality improvement (QI) education in residency training is important and necessary for accreditation. Although the literature on this topic has been growing, some specialties, in particular psychiatry, have been underrepresented.

METHODS

We developed a didactic and experiential QI curriculum within a US psychiatry residency program that included a seminar series and development of QI projects. Evaluation included resident knowledge using the Quality Improvement Knowledge Application Tool, implementation of resident QI projects, and qualitative and quantitative satisfaction with the curriculum.

RESULTS

Our curriculum significantly improved QI knowledge in 2 cohorts of residents (N  =  16) as measured by the Quality Improvement Knowledge Application Tool. All resident QI projects (100%) in the first cohort were implemented. Residents and faculty reported satisfaction with the curriculum.

CONCLUSIONS

Our curriculum incorporated QI education through didactic and experiential learning in a moderately sized US psychiatry residency program. Important factors included a longitudinal experience with protected time for residents to develop QI projects and a process for developing faculty competence in QI. Further studies should use a control group of residents and examine interprofessional QI curricula.

Systems-based practice: Summary of the 2010 Council of Emergency Medicine Residency Directors Academic Assembly Consensus Workgroup--teaching and evaluating the difficult-to-teach competencies

OBJECTIVES

The development of robust Accreditation Council for Graduate Medical Education (ACGME) systems-based practice (SBP) training and validated evaluation tools has been generally challenging for emergency medicine (EM) residency programs. The purpose of this paper is to report the results of a consensus workgroup session of the 2010 Council of Emergency Medicine Residency Directors (CORD) Academic Assembly with the following objectives: 1) to discuss current and preferred local and regional methods for teaching and assessing SBP and 2) to develop consensus within the CORD community using the modified Delphi method with respect to EM-specific SBP domains and link these domains to specific SBP educational and evaluative methods.

METHODS

Consensus was developed using a modified Delphi method. Previously described taxonomy generation methodology was used to create a SBP taxonomy of EM domain-specific knowledge, skills, and attitudes (KSA). The steps in the process consisted of: 1) an 11-question preconference survey, 2) a vetting process conducted at the 2010 CORD Academic Assembly, and 3) the development and ranking of domain-specific SBP educational activities and evaluation criteria for the specialty of EM.

RESULTS

Rank-order lists were created for preferred SBP education and evaluation methods. Expert modeling, informal small group discussion, and formal small group activities were considered to be the optimal methods to teach SBP. Kruskal-Wallis testing revealed that these top three items were rated significantly higher than self-directed learning projects and lectures (p = 0.0317). Post hoc test via permutation testing revealed that the difference was significant between expert modeling and formal small group activity (adjusted p = 0.028), indicating that expert modeling was rated significantly higher than formal small group activity. Direct observation methods were the preferred methods for evaluation. Multiple barriers to training and evaluation were elucidated. We developed a consensus taxonomy of domains that were felt to be most essential and reflective of the practice of EM: multitasking, disposition, and patient safety. Learning formats linked to the domains were created and specific examples of local best practices collected. Domain-specific anchors of observable actions for the three domains were created.

CONCLUSIONS

This consensus process resulted in the development of a taxonomy of EM-specific domains for teaching and observable tasks for evaluating SBP. The concept of SBP is interlinked with the other general competencies and difficult to separate. Rather than develop specific SBP evaluation tools to measure the competency directly, SBP competency evaluation should be considered one element of a coordinated effort to teach and evaluate the six ACGME general competencies.

Systems-based practice in graduate medical education: systems thinking as the missing foundational construct

BACKGROUND

Since 2001, residencies have struggled with teaching and assessing systems-based practice (SBP). One major obstacle may be that the competency alone is not sufficient to support assessment. We believe the foundational construct underlying SBP is systems thinking, absent from the current Accreditation Council for Graduate Medical Education competency language.

SUMMARY

Systems thinking is defined as the ability to analyze systems as a whole. The purpose of this article is to describe psychometric issues that constrain assessment of SBP and elucidate the role of systems thinking in teaching and assessing SBP.

CONCLUSION

Residency programs should incorporate systems thinking models into their curricula. Trainees should be taught to understand systems at an abstract level, in order to analyze their own healthcare systems, and participate in quality and patient safety activities. We suggest that a developmental trajectory for systems thinking be developed, similar to the model described by Dreyfus and Dreyfus.

Teaching patient safety: conference proceedings and consensus statements of the Millennium Conference 2009

PURPOSE

The 2003 Institute of Medicine's report "Health Professions Education: A Bridge to Quality" argued for the education of health professionals in patient safety. In response to this call, a number of organizations and institutions have developed frameworks and curricula that provide the educational foundation essential for learning about patient safety. However, there is limited guidance on strategies for implementation of training programs in patient safety.

SUMMARY

We convened the "Millennium Conference 2009: Patient Safety--Implications for Teaching in the 21st Century" to develop concrete approaches to teach patient safety in undergraduate and graduate medical education. We selected 9 medical schools through a competitive application process to participate as school teams. We led attendees through structured discussions on three topics: (a) promoting a culture of patient safety, (b) implementing patient safety content into preexisting curricula, and (c) providing faculty development. School teams also met to refine their current local initiatives in patient safety teaching.

CONCLUSIONS

A group of committed stakeholders gathered to collectively consider strategies for the integration of patient safety education into undergraduate and graduate medical education. The recommendations from this conference proceed from consensus reached by the participants.

Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills

BACKGROUND

We developed a practice-based learning and improvement (PBLI) curriculum to address important gaps in components of content and experiential learning activities through didactics and participation in systems-level quality improvement projects that focus on making changes in health care processes.

METHODS

We evaluated the impact of our curriculum on resident PBLI knowledge, self-efficacy, and application skills. A quasi-experimental design assessed the impact of a curriculum (PBLI quality improvement systems compared with non-PBLI) on internal medicine residents' learning during a 4-week ambulatory block. We measured application skills, self-efficacy, and knowledge by using the Systems Quality Improvement Training and Assessment Tool. Exit evaluations assessed time invested and experiences related to the team projects and suggestions for improving the curriculum.

RESULTS

The 2 groups showed differences in change scores. Relative to the comparison group, residents in the PBLI curriculum demonstrated a significant increase in the belief about their ability to implement a continuous quality improvement project (P  =  .020), comfort level in developing data collection plans (P  =  .010), and total knowledge scores (P < .001), after adjusting for prior PBLI experience. Participants in the PBLI curriculum also demonstrated significant improvement in providing a more complete aim statement for a proposed project after adjusting for prior PBLI experience (P  =  .001). Exit evaluations were completed by 96% of PBLI curriculum participants who reported high satisfaction with team performance.

CONCLUSION

Residents in our curriculum showed gains in areas fundamental for PBLI competency. The observed improvements were related to fundamental quality improvement knowledge, with limited gain in application skills. This suggests that while heading in the right direction, we need to conceptualize and structure PBLI training in a way that integrates it throughout the residency program and fosters the application of this knowledge and these skills.

Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference

In this article, we outline criteria for good assessment that include: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability. Many of the criteria have been described before and we continue to support their importance here. However, we place particular emphasis on the catalytic effect of the assessment, which is whether the assessment provides results and feedback in a fashion that creates, enhances, and supports education. These criteria do not apply equally well to all situations. Consequently, we discuss how the purpose of the test (summative versus formative) and the perspectives of stakeholders (examinees, patients, teachers-educational institutions, healthcare system, and regulators) influence the importance of the criteria. Finally, we offer a series of practice points as well as next steps that should be taken with the criteria. Specifically, we recommend that the criteria be expanded or modified to take account of: (1) the perspectives of patients and the public, (2) the intimate relationship between assessment, feedback, and continued learning, (3) systems of assessment, and (4) accreditation systems.