"The mirror" and "the village": a new method for teaching practice-based learning and improvement and systems-based practice

Systems-Based Practice: Expert Perspectives on the Origin and Evolution of an Ambiguous Competency

PURPOSE

Systems-based practice (SBP) has been a core competency in graduate medical education in the United States since 1999, but it has been difficult to operationalize in residency programs due to its conceptual ambiguity. The authors explored the historical origin and subsequent development of the SBP competency from the perspective of individuals who were influential across critical phases of its implementation and ensuing development. The goal of this study was to elicit the history of SBP from the perspective of individuals who have expertise in it and to use those findings to inform the current SBP construct.

METHOD

Between March and July 2021, 24 physicians, nurses, educators, and leaders in the field of SBP were individually interviewed about the origin and meaning of SBP as practiced in U.S. medical education using a semistructured guide. Individuals were selected based upon their influence on the origin or evolution of the SBP competency. Data were iteratively collected and analyzed using real-time analytic memos, regular adjudication sessions with the research team, and thematic analysis. Researchers identified themes from participants' perspectives and agreed upon the final results and quotations.

RESULTS

Five themes were identified: SBP has many different definitions, SBP was intentionally designed to be vague, systems thinking was identified as the foundation of the SBP competency, the 6 core competencies established in the United States by the Accreditation Council for Graduate Medical Education were developed to be interdependent, and the SBP and practice-based learning and improvement competencies are uniquely related and synergistic.

CONCLUSIONS

Interview data indicate that since its inception, SBP has been a nuanced and complex competency, resulting in a lack of mutually shared understanding among stakeholders. This deliberate historical examination of expert perspectives provides insight into specific areas for improving how SBP is taught and learned.

A Systematic Review on the Effectiveness of Systems-Based Practice Curricula in Health Professions Education

This systematic review aims to evaluate the effectiveness of systems-based practice (SBP) curricula from the perspective of health professions students and workers. A total of 8468 citations were sourced from six electronic databases and manual searches conducted independently by two researchers, of which 44 studies were eventually included. A meta-analysis using a random effects model and a meta-synthesis using the thematic synthesis approach were conducted. Most studies targeted medical students, residents, and resident physicians from various clinical specialties. Almost half of all studies focused on didactic or knowledge-based interventions to teach SBP. About a third of all studies measured non-self-evaluated knowledge change, clinical abilities, and clinical outcomes. Both meta-analysis and meta-synthesis results revealed positive outcomes of increased knowledge of SBP, increased recognition of SBP as a core competency in one's profession, and increased application of SBP knowledge in one's profession. Meta-synthesis results also revealed negative outcomes at the institutional and teacher/health professions level. This review highlights the importance of SBP education and supports the effectiveness of SBP curricula. There is a need to address the negative outcomes at the institutional and teacher/health professions level. Moreover, future studies could investigate the integration of self-assessment outcomes with comparison to some external standard.

A resident-led project to improve documentation of overweight and obesity in a primary care clinic

Background: Although the prevalence of overweight and obesity (OW/OB) has increased in the last three decades, studies show that these conditions are sub-optimally documented by physicians. Health information technology tools have varying effects on improving documentation of OW/OB but often have to be complemented with other interventions to be effective.

Objective: Upon identifying low rates of documentation of diagnoses of overweight and obesity by resident and attending physicians, despite the use of an electronic health record (EHR) with automated BMI calculations, we performed a quality improvement (QI) project to improve documentation of these diagnoses for patients in our community hospital primary care clinic.

Methods: The EHR was reviewed to determine documentation rates by resident and attending physicians between 1 March 2018 and 31 September 2018. We collected pre-intervention data, developed interventions, and implemented tests of change using Plan-Do-Study-Act (PDSA) cycles to improve documentation of OW/OB.

Results: Documentation of overweight and obesity diagnoses increased from a baseline of 46% to 79% over a 20-week period after initiation of our project.

Conclusion: We demonstrate the successful implementation of resident-led, multi-faceted interventions in a team-based QI project to optimize documentation of OW/OB in the EHR.

A Qualitative Study of Independent Home Visits by Hospice Fellows: Addressing Gaps in ACGME Milestones by Fostering Reflection and Self-Assessment

BACKGROUND

With the rapid growth in the number of fellowship programs in Hospice and Palliative Medicine (HPM), many are in the process of developing ways to demonstrate that fellows are attaining educational milestones. Reflection and self-assessment are key components of 2 Accreditation Council for Graduate Medical Education (ACGME) competencies, practice-based learning and improvement, and systems-based practice, which have both been historically challenging to learn and assess.

OBJECTIVE

This article describes results of a content analysis of narrative data collected from HPM fellows' self-assessments as they performed hospice home visits independently in a new clinical rotation.

DESIGN

This was a prospective qualitative study.

SETTINGS/PARTICIPANTS

Eight fellows completed 217 unsupervised hospice home visits from 2014 to 2016.

MEASUREMENTS

Fellows completed weekly self-assessment forms, which captured both clinical visit information and practice data elicited from responses to open-ended reflection prompts.

RESULTS

Analysis of 29 self-assessment forms generated 6 themes: patient- and family-centered care, self-efficacy, systems-based care, commitment to doing their best, catalyst for professional growth, and purpose and meaning in work. The fellows recognized numerous barriers distinct to providing care in homes. All fellows felt prepared to perform home visits throughout the rotation and after training.

CONCLUSIONS

Narrative data collected during the independent home visit rotation provided evidence that HPM fellows detected gaps in their performance, planned for practice improvements in subsequent visits, and valued working within an interprofessional team. Built-in opportunities for fellows to reflect during training are critical in meeting ACGME milestones, and are integral to their professional development.

Integrating Training in Quality Improvement and Health Equity in Graduate Medical Education: Two Curricula for the Price of One

A recent call to address health care disparities has come from the Accreditation Council for Graduate Medical Education's (ACGME's) Clinical Learning Environment Review (CLER) program. The CLER program aspires that faculty and residents will identify the disparities among the patient populations they serve and engage in quality improvement (QI) activities designed to address them. In this Perspective, the authors provide a framework for integrating QI and health equity principles in graduate medical education to meet these ACGME expectations. The authors illustrate their four-step framework by describing a faculty development workshop that provides strategies and tools for embedding equity into existing QI educational efforts and using QI methods to address equity challenges. Using examples, the authors outline how medical educators can begin to integrate QI and equity initiatives to address health care disparities and involve their residents/fellows in the process. In addition, the authors emphasize the importance of applying an equity lens to QI interventions and of recognizing that QI initiatives will have different impacts on outcomes depending on the patient population. The authors conclude by discussing the need for institutional leadership to build capacity and training to improve data collection and reporting of quality metrics by demographic variables; provide resources to disseminate lessons learned; support faculty development to teach and mentor trainees through equity-related QI work; and prioritize time in the curriculum for learners to participate in equity improvement activities.

Accreditation Council for Graduate Medical Education Core Competencies at a Community Teaching Hospital: Is There a Gap in Awareness?

CONTEXT

Reports evaluating faculty knowledge of the Accreditation Council for Graduate Medical Education (ACGME) core competencies in community hospitals without a dedicated residency program are uncommon.

OBJECTIVE

Faculty evaluation regarding knowledge of ACGME core competencies before a residency program is started.

DESIGN

Physicians at the Kaiser Permanente Fontana Medical Center (N = 480) were surveyed for their knowledge of ACGME core competencies before starting new residency programs.

MAIN OUTCOME MEASURES

Knowledge of ACGME core competencies.

RESULTS

Fifty percent of physicians responded to the survey, and 172 (71%) of respondents were involved in teaching residents. Of physicians who taught residents and had complete responses (N = 164), 65 (39.7%) were unsure of their knowledge of the core competencies. However, most stated that they provided direct teaching to residents related to the knowledge, skills, and attitudes stated in each of the 6 competencies as follows: medical knowledge (96.3%), patient care (95.7%), professionalism (90.7%), interpersonal and communication skills (86.3%), practice-based learning (85.9%), and system-based practice (79.6%). Physician specialty, years in practice (1-10 vs > 10), and number of rotations taught per year (1-6 vs 7-12) were not associated with knowledge of the competencies (p > 0.05); however, full-time faculty (teaching 10-12 rotations per year) were more likely to provide competency-based teaching.

CONCLUSION

Objective assessment of faculty awareness of ACGME core competencies is essential when starting a residency program. Discrepancy between knowledge of the competencies and acclaimed provision of competency-based teaching emphasizes the need for standardized teaching methods that incorporate the values of these competencies.

The Systems SOAP Note: A Systems Learning Tool

OBJECTIVE

Systems-based practice (SBP) is the only Accreditation Council for Graduate Medical Education (ACGME) competency concerned with public health and is relatively neglected in residency curricula. A tool was developed and pilot-tested to improve SBP learning on inpatient psychiatry rotations.

METHODS

A four-step approach was used: (1) literature review, (2) expert consultation, (3) tool development, and (4) pilot testing on four cases and evaluation for completion time and preliminary efficacy.

RESULTS

Out of 51 SBP articles, six (12%) focused on psychiatric residency programs, and none had a practical SBP learning tool. The "systems SOAP (subjective, objective, assessment, plan) note" (S-SOAP) was structured after a clinical SOAP note and was easy to use (mean completion time = 60 min), and residents self-reported more insight into systems issues.

CONCLUSIONS

The S-SOAP tool was effectively integrated into clinical experience and provided insight into systemic complexities. Future research should assess SBP knowledge acquisition after the use of such tools.

The Action Research Program: Experiential Learning in Systems-Based Practice for First-Year Medical Students

PROBLEM

Systems-based practice focuses on the organization, financing, and delivery of medical services. The American Association of Medical Colleges has recommended that systems-based practice be incorporated into medical schools' curricula. However, experiential learning in systems-based practice, including practical strategies to improve the quality and efficiency of clinical care, is often absent from or inconsistently included in medical education.

INTERVENTION

A multidisciplinary clinician and nonclinician faculty team partnered with a cardiology outpatient clinic to design a 9-month clerkship for 1st-year medical students focused on systems-based practice, delivery of clinical care, and strategies to improve the quality and efficiency of clinical operations. The clerkship was called the Action Research Program. In 2013-2014, 8 trainees participated in educational seminars, research activities, and 9-week clinic rotations. A qualitative process and outcome evaluation drew on interviews with students, clinic staff, and supervising physicians, as well as students' detailed field notes.

CONTEXT

The Action Research Program was developed and implemented at the University of California, San Francisco, an academic medical center in the United States. All educational activities took place at the university's medical school and at the medical center's cardiology outpatient clinic.

OUTCOME

Students reported and demonstrated increased understanding of how care delivery systems work, improved clinical skills, growing confidence in interactions with patients, and appreciation for patients' experiences. Clinicians reported increased efficiency at the clinic level and improved performance and job satisfaction among medical assistants as a result of their unprecedented mentoring role with students. Some clinicians felt burdened when students shadowed them and asked questions during interactions with patients. Most student-led improvement projects were not fully implemented.

LESSONS LEARNED

The Action Research Program is a small pilot project that demonstrates an innovative pairing of experiential and didactic training in systems-based practice. Lessons learned include the need for dedicated time and faculty support for students' improvement projects, which were the least successful aspect of the program. We recommend that future projects aiming to combine clinical training and quality improvement projects designate distinct blocks of time for trainees to pursue each of these activities independently. In 2014-2015, the University of California, San Francisco School of Medicine incorporated key features of the Action Research Program into the standard curriculum, with plans to build upon this foundation in future curricular innovations.

Resident Self-Assessment and Learning Goal Development: Evaluation of Resident-Reported Competence and Future Goals

OBJECTIVE

To determine incidence of learning goals by competency area and to assess which goals fall into competency areas with lower self-assessment scores.

METHODS

Cross-sectional analysis of existing deidentified American Academy of Pediatrics' PediaLink individualized learning plan data for the academic year 2009-2010. Residents self-assessed competencies in the 6 Accreditation Council for Graduate Medical Education (ACGME) competency areas and wrote learning goals. Textual responses for goals were mapped to 6 ACGME competency areas, future practice, or personal attributes. Adjusted mean differences and associations were estimated using multiple linear and logistic regression.

RESULTS

A total of 2254 residents reported 6078 goals. Residents self-assessed their systems-based practice (51.8) and medical knowledge (53.0) competencies lowest and professionalism (68.9) and interpersonal and communication skills (62.2) highest. Residents were most likely to identify goals involving medical knowledge (70.5%) and patient care (50.5%) and least likely to write goals on systems-based practice (11.0%) and professionalism (6.9%). In logistic regression analysis adjusting for postgraduate year (PGY), gender, and degree type (MD/DO), resident-reported goal area showed no association with the learner's relative self-assessment score for that competency area. In the conditional logistic regression analysis, with each learner serving as his or her own control, senior residents (PGY2/3+s) who rated themselves relatively lower in a competency area were more likely to write a learning goal in that area than were PGY1s.

CONCLUSIONS

Senior residents appear to develop better skills and/or motivation to explicitly turn self-assessed learning gaps into learning goals, suggesting that individualized learning plans may help improve self-regulated learning during residency.

Putting residents in the office: an effective method to teach the systems-based practice competency

OBJECTIVES

Systems-based practice (SBP) was 1 of 6 core competencies established by the Accreditation Council for Graduate Medical Education and has proven to be one of the most difficult to effectively implement. This pilot study presents an immersion workshop as an effective tool to teach the SBP competency in a way that could easily be integrated into a residency curriculum.

DESIGN

In 2006, 16 surgical residents rotated through 3 stations for 30 minutes each: coding and billing, scheduling operations and return appointments, and patient check-in. Participants were administered a pretest and posttest questionnaire evaluating their knowledge of SBP, and were asked to evaluate the workshop.

SETTING

Outpatient clinic at MedStar Georgetown University Hospital, Washington, DC.

PARTICIPANTS

Residents in the general surgery residency training program at MedStar Georgetown University Hospital.

RESULTS

Most residents (62.5%) improved their score after the workshop, whereas 31.25% showed no change and 6.25% demonstrated a decrease in score. Overall within their training levels, all groups demonstrated an increase in mean test score. Postgraduate year-2 residents demonstrated the greatest change in mean score (20%), whereas postgraduate year-4 residents demonstrated the smallest change in mean score (3.3%).

CONCLUSIONS

An immersion workshop where general surgery residents gained direct exposure to SBP concepts in situ was an effective and practical method of integrating this core competency into the residency curriculum. Such a workshop could complement more formal didactic teaching and be easily incorporated into the curriculum. For example, this workshop could be integrated into the ambulatory care requirement that each resident must fulfill as part of their clinical training.

What can we learn from narratives in medical education?

Medical literature has demonstrated the effectiveness of narrative writing in enhancing self-reflection and empathy, which opens the door for deeper understanding of patients' experiences of illness. Similarly, it promotes practitioner well-being. Therefore, it is no surprise that narrative writing finds a new home in medical education. The Accreditation Council of Graduate Medical Education (ACGME), through its Outcome Project, established six core competencies that every residency program must teach. However, no specific pedagogies were suggested. We explored the role that narrative writing can play in reconciling the ACGME core competencies with daily encounters in medical education. Our study suggests a hidden wealth in reflective writing through narratives with a promising potential for application in medical education. Reflective writing may turn out to be an innovative tool for teaching and evaluating ACGME core competencies.

Workplace-Based Assessment of Internal Medicine Resident Diagnostic Accuracy

BACKGROUND

Making an accurate diagnosis is a core skill residents must develop. Assessments of this skill and decisions to grant residents clinical independence often are based on global impressions. A workplace-based assessment of diagnostic accuracy could be a useful part of a competency-based assessment program and could inform decisions about granting residents independence.

INNOVATION

We developed a method for measuring diagnostic accuracy that was integrated into the workflow of internal medicine residents and attending physicians.

METHODS

Four senior medical residents and 6 attending physicians working in the internal medicine clinical teaching unit of a tertiary hospital participated in this study. To determine their diagnostic accuracy, residents documented a leading diagnosis for each patient they evaluated in the emergency department. After reviewing each case with the resident and after examining the patient, the resident's attending physician documented the diagnosis. Discharge diagnosis was determined by retrospective chart review to allow determination of resident and attending physician diagnostic accuracy. Data were collected for 240 consecutive patients referred for a medicine consultation.

RESULTS

Resident diagnostic accuracy was 66% (95% CI 60-72), whereas attending physician accuracy was significantly higher at 79% (95% CI 74-84, P < .001). By logistic regression, the accuracy of the attending physician was found to be influenced by the accuracy of the resident. Participants felt this process motivated them to improve their clinical reasoning.

CONCLUSIONS

Measuring resident diagnostic accuracy provides information that could be used in a competency-based assessment program to provide feedback and motivation to stimulate performance improvement.

Psychiatry residents' experiences with systems-based practice: a qualitative survey

OBJECTIVE

The aim of this study is to analyze qualitative data collected during field-testing of an instrument to assess psychiatric residents' experiences with systems-based practice (SBP).

METHODS

A total of 237 psychiatry residents from 6 levels of training in 12 different psychiatry residency training programs responded to a 60-item instrument measuring their experiences with SBP during residency. Qualitative techniques adapted from content analysis were used to review narrative responses to open-ended questions on the instrument.

RESULTS

Certain themes emerged in the residents' answers reflecting their opinions about the opportunities for (and barriers to) performing SBP in their work.

CONCLUSIONS

Psychiatric residents express an eagerness for opportunities to learn about and perform SBP but often feel constrained by the lack of resources, teaching, and supervision. Moreover, many residents desire a better understanding of healthcare economics and how to factor cost consideration into clinical care.

Development and initial validation of a project-based rubric to assess the systems-based practice competency of residents in the clinical chemistry rotation of a pathology residency

CONTEXT

Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature.

OBJECTIVE

To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry.

DESIGN

Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects.

RESULTS

Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability.

CONCLUSIONS

We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Education in quality improvement for practice in primary care during residency training and subsequent activities in practice

BACKGROUND

Quality improvement (QI) is an integral aspect of graduate medical education and an important competence for physicians.

OBJECTIVE

We examined the QI activities of recent family medicine residency graduates and whether a standardized curriculum in QI during residency resulted in greater self-reported participation in QI activities in practice after graduation.

METHODS

The family medicine residency programs affiliated with the South Carolina Area Health Education Consortium (N  =  7) were invited to participate in this study. Following completion of introductory educational activities, each site implemented regularly occurring (at least monthly) educational and patient care activities using QI principles and tools. Semiannually, representatives from each participating site met to review project aims and to provide updates regarding the QI activities in their program. To examine the impact of this project on QI activities, we surveyed graduates from participating programs from the year prior to and 2 years after the implementation of the curriculum.

RESULTS

Graduates in the preimplementation and postimplementation cohorts reported participating in periodic patient care data review, patient care registries, QI projects, and disease-specific activities (57%-71% and 54%-63%, respectively). There were no significant differences in QI activities between the 2 groups except in activities associated with status of their practice as a patient-centered medical home.

CONCLUSIONS

Most but not all family medicine graduates reported they were actively involved in QI activities within their practices, independent of their exposure to a QI curriculum during training.

Informal learning from error in hospitals: what do we learn, how do we learn and how can informal learning be enhanced? A narrative review

Learning from error is not just an individual endeavour. Organisations also learn from error. Hospitals provide many learning opportunities, which can be formal or informal. Informal learning from error in hospitals has not been researched in much depth so this narrative review focuses on five learning opportunities: morbidity and mortality conferences, incident reporting systems, patient claims and complaints, chart review and prospective risk analysis. For each of them we describe: (1) what can be learnt, categorised according to the seven CanMEDS competencies; (2) how it is possible to learn from them, analysed against a model of informal and incidental learning; and (3) how this learning can be enhanced. All CanMEDS competencies could be enhanced, but there was a particular focus on the roles of medical expert and manager. Informal learning occurred mostly through reflection and action and was often linked to the learning of others. Most important to enhance informal learning from these learning opportunities was the realisation of a climate of collaboration and trust. Possible new directions for future research on informal learning from error in hospitals might focus on ways to measure informal learning and the balance between formal and informal learning. Finally, 12 recommendations about how hospitals could enhance informal learning within their organisation are given.

Instituting systems-based practice and practice-based learning and improvement: a curriculum of inquiry

BACKGROUND

The Accreditation Council for Graduate Medical Education (ACGME) requires that training programs integrate system-based practice (SBP) and practice-based learning and improvement (PBLI) into internal medicine residency curricula. CONTEXT AND SETTING: We instituted a seminar series and year-long-mentored curriculum designed to engage internal medicine residents in these competencies.

METHODS

Residents participate in a seminar series that includes assigned reading and structured discussion with faculty who assist in the development of quality improvement or research projects. Residents pursue projects over the remainder of the year. Monthly works in progress meetings, protected time for inquiry, and continued faculty mentorship guide the residents in their project development. Trainees present their work at hospital-wide grand rounds at the end of the academic year. We performed a survey of residents to assess their self-reported knowledge, attitudes and skills in SBP and PBLI. In addition, blinded faculty scored projects for appropriateness, impact, and feasibility.

OUTCOMES

We measured resident self-reported knowledge, attitudes, and skills at the end of the academic year. We found evidence that participants improved their understanding of the context in which they were practicing, and that their ability to engage in quality improvement projects increased. Blinded faculty reviewers favorably ranked the projects' feasibility, impact, and appropriateness. The 'Curriculum of Inquiry' generated 11 quality improvement and research projects during the study period. Barriers to the ongoing work include a limited supply of mentors and delays due to Institutional Review Board approval. Hospital leadership recognizes the importance of the curriculum, and our accreditation manager now cites our ongoing work.

CONCLUSIONS

A structured residency-based curriculum facilitates resident demonstration of SBP and practice-based learning and improvement. Residents gain knowledge and skills though this enterprise and hospitals gain access to trainees who help to solve ongoing problems and meet accreditation requirements.

Perspectives on the changing healthcare system: teaching systems-based practice to medical residents

PURPOSE

The Accreditation Council for Graduate Medical Education restructured its accreditation system to be based on educational outcomes in six core competencies. Systems-based practice is one of the six core competencies. The purpose of this report is to describe Weill Cornell Medical College's Internal Medicine Residency program curriculum for systems-based practice (SBP) and its evaluation process.

METHODS

To examine potential outcomes of the POCHS curriculum, an evaluation was conducted, examining participants': (1) knowledge gain; (2) course ratings; and (3) qualitative feedback.

RESULTS

On average, there was a 19 percentage point increase in knowledge test scores for all three cohorts. The course was rated overall highly, receiving an average of 4.6 on a 1-5 scale. Lastly, the qualitative comments supported that the material is needed and valued.

CONCLUSION

The course, entitled Perspectives on the Changing Healthcare System (POCHS) and its evaluation process support that systems-based practice is crucial to residency education. The course is designed not only to educate residents about the current health care system but also to enable them to think critically about the risk and benefits of the changes. POCHS provides a framework for teaching and assessing this competency and can serve as a template for other residency programs looking to create or restructure their SBP curriculum.

Teaching medical management and operations engineering for systems-based practice to radiology residents

RATIONALE AND OBJECTIVES

To better prepare radiology residents for providing care within the context of the larger health care system, this study evaluated the feasibility and impact of a curriculum to enhance radiology residents' understanding and ability to apply concepts from medical management and industrial and operational engineering to systems-based practice problems in radiology practice.

MATERIALS AND METHODS

A multiprofessional team including radiology, medical education, and industrial and operational engineering professionals collaborated in developing a seven-module curriculum, including didactic lectures, interactive large-group analysis, and small-group discussions with case-based radiology examples, which illustrated real-life management issues and the roles physicians held. Residents and faculty participated in topic selection. Pre- and post-instruction formative assessments were administered, and results were shared with residents during teaching sessions.

RESULTS

Attendance and participation in case-based scenario resolutions indicate the feasibility and impact of the interactive curriculum on residents' interest and ability to apply curricular concepts to systems-based practice in radiology. Paired t test analyses (P < .05) and effect sizes showed residents significantly increased their knowledge and ability to apply concepts to systems-based practice issues in radiology.

CONCLUSIONS

Our iterative curriculum development and implementation process demonstrated need and support for a multiprofessional team approach to teach management and operational engineering concepts. Curriculum topics are congruent with Accreditation Council for Graduate Medical Education requirements for systems-based practice. The case-based curriculum using a mixed educational format of didactic lectures and small-group discussion and problem analysis could be adopted for other radiology programs, for both residents and continuing medical education applications.

Using lean methodology to teach quality improvement to internal medicine residents at a safety net hospital

The overall objective of this initiative was to develop a quality improvement (QI) curriculum using Lean methodology for internal medicine residents at Boston Medical Center, a safety net academic hospital. A total of 90 residents and 8 School of Public Health students participated in a series of four, 60- to 90-minute interactive and hands-on QI sessions. Seventeen QI project plans were created and conducted over a 4-month period. The curriculum facilitated internal medicine residents' learning about QI and development of positive attitudes toward QI (assessed using pre- and post-attitude surveys) and exposed them to an interprofessional team structure that duplicates future working relationships. This QI curriculum can be an educational model of how health care trainees can work collaboratively to improve health care quality.

A case for an international health elective training program during residency: a four-points call for action

BACKGROUND

International Health Electives (IHE) are becoming more popular among graduate training programs. This is likely due to the high demands from graduating medical students who are seeking to have an international health experience during their post-graduate training. Despite the important educational experiences associated with an IHE, this opportunity does not exist in all graduate medical programs and fewer have formal established programs.

SUMMARY

We are suggesting that graduate training programs are in a unique position to provide such experiences to our future physicians, in turn creating immediate benefits to host nations as well as long-term impacts on our society in the United States. We are proposing Four Points for stakeholders involved in training future physicians to use as they consider designing such opportunities for future trainees. The four points include: residents are capable of providing service to host nations, improve the quality of care to communities in the U.S., foster graduating medical students' global health interests and increase global health mentorship.

CONCLUSIONS

We hope that addressing these four points will reemphasize the importance of establishing an IHE in all graduate training programs.

Systems-based content in medical morbidity and mortality conferences: a decade of change

BACKGROUND

Following the Accreditation Council for Graduate Medical Education recommendations in 1999 to foster education in the systems-based practice (SBP) competency by examining adverse clinical events, institutions have modified the morbidity and mortality conference (MMC) to increase SBP-related discussion. We sought to examine the extent to which SBP-related content has increased in our department's MMCs compared with MMCs 10 years prior.

METHOD

We qualitatively analyzed audio recordings of our MMCs during 2 academic years, 1999-2000 (n  =  30) and 2010-2011 (n  =  30). We categorized comments and questions from moderators and faculty as SBP or non-SBP and characterized conferences by whether adverse events were presented and which systems issues were discussed.

RESULTS

Compared with MMCs in 1999-2000, present-day MMCs included a greater average percentage of SBP comments stated (69% versus 12%; P ≤ .001) and questions asked (13% versus 1%; P  =  .001) by the moderator, SBP comments stated (44% versus 4%; P ≤ .001) and questions asked (19% versus 1%; P ≤ .001) by faculty, and were more likely to present adverse events (87% versus 13%; P < .001). Interrater reliability for the distinction between SBP and non-SBP content was good (κ  =  0.647). Most common categories of systems issues discussed in 2010-2011 were critical laboratory value processing and reporting, institutional policies, and hospital-based factors.

CONCLUSIONS

Over the past decade, our MMC has transformed to include more discussion of SBP-related content and adverse events. The MMC can be used to educate residents in SBP and can also serve as a cornerstone for departmental quality and safety initiatives.

Assessing systems-based practice

The conceptual definition of systems-based practice (SBP) does not easily translate into directly observable actions or behaviors that can be easily assessed. At the Academic Emergency Medicine consensus conference on education research in emergency medicine (EM), a breakout group presented a review of the literature on existing assessment tools for SBP, discussed the recommendations for research tool development during breakout sessions, and developed a research agenda based on this discussion.

International electives at the university of Minnesota global pediatric residency program: opportunities for education in all Accreditation Council for Graduate Medical Education competencies

PURPOSE

Globally competent pediatricians are in demand because of the increasing numbers of children from immigrant families living in the United States and the shortages of health care workers in low-income countries where the majority of the worlds' children live. This study sought to better understand the educational outcomes of international electives taken by pediatric residents training in global health.

METHODS

Thirty-two pediatric residents who participated in an international elective as part of a global health curriculum completed reflective essays which were analyzed for themes from 2006 to 2010. During the first-order analysis, the emergent themes mapped to the Accreditation Council for Graduate Medical Education (ACGME) competencies. In response, a second-order analysis re-examined the essays with an additional researcher to support categorization consistent with the ACGME competencies.

RESULTS

More than 90% of essays described experiences related to medical knowledge, patient care and systems-based practice. More than 50% included reflections on practice-based learning and improvement, professionalism, and interpersonal and communication skills. Residents also described the impact on their personal and professional development.

CONCLUSION

International electives can provide educational opportunities for residents to develop competency in each of the 6 ACGME domains and to reevaluate their life purpose and career goals. In addition to opportunities to increase their medical knowledge, patient care and communication skills, residents find international electives rich learning environments for systems-based practice, practice-based learning/improvement, and professionalism, domains that can be challenging to teach. These findings support the importance of international electives in global health in meeting core requirements in residency training.

Quality assurance in postgraduate medical education: implications for dermatology residency training programs

BACKGROUND

In the past few years, quality assurance has become an increasingly important part of medical education for both Canadian and American training programs. Since this emphasis on quality assurance in residency programs is recent, most faculty members involved in teaching residents in dermatology training programs would not themselves have had experience with quality assurance. As a result, satisfying this requirement may be a challenge.

OBJECTIVES

In this article, we review published reports in which various residency training programs have satisfied this requirement and propose projects in which dermatology residency training programs may satisfy quality assurance requirements.

METHODS

Using the key words residency, training, project, quality, assurance, improvement, medical errors, and safety, a literature search was conducted of English-language articles published after January 1990.

RESULTS/CONCLUSIONS

There are many innovative and effective ways program directors in dermatology training programs should be able to develop projects that improve patient care, enhance resident education, and fulfill accreditation requirements.

Addressing the Scholarly Activity Requirements for Residents: One Program's Solution

BACKGROUND

Scholarly activity as a component of residency education is becoming increasingly emphasized by the Accreditation Council for Graduate Medical Education. "Limited or no evidence of resident or faculty scholarly activity" is a common citation given to family medicine residency programs by the Review Committee for Family Medicine.

OBJECTIVE

The objective was to provide a model scholarly activity curriculum that has been successful in improving the quality of graduate medical education in a family medicine residency program, as evidenced by a record of resident academic presentations and publications.

METHODS

We provide a description of the Clinical Scholars Program that has been implemented into the curriculum of the Trident/Medical University of South Carolina Family Medicine Residency Program.

RESULTS

During the most recent 10-year academic period (2000-2010), a total of 111 residents completed training and participated in the Clinical Scholars Program. This program has produced more than 24 presentations during national and international meetings of medical societies and 15 publications in peer-reviewed medical journals. In addition, many of the projects have been presented during meetings of state and regional medical organizations.

CONCLUSIONS

This paper presents a model curriculum for teaching about scholarship to family medicine residents. The success of this program is evidenced by the numerous presentations and publications by participating residents.

The use of a multidisciplinary morbidity and mortality conference to incorporate ACGME general competencies

BACKGROUND

The Surgical Morbidity and Mortality conference has long been used as an opportunity for both process improvement and resident education. With recent heightened focus on creating environments of safety and on meeting the Accreditation Council for Graduate Medical Education (ACGME) General Competencies, novel approaches are required. With the understanding that the provision of medical care is an inherently multidisciplinary enterprise, we advocate the creation and use of a Multidisciplinary Morbidity and Mortality conference (MM&M) as a means to establish this culture of safety while teaching the ACGME General Competencies to surgery residents.

METHODS

A quarterly MM&M conference was implemented to foster communication between disciplines, provide a forum for quality improvement, and enhance patient care. All stakeholders in the perioperative enterprise attend, including the departments of surgery, anesthesia, radiology, pharmacy, nursing, environmental services, risk management, and patient services. Cases that expose system issues with potential to harm patients are discussed in an open, nonconfrontational forum. Solutions are presented and initiatives developed to improve patient outcomes. We retrospectively reviewed the topics presented since the conference's inception, grouping them into 1 of 7 categories. We then evaluated the completion of the improvement initiatives developed after discussion at the conference.

RESULTS

Over a 21-month period, 11 cases were discussed with 23 "actionable" initiatives for quality improvement. Cases were grouped by category; procedures (36.5%), process (36.5%), patient-related (9%), communication (9%), medication (9%), device (0%), and ethics (0%). All cases discussed addressed at least 4 of the 6 ACGME General Competencies.

CONCLUSIONS

Like the practice of medicine, the occurrence of adverse outcomes is frequently multidisciplinary. An MM&M conference is useful in its potential to meet ACGME General Competencies, engender a culture of patient safety, and rapidly achieve quality improvement and systems health care delivery initiatives in a large academic medical center.

Systems-based practice in graduate medical education: systems thinking as the missing foundational construct

BACKGROUND

Since 2001, residencies have struggled with teaching and assessing systems-based practice (SBP). One major obstacle may be that the competency alone is not sufficient to support assessment. We believe the foundational construct underlying SBP is systems thinking, absent from the current Accreditation Council for Graduate Medical Education competency language.

SUMMARY

Systems thinking is defined as the ability to analyze systems as a whole. The purpose of this article is to describe psychometric issues that constrain assessment of SBP and elucidate the role of systems thinking in teaching and assessing SBP.

CONCLUSION

Residency programs should incorporate systems thinking models into their curricula. Trainees should be taught to understand systems at an abstract level, in order to analyze their own healthcare systems, and participate in quality and patient safety activities. We suggest that a developmental trajectory for systems thinking be developed, similar to the model described by Dreyfus and Dreyfus.

A multidisciplinary approach for teaching systems-based practice to internal medicine residents

BACKGROUND

Rapid growth in the complexity of the health care environment (including monitoring systems for health care quality and patient safety) may result in graduating internists not being adequately prepared for the demands the system places on them. In response, the Residency Review Committee for Internal Medicine created the Educational Innovations Project (EIP) to encourage select residency training programs to develop new strategies and methods to meet changing demands in graduate medical education.

METHODS

As part of the EIP, our program created an innovative administrative internship. This multiyear curriculum provides systems-based practice training and consists of a series of rotations that take place during the 3 years of internal medicine residency. Each session involves close interaction with the nonphysician personnel who are instrumental in making our institution a functional and cohesive unit. To assess the potential impact of the rotations, we survey senior residents, recent graduates, and faculty educators. In conjunction with the Performance and Patient Experience departments of the hospital, we track several systems-based practice metrics for residents, including compliance with core health care measures, length of stay, and patient satisfaction.

RESULTS

Residents recognize the need to develop systems-based practice skills, to readily participate in structured curricula designed to enhance such skills, and to provide leadership in organizing and publishing quality improvement initiatives, and upon graduation, they may lament that they did not receive even more vigorous training in these areas.

CONCLUSION

Although internal medicine residencies continue to improve their training in systems-based practice, our experience suggests that an even greater emphasis on these skills may be warranted.

Relationship between critical reflection and quality improvement proposal scores in resident doctors

OBJECTIVES

transformative learning theory supports the idea that reflection on quality improvement (QI) opportunities and the ability to develop successful QI projects may be fundamentally linked. We used validated methods to explore associations between resident doctors' reflections on QI opportunities and the quality of their QI project proposals.

METHODS

eighty-six residents completed written reflections on practice improvement opportunities and developed QI proposals. Two faculty members assessed residents' reflections using the 18-item Mayo Evaluation of Reflection on Improvement Tool (MERIT), and assessed residents' QI proposals using the seven-item Quality Improvement Project Assessment Tool (QIPAT-7). Both instruments have been validated in previous work. Associations between MERIT and QIPAT-7 scores were determined. Internal consistency reliabilities of QIPAT-7 and MERIT scores were calculated.

RESULTS

there were no significant associations between MERIT overall and domain scores, and QIPAT-7 overall and item scores. The internal consistency of MERIT and QIPAT-7 item groups were acceptable (Cronbach's α 0.76-0.94).

CONCLUSIONS

the lack of association between MERIT and QIPAT-7 scores indicates a distinction between resident doctors' skills at reflection on QI opportunities and their abilities to develop QI projects. These findings suggest that practice-based reflection and QI project development are separate constructs, and that skilful reflection may not predict the ability to design meaningful QI initiatives. Future QI curricula should consider teaching and assessing QI reflection and project development as distinct components.

Perspective: Transformative learning: a framework using critical reflection to link the improvement competencies in graduate medical education

Quality improvement (QI) in health care involves activities ranging from enhancing personal practice to reforming the larger health care system. The Accreditation Council for Graduate Medical Education recognizes this broad definition of QI in its requirement that physicians-in-training demonstrate competence in practice-based learning and improvement (PBLI) and systems-based practice (SBP). Creative metaphors have been used to teach the PBLI and SBP competencies, but conceptual frameworks describing the relationship between these competencies are needed. Transformative learning is an adult education theory that states individuals must critically reflect on life events in order to change their beliefs or behaviors. The authors propose that critical reflection during transformative learning can conceptually link PBLI and SBP. Reflection on personal experience with suboptimal patient care leads to recognizing personal or system limitations. Addressing personal limitations improves individual practice (PBLI), whereas applying QI methodologies leads to large-scale improvements (SBP). Educators who adopt the transformative learning framework should be able to design meaningful QI curricula that encourage residents to be reflective and empower them with QI skills.

Students learn systems-based care and facilitate system change as stakeholders in a free clinic experience

Systems-based practice (SBP) is rarely taught or evaluated during medical school, yet is one of the required competencies once students enter residency. We believe Texas A&M College of Medicine students learn about systems issues informally, as they care for patients at a free clinic in Temple, TX. The mandatory free clinic rotation is part of the Internal Medicine clerkship and does not include formal instruction in SBP. During 2008-2009, a sample of students (n = 31) on the IMED clerkship's free clinic rotation participated in a program evaluation/study regarding their experiences. Focus groups (M = 5 students/group) were held at the end of each outpatient rotation. Students were asked: "Are you aware of any system issues which can affect either the delivery of or access to care at the free clinic?" Data saturation was reached after six focus groups, when investigators noted a repetition of responses. Based upon investigator consensus opinion, data collection was discontinued. Based upon a content analysis, six themes were identified: access to specialists, including OB-GYN, was limited; cost containment; lack of resources affects delivery of care; delays in care due to lack of insurance; understanding of larger healthcare system and free clinic role; and delays in tests due to language barriers. Medical students were able to learn about SBP issues during free clinic rotations. Students experienced how SBP issues affected the health care of uninsured individuals. We believe these findings may be transferable to medical schools with mandatory free clinic rotations.

Teaching quality improvement and patient safety to trainees: a systematic review

PURPOSE

To systematically review published quality improvement (QI) and patient safety (PS) curricula for medical students and/or residents to (1) determine educational content and teaching methods, (2) assess learning outcomes achieved, and (3) identify factors promoting or hindering curricular implementation.

METHOD

Data sources included Medline (to January 2009), EMBASE, HealthSTAR, and article bibliographies. Studies selected reported curricula outlining specific educational content and teaching format. For articles with an evaluative component, the authors abstracted methodological features, such as study design. For all articles, they conducted a thematic analysis to identify factors influencing successful implementation of the included curricula.

RESULTS

Of 41 curricula that met the authors' criteria, 14 targeted medical students, 24 targeted residents, and 3 targeted both. Common educational content included continuous QI, root cause analysis, and systems thinking. Among 27 reports that included an evaluation, curricula were generally well accepted. Most curricula demonstrated improved knowledge. Thirteen studies (32%) successfully implemented local changes in care delivery, and seven (17%) significantly improved target processes of care. Factors that affected the successful curricular implementation included having sufficient numbers of faculty familiar with QI and PS content, addressing competing educational demands, and ensuring learners' buy-in and enthusiasm. Participants in some curricula also commented on discrepancies between curricular material and local institutional practice or culture.

CONCLUSIONS

QI and PS curricula that target trainees usually improve learners' knowledge and frequently result in changes in clinical processes. However, successfully implementing such curricula requires attention to a number of learner, faculty, and organizational factors.

Teaching and learning in morbidity and mortality rounds: an ethnographic study

OBJECTIVES In keeping with the current emphasis on quality improvement and patient safety, a Canadian division of general internal medicine began holding weekly morbidity and mortality rounds (M&MRs) with postgraduate trainees. Grounded in the medical education and social sciences literatures about such rounds, we sought to explore the teaching and learning processes that occur in M&MRs in order to understand their role in, and contribution to, the current medical education context. METHODS We conducted an ethnography of these M&MRs. We observed the rounds, conducted interviews with both staff doctors and residents and triangulated the resultant data. Concurrent, iterative data collection and analysis enabled sampling to saturation. RESULTS Staff doctors had differing understandings of the role of M&MRs and valued different kinds of teaching. They did not think they were teaching medical content knowledge at these rounds, but rather that they were role-modelling six skills, attitudes and behaviours, including 'identifying and addressing process and systems issues affecting care'. Residents primarily wanted to learn content knowledge and tried to extract such knowledge out of the rounds. They did recognise and value that they were learning about process and systems issues. They also agreed that staff doctors were role-modelling other things, but had varying perceptions of what those were; most did not value this role-modelled learning as much as they valued the acquisition of content knowledge. CONCLUSIONS These M&MRs were effective forums for addressing patient safety and quality improvement competencies. They carried none of the negative functions attributed to such rounds in the sociology literature, focusing neither on absolving responsibility nor on learning socially acceptable ways to discuss death in public. However, this study revealed a marked disjunction between the teaching valued by staff doctors and the learning valued by their trainees.

Commentary: dinosaurs fated for extinction? Health care delivery at academic health centers

Health care delivery at academic health centers (AHCs) can be viewed as dinosaur-like. Both are large and complex entities that consume many resources and are slow to adapt to competitive predatory forces. The potential for severe climate shifts, with changes in payer mix, competition from the private sector, and health care reform all occurring in the current health care system, could precipitate either the beginning of extinction for the AHC dinosaur or, hopefully, stimulate its evolution and development into a new model of health care delivery.Given the importance of clinical revenue to the entirety of the AHC enterprise, there is incentive for AHCs to maintain and indeed expand their clinical care delivery mechanisms. Yet, AHCs are institutions of investigation and inquiry. New models of care delivery and their impact on the current clinical care system must be developed through local demonstration projects and experimental clinical models. These models must be studied, and the findings should be shared with the community.The authors argue that this course of action will be challenging because traditional workflows must be restricted to improve care coordination and a changing workforce demographic. It will also require thoughtful approaches to reward innovative clinical work and new directions in strategic management by institution leaders. This commentary outlines recommendations to stave off extinction and enhance the next generation of clinical care delivery at AHCs.

Abortion training at multiple sites: an unexpected curriculum for teaching systems-based practice

BACKGROUND

In 1999, the Accreditation Council for Graduate Medical Education endorsed systems-based practice as one of six general competencies.

PURPOSE

The objective is to explore the paradigm of teaching residents systems-based practice during a women's health rotation that included abortion training in multiple settings.

METHODS

During a routine women's health rotation, residents from two urban family medicine residency programs received early abortion training at a high-volume abortion clinic and their continuity clinic. Thirty-min semistructured interviews were conducted with all 26 residents who rotated between July 2005 and August 2006. Transcripts were analyzed using thematic codes.

RESULTS

Through exposure to different healthcare delivery systems, residents learned about systems-based practice, including understanding the failure of the larger system to meet patients' reproductive healthcare needs, differences between two systems, and potential systems barriers they might face as providers.

CONCLUSIONS

Abortion training in multiple settings may serve as a paradigm for teaching systems-based practice during other rotations that include training in multiple sites.

Internal medicine residency graduates' perceptions of the systems-based practice and practice-based learning and improvement competencies

BACKGROUND

Resident education in Systems-Based Practice (SBP) and Practice-Based Learning and Improvement (PBLI) is required but underemphasized.

PURPOSES

The objectives are to identify SBP and PBLI knowledge and skills with the most relevance to our graduates' practices and to determine how well they were prepared during residency training to address these issues.

METHODS

A survey was drafted based on Accreditation Council for Graduate Medical Education competency definitions and published literature on SBP and PBLI. Respondents indicated the extent to which each item is relevant to their practice and the adequacy of instruction received on a 5-point Likert scale.

RESULTS

All topics had high perceived relevance to practice with most topics rated low for adequacy of training. Topics of practice management and health care economics contained the largest gaps between mean ratings of relevance and adequacy of training (p < .001). Few differences in ratings were seen based on graduate demographics.

CONCLUSIONS

This survey has allowed us to prioritize SBP and PBLI curricula to meet the needs of our graduates.

The patient panel conference experience: what patients can teach our residents about competency issues

PURPOSE

In 2007, the Scott & White/Texas A&M HSC College of Medicine began requiring all internal medicine residents to attend quarterly patient panel conferences, during which former Scott & White patients speak frankly about their inpatient and outpatient experiences. The main purpose of this mixed-methods pilot study was to determine whether residents' competency education could be enhanced via the conferences.

METHOD

Of the 54 internal medicine residents in the residency program, 31, 39, and 41 participated in three patient panel conferences, respectively, between December 2007 and August 2008. Each resident completed an assessment that included a reflection on his or her own practice and the identification of competency issues highlighted by patients' oral narratives. Content analyses of responses to open-ended questions were performed. Consensus on themes was reached. Descriptive statistics were run on quantitative data.

RESULTS

Six themes were identified: improve communication with patients/families, improve patient care, improve professional behaviors, empathize with patients/families, display sensitivity to patients'/families' needs/concerns, and recognize system issues. When asked if the conference highlighted competency problems, residents answered "agree" or "strongly agree" as follows: 82% for professionalism, 82.9% for systems-based practice, 85.2% for interpersonal and communication skills, and 84.4% for patient care. The majority were able to provide examples of competency issues.

CONCLUSIONS

The patient panel conference experience was a powerful mechanism for enhancing competency education. The conferences were an effective means of presenting real-life examples of systems issues in the context of a hospital system.

Development of the objective structured system-interaction examination

STUDY OBJECTIVE

The purpose of this study was to develop an objective method of evaluating resident competency in systems-based practice.

STUDY DESIGN

Faculty developed a 12-station examination, the Objective Structured System-Interaction Examination (OSSIE), patterned after the Objective Structured Clinical Examinations (OSCEs), to evaluate residents' ability to effectively work within the complex medical system of care. Scenarios consisted of multiple situations, such as patient hand-offs, consultations, complicated discharges, and family meetings, in which residents interacted with simulated professionals, simulated patients, and simulated family members to demonstrate the systems-based skills. Twelve second-year residents participated in the OSSIE.

FINDINGS

Along with the standardized professionals, a faculty member provided the resident with immediate feedback and completed an evaluation form designed specifically to assess systems-based practice. Residents, faculty, and staff evaluated the OSSIE and felt it provided a rich learning experience and was a beneficial means of formative assessment. The residents' third-year learning experiences were adapted to meet their needs, and suggestions were offered for curriculum revision.

DISCUSSION

The OSSIE is unique in that it uses standardized professionals, involves scenarios in a variety of settings, and incorporates current technology, including an electronic health record and a state-of-the-art simulation laboratory, into the examination. Challenges to implementation include faculty time, scheduling of residents, and availability of resources.

CONCLUSION

By using the OSSIE, faculty are able to assess, provide constructive feedback, and tailor training opportunities to improve resident competence in systems-based practice. Reliability and validity of an instrument developed for use with the OSSIE are currently being determined.

Systems-based practice defined: taxonomy development and role identification for competency assessment of residents

PURPOSE

To demonstrate a methodology for coding and taxonomy development and to operationally define residents' competence in systems-based practice (SBP) in terms of observable roles, actions, and behaviors.

METHODS

The Accreditation Council for Graduate Medical Education's (ACGME's) full-text definition of SBP and the 6 discrete expectations it contains were content analyzed. Structured interviews of 88 health care professionals using a variant of focus group interviews called nominal group processes were conducted and qualitatively analyzed to identify the key attributes of SBP. Themes obtained from these 2 procedures were conceptually matched and organized to create a taxonomy of observable SPB behaviors and the SBP domain.

RESULTS

Six general resident roles emerged, under which 35 specific behavioral attributes were subsumed. From the SBP domain specified. Sample SBP items categorized by roles were derived that reflected "in-context" representations of ACGME SBP expectations.

CONCLUSION

Our comprehensive analysis created an operational representation of the SBP competency. The taxonomy development model provides a framework for constructing assessment instrument(s) that could be applied to the other ACGME competencies or complex concepts in medical education.