Lessons from industry: one school's transformation toward "lean" curricular governance

Legitimizing Continuous Quality Improvement (CQI): Navigating Rationality in Undergraduate Medical Education

In July of 2015, the Liaison Committee on Medical Education (LCME)-the primary accrediting body for North American allopathic medical schools-formally advanced a model of "formative accreditation" by requiring that medical schools engage in "ongoing planning and continuous quality improvement processes that establish short and long-term programmatic goals, result in the achievement of measurable outcomes that are used to improve programmatic quality, and ensure effective monitoring of the medical education program's compliance with accreditation standards."As these and parallel forces redefine undergraduate medical education (UME) in increasingly rationalistic (i.e., operational, measureable, controllable) terms, efforts to implement meaningful continuous quality improvement (CQI) processes may be challenged to overcome perceptions of questionable purpose, worth, and impact often associated with administration mandates. This commentary discusses potential factors underlying the growing rationalism in UME and offers practical strategies to shield CQI from being passively dismissed, excessively routinized, or redirected toward other institutional ends-remaining, instead, purposefully focused on the task at hand: Enhancing teaching and learning in undergraduate medical curricula.

Assessment for Systems Learning: A Holistic Assessment Framework to Support Decision Making Across the Medical Education Continuum

Viewing health care from a systems perspective-that is, "a collection of different things which, working together, produce a result not achievable by the things alone"-raises awareness of the complex interrelationships involved in meeting society's goals for accessible, cost-effective, high-quality health care. This perspective also emphasizes the far-reaching consequences of changes in one sector of a system on other components' performance. Medical education promotes this holistic view of health care in its curricula and competency requirements for graduation at the undergraduate and graduate training levels. But how completely does medical education apply a systems lens to itself?The continuum of medical training has undergone a series of changes that have moved it more closely to a systems organizational model. Competency assessment criteria have been expanded and more explicitly defined for learners at all levels of training. Outcomes data, in multiple domains, are monitored by external reviewers for program accreditation. However, translating increasing amounts of individual outcomes into actionable intelligence for decision making poses a formidable information management challenge.Assessment in systems is designed to impart a "big picture" of overall system performance through the synthesis, analysis, and interpretation of outcomes data to provide actionable information for continuous systems improvement, innovation, and long-term planning. A systems-based framework is presented for use across the medical education continuum to facilitate timely improvements in individual curriculum components, continuous improvement in overall program performance, and program decision making on changes required to better address society's health care needs.

Exploring the Realities of Curriculum-by-Random-Opportunity: The Case of Geriatrics on the Internal Medicine Clerkship Rotation

BACKGROUND

While major clerkship blocks may have objectives related to specialized areas such as geriatrics, gay and lesbian bisexual transgender health, and palliative care, there is concern that teaching activities may not attend sufficiently to these objectives. Rather, these objectives are assumed to be met "by random opportunity".((1)) This study explored the case of geriatric learning opportunities on internal medicine clinical teaching units, to better understand the affordances and limitations of curriculum by random opportunity.

METHODS

Using audio-recordings of morning case review discussions of 13 patients > 65 years old and the Canadian geriatric core competencies for medical students, we conducted a content analysis of each case for potential geriatric and non-geriatric learning opportunities. These learning opportunities were compared with attendings' case review teaching discussions. The 13 cases contained 40 geriatric-related and 110 non-geriatric-related issues. While many of the geriatric issues (e.g., delirium, falls) were directly relevant to the presenting illness, attendings' teaching discussions focused almost exclusively on non-geriatric medical issues, such as management of diabetes and anemia, many of which were less directly relevant to the reason for presenting to hospital.

RESULTS

The authors found that the general medicine rotation provides opportunities to acquire geriatric competencies. However, the rare uptake of opportunities in this study suggests that, in curriculum-by-random-opportunity, presence of an opportunity does not justify the assumption that learning objectives will be met.

CONCLUSIONS

More studies are required to investigate whether these findings are transferrable to other vulnerable populations about which undergraduate students are expected to learn through curriculum by random opportunity.

Orthopedic resident work-shift analysis: are we making the best use of resident work hours?

BACKGROUND

Surgery programs have been tasked to meet rising demands in patient surgical care while simultaneously providing adequate resident training in the midst of increasing resident work-hour restrictions. The purpose of this study was to quantify orthopedic surgery resident workflow and identify areas needing improved resident efficiency. We hypothesize that residents spend a disproportionate amount of time involved in activities that do not relate directly to patient care or maximize resident education.

METHODS

We observed 4 orthopedic surgery residents on the orthopedic consult service at a major tertiary care center for 72 consecutive hours (6 consecutive shifts). We collected minute-by-minute data using predefined work-task criteria: direct new patient contact, direct existing patient contact, communications with other providers, documentation/administrative time, transit time, and basic human needs. A seventh category comprised remaining less-productive work was termed as standby.

RESULTS

In a 720-minute shift, residents spent on an average: 191 minutes (26.5%) performing documentation/administrative duties, 167.0 minutes (23.2%) in direct contact with new patient consults, 129.6 minutes (17.1%) in communication with other providers regarding patients, 116.2 (16.1%) minutes in standby, 63.7 minutes (8.8%) in transit, 32.6 minutes (4.5%) with existing patients, and 20 minutes (2.7%) attending to basic human needs. Residents performed an additional 130 minutes of administrative work off duty. Secondary analysis revealed residents were more likely to perform administrative work rather than directly interact with existing patients (p = 0.006) or attend to basic human needs (p = 0.003).

CONCLUSIONS

Orthopedic surgery residents spend a large proportion of their time performing documentation/administrative-type work and their workday can be operationally optimized to minimize nonvalue-adding tasks. Formal workflow analysis may aid program directors in systematic process improvements to better align resident skills with tasks.

LEVEL OF EVIDENCE

III.

Assessing the quality of clinical teaching: a preliminary study

OBJECTIVES

Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations.

METHODS

We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur.

RESULTS

The internal consistency of the 15-item checklist was good (alpha = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items.

CONCLUSIONS

Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.

The Educational Kanban: promoting effective self-directed adult learning in medical education

The author reviews the many forces that have driven contemporary medical education approaches to evaluation and places them in an adult learning theory context. After noting their strengths and limitations, the author looks to lessons learned from manufacturing on both efficacy and efficiency and explores how these can be applied to the process of trainee assessment in medical education.Building on this, the author describes the rationale for and development of the Educational Kanban (EK) at Children's Hospital Boston--specifically, how it was designed to integrate adult learning theory, Japanese manufacturing models, and educator observations into a unique form of teacher-student collaboration that allows for continuous improvement. It is a formative tool, built on the Accreditation Council for Graduate Medical Education's six core competencies, that guides educational efforts to optimize teaching and learning, promotes adult learner responsibility and efficacy, and takes advantage of the labor-intensive clinical educational setting. The author discusses how this model, which will be implemented in July 2009, will lead to training that is highly individualized, optimizes faculty and student educational efforts, and ultimately conserves faculty resources. A model EK is provided for general reference.The EK represents a novel approach to adult learning that will enhance educational effectiveness and efficiency and complement existing evaluative models. Described here in a specific graduate medical setting, it can readily be adapted and integrated into a wide range of undergraduate and graduate clinical educational environments.

Consequences within medical schools for students with poor performance on a medical school standardized patient comprehensive assessment

PURPOSE

Medical schools increasingly employ comprehensive standardized patient assessments to ensure medical students' clinical competence. The consequences of poor performance on the assessment and the institutional factors associated with imposing consequences are unknown.

METHOD

In 2006, the investigators surveyed 122 U.S. medical school curriculum deans about comprehensive assessments using standardized patients after core clerkships, with questions about exam characteristics, institutional commitment to the examination (years of experience, exam infrastructure, clerkship director involvement), academic consequences of failing the assessment, and satisfaction with remediation.

RESULTS

Ninety-three of 122 (76%) deans responded. Eighty-two (88%) conducted a comprehensive assessment in years three or four of medical school. Of those, required remediation was the only consequence of failing employed by 61 schools (74%), and only 39 (47%) required retesting for graduation. Participants were somewhat satisfied with (mean 3.45 out of maximum 5, SD 1.08) and confident in (3.37, SD 1.17) their remediation process. Satisfaction and confidence were associated with requiring remediation (P = .003) and retesting (P < .001), but experience with the exam, exam infrastructure, and clerkship director involvement were not. No school demographic characteristics or measures of institutional commitment were related to external reporting of students' comprehensive assessment scores.

CONCLUSIONS

Despite the prevalence of comprehensive assessments, schools attach few academic consequences to poor performance. Educators are only moderately satisfied with their efforts to remediate poor performers. However, schools with greater trust in their remediation process than other schools are more likely to enforce consequences of poor performance.