Integrating improvement competencies into residency education: a pilot project from a pediatric continuity clinic

A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts?

BACKGROUND

With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes.

METHODS

A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts.

RESULTS

18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes.

CONCLUSION

This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.

Impact of Quality Improvement on Care of Chronic Obstructive Pulmonary Disease Patients in an Internal Medicine Resident Clinic

Chronic obstructive pulmonary disease (COPD) is a leading cause of morbidity and mortality. Guideline-discordant care of COPD is not uncommon. Further, there is a push to incorporate quality improvement (QI) training into internal medicine (IM) residency curricula. This study compared quality of care of COPD patients in an IM residents' clinic and a pulmonary fellows' clinic and, subsequently, the results of a quality improvement program in the residents' clinic. Pre-intervention rates of quality measure adherence were compared between the IM teaching clinic (n = 451) and pulmonary fellows' clinic (n = 177). Patient encounters in the residents' teaching clinic after quality improvement intervention (n = 119) were reviewed and compared with pre-intervention data. Prior to intervention, fellows were significantly more likely to offer smoking cessation counseling (p = 0.024) and document spirometry showing airway obstruction (p < 0.001). Smoking cessation counseling, pneumococcal vaccination, and diagnosis of COPD by spirometry were targets for QI. A single-cycle, resident-led QI project was initiated. After, residents numerically improved in the utilization of spirometry (66.5% vs. 74.8%) and smoking cessation counseling (81.8% vs. 86.6%), and significantly improved rates of pneumococcal vaccination (p = 0.024). One cycle of resident-led QI significantly improved the rates of pneumococcal vaccination, with numerical improvement in other areas of COPD care.

Interventions to increase pediatric vaccine uptake: An overview of recent findings

Although much is known about factors contributing to variation in pediatric immunization uptake, there is a need for synthesis of effective vaccine promotion strategies. With growing public health concern on how to best sustain high pediatric immunization rates, and improve where the rates are suboptimal, this review offers evidence gathered from several studies on the achievement of these goals. We identified and analyzed reported findings on childhood (primarily ≤ 7 years) immunization outcomes from tested intervention strategies that focused on parents, guardians, and caregivers, as well as providers, clinics/practices, and communities. The findings suggest that targeted and tailored interventions offer substantial possibilities, especially in a combined manner. We describe promising intervention models that have been operationalized with success and provide evidence for scalability across contexts. Moreover, they are sensitive to parents' and providers' needs, are feasibly integrated in daily clinical practice, and account for broader community concerns and issues.

How we used a patient visit tracker tool to advance experiential learning in systems-based practice and quality improvement in a medical student clinic

Poorly designed healthcare systems increase costs and preventable medical errors. To address these issues, systems-based practice (SBP) education provides future physicians with the tools to identify systemic errors and implement quality improvement (QI) initiatives to enhance the delivery of cost-effective, safe and multi-disciplinary care. Although SBP education is being implemented in residency programs and is mandated by the Accreditation Council for Graduate Medical Education (ACGME) as one of its core competencies, it has largely not been integrated into undergraduate medical education. We propose that Medical Student-Faculty Collaborative Clinics (MSFCCs) may be the ideal environment in which to train medical students in SBPs and QI initiatives, as they allow students to play pivotal roles in project development, administration, and management. Here we describe a process of experiential learning that was developed within a newly established MSFCC, which challenged students to identify inefficiencies, implement interventions, and track the results. After identifying bottlenecks in clinic operations, our students designed a patient visit tracker tool to monitor clinic flow and implemented solutions to decrease patient visit times. Our model allowed students to drive their own active learning in a practical clinical setting, providing early and unique training in crucial QI skills.

Assessing EM Patient Safety and Quality Improvement Milestones Using a Novel Debate Format

Graduate medical education is increasingly focused on patient safety and quality improvement; training programs must adapt their curriculum to address these changes. We propose a novel curriculum for emergency medicine (EM) residency training programs specifically addressing patient safety, systems-based management, and practice-based performance improvement, called “EM Debates.” Following implementation of this educational curriculum, we performed a cross-sectional study to evaluate the curriculum through resident self-assessment. Additionally, a cross-sectional study to determine the ED clinical competency committee’s (CCC) ability to assess residents on specific competencies was performed. Residents were overall very positive towards the implementation of the debates. Of those participating in a debate, 71% felt that it improved their individual performance within a specific topic, and 100% of those that led a debate felt that they could propose an evidence-based approach to a specific topic. The CCC found that it was easier to assess milestones in patient safety, systems-based management, and practice-based performance improvement (sub-competencies 16, 17, and 19) compared to prior to the implementation of the debates. The debates have been a helpful venue to teach EM residents about patient safety concepts, identifying medical errors, and process improvement.

Republished: Key characteristics of successful quality improvement curricula in physician education: a realist review

PURPOSE

Quality improvement (QI) is a common competency that must be taught in all physician training programmes, yet, there is no clear best approach to teach this content in clinical settings. We conducted a realist systematic review of the existing literature in QI curricula within the clinical setting, highlighting examples of trainees learning QI by doing QI.

METHOD

Candidate theories describing successful QI curricula were articulated a priori. We searched MEDLINE (1 January 2000 to 12 March 2013), the Cochrane Library (2013) and Web of Science (15 March 2013) and reviewed references of prior systematic reviews. Inclusion criteria included study design, setting, population, interventions, clinical and educational outcomes. The data abstraction tool included categories for setting, population, intervention, outcomes and qualitative comments. Themes were iteratively developed and synthesised using realist review methodology. A methodological quality tool assessed the biases, confounders, secular trends, reporting and study quality.

RESULTS

Among 39 studies, most were before-after design with resident physicians as the primary population. Twenty-one described clinical interventions and 18 described educational interventions with a mean intervention length of 6.58 (SD=9.16) months. Twenty-eight reported successful clinical improvements; no studies reported clinical outcomes that worsened. Characteristics of successful clinical QI curricula include attention to the interface of educational and clinical systems, careful choice of QI work for the trainees and appropriately trained local faculty.

CONCLUSIONS

This realist review identified success characteristics to guide training programmes, medical schools, faculty, trainees, accrediting organisations and funders to further develop educational and improvement resources in QI educational programmes.

Impact of a competency based curriculum on quality improvement among internal medicine residents

BACKGROUND

Teaching quality improvement (QI) principles during residency is an important component of promoting patient safety and improving quality of care. The literature on QI curricula for internal medicine residents is limited. We sought to evaluate the impact of a competency based curriculum on QI among internal medicine residents.

METHODS

This was a prospective, cohort study over four years (2007-2011) using pre-post curriculum comparison design in an internal medicine residency program in Canada. Overall 175 post-graduate year one internal medicine residents participated. A two-phase, competency based curriculum on QI was developed with didactic workshops and longitudinal, team-based QI projects. The main outcome measures included self-assessment, objective assessment using the Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess QI knowledge, and performance-based assessment via presentation of longitudinal QI projects.

RESULTS

Overall 175 residents participated, with a response rate of 160/175 (91%) post-curriculum and 114/175 (65%) after conducting their longitudinal QI project. Residents' self-reported confidence in making changes to improve health increased and was sustained at twelve months post-curriculum. Self-assessment scores of QI skills improved significantly from pre-curriculum (53.4 to 69.2 percent post-curriculum [p-value 0.002]) and scores were sustained at twelve months after conducting their longitudinal QI projects (53.4 to 72.2 percent [p-value 0.005]). Objective scores using the QIKAT increased post-curriculum from 8.3 to 10.1 out of 15 (p-value for difference <0.001) and this change was sustained at twelve months post-project with average individual scores of 10.7 out of 15 (p-value for difference from pre-curriculum <0.001). Performance-based assessment occurred via presentation of all projects at the annual QI Project Podium Presentation Day.

CONCLUSION

The competency based curriculum on QI improved residents' QI knowledge and skills during residency training. Importantly, residents perceived that their QI knowledge improved after the curriculum and this also correlated to improved QIKAT scores. Experiential QI project work appeared to contribute to sustaining QI knowledge at twelve months.

Combining quality improvement and geriatrics training: the nursing home polypharmacy outcomes project

To examine sustained effects of an educational intervention, the authors repeated a successful quality improvement (QI) project on medication safety and cost effectiveness. In October 2007 and August 2008, the facility leadership and geriatrics faculty identified all patients receiving nine or more medications (polypharmacy cohort) in a 170-bed teaching nursing home. They then taught Geriatric Medicine fellows (n = 12 in 2007, 11 in 2008) to (a) systematically collect medication data; (b) generate medication recommendations (stop, taper, or continue) based on expert criteria (Beers criteria) or drug-drug interaction programs; (c) discuss recommendations with patients' attending physicians; and (d) implement approved recommendations. Over the two projects, the polypharmacy cohorts demonstrated decreased potentially inappropriate medications (odds ratio [OR] = .78, 95% confidence interval [95% CI] [0.69, 0.88], p < .001), contraindicated medications (OR = .63, 95% CI [0.47, 0.85], p = .002) and medication costs (OR = .97, 95% CI [0.96, 0.99], p < .001). Findings suggest that programs planning educational QI projects for trainees may benefit from a multiyear approach to maximize clinical and educational benefits.

Quality improvement educational practices in pediatric residency programs: survey of pediatric program directors

BACKGROUND

The Accreditation Council for Graduate Medical Education requires residents to learn quality improvement (QI) methods to analyze, change, and improve their practice. Little is known about how pediatric residency programs design, implement, and evaluate QI curricula to achieve this goal. We sought to describe current QI educational practices, evaluation methods, and program director perceptions through a national survey.

METHODS

A survey of QI curricula was developed, pilot tested, approved by the Association of Pediatric Program Directors (APPD), and distributed to pediatric program directors. Descriptive statistics were used to analyze the data.

RESULTS

The response rate was 53% (104 of 197). Most respondents reported presence of a QI curriculum (85%, 88 of 104), including didactic sessions (83%) and resident QI projects (88%). Continuous process improvement was the most common methodology addressed (65%). The most frequent topics taught were "Making a Case for QI" (68%), "PDSA [plan-do-study-act] Cycles" (66%), and "Measurement in QI" (60%). Projects were most frequently designed to improve clinical care (90%), hospital operations (65%), and the residency (61%). Only 35% evaluated patient outcomes, and 17% had no formal evaluation. Programs had a mean of 6 faculty members (standard deviation 4.4, range 2-20) involved in teaching residents QI. Programs with more faculty involved were more likely to have had a resident submit an abstract to a professional meeting about their QI project (<5 faculty, 38%; 5-9, 64%; >9, 92%; P = .003). Barriers to teaching QI included time (66%), funding constraints (39%), and absent local QI expertise (33%). Most PPDs (65%) believed that resident input in hospital QI was important, but only 24% reported resident involvement. Critical factors for success included an experiential component (56%) and faculty with QI expertise (50%).

CONCLUSIONS

QI curricular practices vary greatly across pediatric residency programs. Although pediatric residency programs commit a fair number of resources to QI education and believe that resident involvement in QI is important, fundamental QI topics are overlooked in many programs, and evaluation of existing curricula is limited. Success as perceived by pediatric program directors appears to be related to the inclusion of a QI project and the availability of faculty mentors.

Quality improvement skills for pediatric residents: from lecture to implementation and sustainability

Quality improvement (QI) skills are relevant to efforts to improve the health care system. The Accreditation Council for Graduate Medical Education (ACGME) program requirements call for resident participation in local and institutional QI efforts, and the move to outcomes-based accreditation is resulting in greater focus on the resulting learning and clinical outcomes. Many programs have enhanced practice-based learning and improvement (PBLI) and systems based practice (SBP) curricula, although efforts to actively involve residents in QI activities appear to be lagging. Using information from the extensive experience of Cincinnati Children's Hospital Medical Center, we offer recommendations for how to create meaningful QI experiences for residents meet ACGME requirements and the expectations of the Clinical Learning Environment Review (CLER) process. Resident involvement in QI requires a multipronged approach that overcomes barriers and limitations that have frustrated earlier efforts to move this education from lectures to immersion experiences at the bedside and in the clinic. We present 5 dimensions of effective programs that facilitate active resident participation in improvement work and enhance their QI skills: 1) providing curricula and education models that ground residents in QI principles; 2) ensuring faculty development to prepare physicians for their role in teaching QI and demonstrating it in day-to-day practice; 3) ensuring all residents receive meaningful QI education and practical exposure to improvement projects; 4) overcoming time and other constraints to allow residents to apply their newly developed QI skills; and 5) assessing the effect of exposure to QI on resident competence and project outcomes.

Using lean methodology to teach quality improvement to internal medicine residents at a safety net hospital

The overall objective of this initiative was to develop a quality improvement (QI) curriculum using Lean methodology for internal medicine residents at Boston Medical Center, a safety net academic hospital. A total of 90 residents and 8 School of Public Health students participated in a series of four, 60- to 90-minute interactive and hands-on QI sessions. Seventeen QI project plans were created and conducted over a 4-month period. The curriculum facilitated internal medicine residents' learning about QI and development of positive attitudes toward QI (assessed using pre- and post-attitude surveys) and exposed them to an interprofessional team structure that duplicates future working relationships. This QI curriculum can be an educational model of how health care trainees can work collaboratively to improve health care quality.

Trends in medical error education: are we failing our residents?

BACKGROUND

The Institute of Medicine has called for physician education as a key step in medical error prevention. In our 2002 national survey, pediatric resident education about medical error prevention was sporadic. We sought to describe the amount and type of pediatric resident training about medical errors and to assess the change in training since 2002.

METHODS

We surveyed a national sample of 50 pediatric chief residents randomly selected from the 198 Accreditation Council for Graduate Medical Education-accredited residency programs from August to November 2010. The 31-item telephone survey was developed from the 2002 survey, with the addition of 10 items about electronic learning and resident quality improvement projects. The survey included 4 domains: current patient safety curriculum, chief resident knowledge, learning from medical errors, and demographics.

RESULTS

We phoned 55 chief residents and contacted 51. Fifty participated (90% participation rate). Ninety-four percent of chief residents stated that their program had a formalized curriculum to discuss medical errors, compared to only 50% (P < .001) in 2002. Ninety-six percent understood that the response to a medical error should be systemic change. The primary method for educating residents about medical error reported was informal teaching. Ninety-two percent reported never or rarely discussing medical error in an outpatient setting. Seventy-four percent of chief residents reported that they never or rarely learn from an error made by an attending physician, and 50% never or rarely learned from an error made by a fellow resident.

CONCLUSIONS

Although resident education about medical errors has improved since 2002, opportunities to model learning from mistakes are frequently missed.

An alternative practice model: residents transform continuity clinic and become systems thinkers

BACKGROUND

A changing health care environment has created a need for physicians trained in health system improvement. Residency programs have struggled to teach and assess practice-based learning and improvement and systems-based practice competencies, particularly within ambulatory settings.

INTERVENTION

We describe a resident-created and resident-led quality and practice-improvement council in an internal medicine continuity clinic. We conducted focus groups and report on residents' perspectives on council membership, practice management experiences, quality improvement projects, and resident satisfaction.

METHOD

Focus groups were held from May 2009 to March 2010 with internal medicine residents (N  =  5/focus group) who participated in the Continuity Clinic Ownership in Resident Education (CCORE) council. Data were analyzed with a grounded theory approach.

RESULTS

DURING THE FOCUS GROUPS, RESIDENTS RESPONDED TO THE QUESTION: "Do you have any new insights into delivering quality patient care in an outpatient clinic as a result of this experience (CCORE membership)?" The qualitative analysis resulted in 6 themes: systems thinking and systems-based care skills; improving quality of patient care; improved clinic efficiency; ownership of patients; need for improved communication of practice changes; and a springboard for research.

CONCLUSIONS

CCORE residents participated in system changes and acquired leadership skills while working on practice-based and system problems in a clinic microsystem. We believe this model can be implemented by other residency programs to promote the development of systems thinking in residents, increase their ownership of continuity clinic, and empower them to implement system changes.

Quality assurance in postgraduate medical education: implications for dermatology residency training programs

BACKGROUND

In the past few years, quality assurance has become an increasingly important part of medical education for both Canadian and American training programs. Since this emphasis on quality assurance in residency programs is recent, most faculty members involved in teaching residents in dermatology training programs would not themselves have had experience with quality assurance. As a result, satisfying this requirement may be a challenge.

OBJECTIVES

In this article, we review published reports in which various residency training programs have satisfied this requirement and propose projects in which dermatology residency training programs may satisfy quality assurance requirements.

METHODS

Using the key words residency, training, project, quality, assurance, improvement, medical errors, and safety, a literature search was conducted of English-language articles published after January 1990.

RESULTS/CONCLUSIONS

There are many innovative and effective ways program directors in dermatology training programs should be able to develop projects that improve patient care, enhance resident education, and fulfill accreditation requirements.

Quality improvement in medical education: current state and future directions

CONTEXT

During the last decade, there has been a drive to improve the quality of patient care and prevent the occurrence of avoidable errors. This review describes current efforts to teach or engage trainees in patient safety and quality improvement (QI), summarises progress to date, as well as successes and challenges, and lists our recommendations for the next steps that will shape the future of patient safety and QI in medical education.

CURRENT STATUS

Trainees encounter patient safety and QI through three main groups of activity. First are formal curricula that teach concepts or methods intended to facilitate trainees' participation in QI activities. These curricula increase learner knowledge and may improve clinical processes, but demonstrate limited capacity to modify learner behaviours. Second are educational activities that impart specific skills related to safety or quality which are considered to represent core doctor competencies (e.g. effective patient handover). These are frequently taught effectively, but without emphasis on the general safety or quality principles that inform the relevant skills. Third are real-life QI initiatives that involve trainees as active or passive participants. These innovative approaches expose trainees to safety and quality by integrating QI activities into trainees' day-to-day work. However, this integration can be challenging and can sometimes result in tension with broader educational goals.

FUTURE DIRECTIONS

To prepare the next generation of doctors to make meaningful contributions to the quality mission, we propose the following call to action. Firstly, a major effort to build faculty capacity, especially among teachers of QI, should be instigated. Secondly, accreditation standards and assessment methods, both during training and at end-of-training certification examinations, should explicitly target these competencies. Finally, and perhaps most importantly, we must refocus our attention at all levels of training and instil fundamental, collaborative, open-minded behaviours so that future clinicians are primed to promote a culture of safer, higher-quality care.

Primary care strategies to improve childhood immunisation uptake in developed countries: systematic review

Objectives

To conduct a systematic review of strategies to optimize immunisation uptake within preschool children in developed countries.

Design

Systematic review.

Setting

Developed countries

Participants

Preschool children who were due, or overdue, one or more of their routine primary immunisations.

Main outcome measures

Increase in the proportion of the target population up to date with standard recommended universal vaccinations.

Results

Forty-six studies were included for analysis, published between 1980 and 2009. Twenty-six studies were randomized controlled trials, 11 were before and after trials, and nine were controlled intervention trials. Parental reminders showed a statistically significant increase in immunisation rates in 34% of included intervention arms. These effects were reported with both generic and specific reminders and with all methods of reminders and recall. Strategies aimed at immunisation providers were also shown to improve immunisation rates with a median change in immunisation rates of 7% when reminders were used, 8% when educational programmes were used and 19% when feedback programmes were used.

Conclusion

General practitioners are uniquely positioned to influence parental decisions on childhood immunisation. A variety of strategies studied in primary care settings have been shown to improve immunisation rates, including parental and healthcare provider reminders.

Systems-based practice assessed with a performance-based examination simulated and scored by standardized participants in the health care system: feasibility and psychometric properties

BACKGROUND

Systems-based practice is one of the six general competencies proposed by the Accreditation Council for Graduate Medical Education in their Outcome Project. However, little has been published on its assessment--possibly because the systems-based practice competency has been viewed as difficult to define and measure.

PURPOSE

The purpose of this study was to determine whether a full performance-based examination of systems-based practice cases simulated and scored by standardized participants in the health care system could feasibly be constructed and implemented that would provide reliable and valid measurements.

METHODS

In the 1st year of the project (2008), four systems-based practice cases were developed and pilot tested with 13 residents. Videotapes of residents were studied to develop an instrument for subsequent assessment of performance by standardized participants. In the 2nd year (2009), the examination was expanded to a full 12 cases, which were completed by 11 second-year residents, and psychometric analyses were performed on the scores.

RESULTS

The generalizability coefficient for the full 12-case examination based on scoring by standardized participants was .71, which is nearly equal to that based on scoring by faculty physician observers, which was .78. The correlation between total scores obtained with standardized participants and physician observers was .78.

CONCLUSIONS

A performance-based examination can provide a feasible and reliable assessment of systems-based practice. However, attempts to evaluate convergent validity and discriminant validity-by correlating systems-based practice performance assessments with mean global ratings of residents on the 6 competencies by faculty throughout training-were unsuccessful, due to a lack of independence between the rated dimensions.

A resident-led quality improvement initiative to improve obesity screening

Instruction on quality improvement (QI) methods is required as part of residency education; however, there is limited evidence regarding whether internal medicine residents can improve patient care using these methods. Because obesity screening is not done routinely in clinical practice, residents aimed to improve screening using QI techniques. Residents streamlined body mass index (BMI) documentation, created educational materials about obesity, and launched an obesity screening QI initiative in a residency clinic. Residents designed plan-do-study-act cycles focused on increasing awareness and maintaining improvements in screening over a 1-year period. Documentation rates were collected at baseline, 2 weeks, 6 months, and 1 year post-intervention. At 1 year, obesity treatment rates also were collected. BMI documentation rates after 1 year were higher than baseline (43% vs 4%, P < .0001). In obese patients, BMI documentation was associated with lifestyle counseling (34% vs 14%, P < .01). An internal medicine resident-led QI project targeting obesity can improve screening.

A multidisciplinary approach for teaching systems-based practice to internal medicine residents

BACKGROUND

Rapid growth in the complexity of the health care environment (including monitoring systems for health care quality and patient safety) may result in graduating internists not being adequately prepared for the demands the system places on them. In response, the Residency Review Committee for Internal Medicine created the Educational Innovations Project (EIP) to encourage select residency training programs to develop new strategies and methods to meet changing demands in graduate medical education.

METHODS

As part of the EIP, our program created an innovative administrative internship. This multiyear curriculum provides systems-based practice training and consists of a series of rotations that take place during the 3 years of internal medicine residency. Each session involves close interaction with the nonphysician personnel who are instrumental in making our institution a functional and cohesive unit. To assess the potential impact of the rotations, we survey senior residents, recent graduates, and faculty educators. In conjunction with the Performance and Patient Experience departments of the hospital, we track several systems-based practice metrics for residents, including compliance with core health care measures, length of stay, and patient satisfaction.

RESULTS

Residents recognize the need to develop systems-based practice skills, to readily participate in structured curricula designed to enhance such skills, and to provide leadership in organizing and publishing quality improvement initiatives, and upon graduation, they may lament that they did not receive even more vigorous training in these areas.

CONCLUSION

Although internal medicine residencies continue to improve their training in systems-based practice, our experience suggests that an even greater emphasis on these skills may be warranted.

Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills

BACKGROUND

We developed a practice-based learning and improvement (PBLI) curriculum to address important gaps in components of content and experiential learning activities through didactics and participation in systems-level quality improvement projects that focus on making changes in health care processes.

METHODS

We evaluated the impact of our curriculum on resident PBLI knowledge, self-efficacy, and application skills. A quasi-experimental design assessed the impact of a curriculum (PBLI quality improvement systems compared with non-PBLI) on internal medicine residents' learning during a 4-week ambulatory block. We measured application skills, self-efficacy, and knowledge by using the Systems Quality Improvement Training and Assessment Tool. Exit evaluations assessed time invested and experiences related to the team projects and suggestions for improving the curriculum.

RESULTS

The 2 groups showed differences in change scores. Relative to the comparison group, residents in the PBLI curriculum demonstrated a significant increase in the belief about their ability to implement a continuous quality improvement project (P  =  .020), comfort level in developing data collection plans (P  =  .010), and total knowledge scores (P < .001), after adjusting for prior PBLI experience. Participants in the PBLI curriculum also demonstrated significant improvement in providing a more complete aim statement for a proposed project after adjusting for prior PBLI experience (P  =  .001). Exit evaluations were completed by 96% of PBLI curriculum participants who reported high satisfaction with team performance.

CONCLUSION

Residents in our curriculum showed gains in areas fundamental for PBLI competency. The observed improvements were related to fundamental quality improvement knowledge, with limited gain in application skills. This suggests that while heading in the right direction, we need to conceptualize and structure PBLI training in a way that integrates it throughout the residency program and fosters the application of this knowledge and these skills.

Perspective: Transformative learning: a framework using critical reflection to link the improvement competencies in graduate medical education

Quality improvement (QI) in health care involves activities ranging from enhancing personal practice to reforming the larger health care system. The Accreditation Council for Graduate Medical Education recognizes this broad definition of QI in its requirement that physicians-in-training demonstrate competence in practice-based learning and improvement (PBLI) and systems-based practice (SBP). Creative metaphors have been used to teach the PBLI and SBP competencies, but conceptual frameworks describing the relationship between these competencies are needed. Transformative learning is an adult education theory that states individuals must critically reflect on life events in order to change their beliefs or behaviors. The authors propose that critical reflection during transformative learning can conceptually link PBLI and SBP. Reflection on personal experience with suboptimal patient care leads to recognizing personal or system limitations. Addressing personal limitations improves individual practice (PBLI), whereas applying QI methodologies leads to large-scale improvements (SBP). Educators who adopt the transformative learning framework should be able to design meaningful QI curricula that encourage residents to be reflective and empower them with QI skills.

Teaching quality improvement and patient safety to trainees: a systematic review

PURPOSE

To systematically review published quality improvement (QI) and patient safety (PS) curricula for medical students and/or residents to (1) determine educational content and teaching methods, (2) assess learning outcomes achieved, and (3) identify factors promoting or hindering curricular implementation.

METHOD

Data sources included Medline (to January 2009), EMBASE, HealthSTAR, and article bibliographies. Studies selected reported curricula outlining specific educational content and teaching format. For articles with an evaluative component, the authors abstracted methodological features, such as study design. For all articles, they conducted a thematic analysis to identify factors influencing successful implementation of the included curricula.

RESULTS

Of 41 curricula that met the authors' criteria, 14 targeted medical students, 24 targeted residents, and 3 targeted both. Common educational content included continuous QI, root cause analysis, and systems thinking. Among 27 reports that included an evaluation, curricula were generally well accepted. Most curricula demonstrated improved knowledge. Thirteen studies (32%) successfully implemented local changes in care delivery, and seven (17%) significantly improved target processes of care. Factors that affected the successful curricular implementation included having sufficient numbers of faculty familiar with QI and PS content, addressing competing educational demands, and ensuring learners' buy-in and enthusiasm. Participants in some curricula also commented on discrepancies between curricular material and local institutional practice or culture.

CONCLUSIONS

QI and PS curricula that target trainees usually improve learners' knowledge and frequently result in changes in clinical processes. However, successfully implementing such curricula requires attention to a number of learner, faculty, and organizational factors.

Teaching and assessing systems-based practice: a pilot course in health care policy, finance, and law for radiation oncology residents

BACKGROUND

Under the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project, residency programs are required to provide data on educational outcomes and evidence for how this information is used to improve resident education.

OBJECTIVE

To teach and assess systems-based practice through a course in health care policy, finance, and law for radiation oncology residents, and to determine its efficacy.

METHODS AND MATERIALS

We designed a pilot course in health care policy, finance, and law related to radiation oncology. Invited experts gave lectures on policy issues important to radiation oncology and half of the participants attended the American Society for Therapeutic Radiation and Oncology (ASTRO) Advocacy Day. Participants completed pre- and postcourse tests to assess their knowledge of health policy.

RESULTS

Six radiation oncology residents participated, with 5 (84%) completing all components. For the 5 residents completing all assessments, the mean precourse score was 64% and the mean postcourse score was 84% (P  =  .05). Improvement was noted in all 3 sections of health policy, finance, and medical law. At the end of the course, 5 of 6 residents were motivated to learn about health policy, and 4 of 6 agreed it was important for physicians to be involved in policy matters.

CONCLUSIONS

Teaching radiation oncology residents systems-based practice through a course on health policy, finance, and law is feasible and was well received. Such a course can help teaching programs comply with the ACGME Outcome Project and would also be applicable to trainees in other specialties.

Abortion training at multiple sites: an unexpected curriculum for teaching systems-based practice

BACKGROUND

In 1999, the Accreditation Council for Graduate Medical Education endorsed systems-based practice as one of six general competencies.

PURPOSE

The objective is to explore the paradigm of teaching residents systems-based practice during a women's health rotation that included abortion training in multiple settings.

METHODS

During a routine women's health rotation, residents from two urban family medicine residency programs received early abortion training at a high-volume abortion clinic and their continuity clinic. Thirty-min semistructured interviews were conducted with all 26 residents who rotated between July 2005 and August 2006. Transcripts were analyzed using thematic codes.

RESULTS

Through exposure to different healthcare delivery systems, residents learned about systems-based practice, including understanding the failure of the larger system to meet patients' reproductive healthcare needs, differences between two systems, and potential systems barriers they might face as providers.

CONCLUSIONS

Abortion training in multiple settings may serve as a paradigm for teaching systems-based practice during other rotations that include training in multiple sites.

Residents' engagement in quality improvement: a systematic review of the literature

PURPOSE

Residents are being asked to participate in quality improvement (QI) initiatives in hospitals and clinics with increasing frequency; however, the effectiveness of improving patient care through residents' participation in QI initiatives is unknown.

METHOD

A thorough, systematic review of the English-language medical literature published between 1987 and October 2008 was performed to identify clinical QI initiatives in which there was active engagement of residents. Multiple search strategies were employed using PubMed, EMBASE, CINAHL, and ERIC. Articles were excluded in which residents played a passive or peripheral role in the QI initiative.

RESULTS

Twenty-eight articles were identified that documented residents' active leadership, development, or participation in a clinical QI initiative, such as curriculum change, clinical guideline implementation, or involvement with a clinical QI team. The role and participation of residents varied widely. Measures of patient health are described as outcomes in the QI initiatives of 5 of the 28 articles. Twenty-three articles described process improvements in patient care or residents' education as the outcome measure.

CONCLUSION

There are few articles that describe the clinical or educational effectiveness of residents' participation in QI efforts; the authors describe barriers that may be partly responsible. They conclude that there is a great need for additional research on the effectiveness of residents' participation in QI initiatives, particularly as they affect patient health outcomes.

Methodological rigor of quality improvement curricula for physician trainees: a systematic review and recommendations for change

PURPOSE

To systematically determine whether published quality improvement (QI) curricula for physician trainees adhere to QI guidelines and meet standards for study quality in medical education research.

METHOD

The authors searched MEDLINE, EMBASE, CINAHL, and ERIC between 1980 and April 2008 for physician trainee QI curricula and assessed (1) adherence to seven domains containing 35 QI objectives, and (2) study quality using the Medical Education Research Study Quality Instrument (MERSQI).

RESULTS

Eighteen curricula met eligibility criteria; 5 involved medical students and 13 targeted residents. Three curricula (18%) measured health care outcomes. Attitudes about QI were high, and many behavior and patient-related outcomes showed positive results. Curricula addressed a mean of 4.3 (SD 1.8) QI domains. Student initiatives included 38.2% [95% CI, 12.2%-64.2%] beginning student-level objectives and 23.0% [95% CI, -4.0% to 50.0%] advanced student-level objectives. Resident curricula addressed 42.3% [95% CI, 29.8%-54.8%] beginning resident-level objectives and 33.7% [95% CI, 23.2%-44.1%] advanced resident-level objectives. The mean (SD) total MERSQI score was 9.86 (2.92) with a range of 5 of 14 [total possible range 5-18]; 35% of curricula demonstrated lower study quality (MERSQI score < or = 7). Curricula varied widely in quality of reporting, teaching strategies, evaluation instruments, and funding obtained.

CONCLUSIONS

Many QI curricula in this study inadequately addressed QI educational objectives and had relatively weak research quality. Educators seeking to improve QI curricula should use recommended curricular and reporting guidelines, stronger methodologic rigor through development and use of validated instruments, available QI resources already present in health care settings, and outside funding opportunities.

Lessons learned from a 5-year experience with a 4-week experiential quality improvement curriculum in a preventive medicine fellowship

BACKGROUND

Competency in practice-based learning and improvement (PBLI) and systems-based practice (SBP) empowers learners with the skills to plan, lead, and execute health care systems improvement efforts. Experiences from several graduate medical education programs describe the implementation of PBLI and SBP curricula as challenging because of lack of adequate curricular time and faculty resources, as well as a perception that PBLI and SBP are not relevant to future careers. A dedicated experiential rotation that requires fellow participation in a specialty-specific quality improvement project (QIP) may address some of these challenges.

METHOD

We describe a retrospective analysis of our 5-year experience with a dedicated 3-week PBLI-SBP experiential curriculum in a preventive medicine fellowship program at Mayo Clinic, Rochester, Minnesota.

RESULTS

Between 2004 and 2008, 19 learners including 7 preventive medicine fellows participated in the rotation. Using just-in-time learning, fellows work together on a relatively complex QIP of community or institutional significance. Since 2004, all 19 learners (100%) participating in this rotation have consistently demonstrated statistically significant increase in their quality improvement knowledge application tool (QIKAT) scores at the end of the rotation. At the end of the rotation, all 19 learners stated that they were either confident or very confident of making a change to improve health care in a local setting. Most of the QIPs resulted in sustainable practice improvements, and resultant solutions have been disseminated beyond the location of the original QIP.

CONCLUSION

A dedicated experiential rotation that requires learner participation in a QIP is one of the effective methods to address the needs of the SBP and PBLI competencies.

Residents Learn to Improve Care Using the ACGME Core Competencies and Institute of Medicine Aims for Improvement: the Health Care Matrix

OBJECTIVES

This article describes how internal medicine residents at Vanderbilt University Medical Center learn to assess and improve care using the Institute of Medicine aims for improvement and the Accreditation Council for Graduate Medical Education core competencies combined in a tool called the health care matrix. The most important and popular use of the health care matrix has been with suboptimal care, in which care is not safe, timely, effective, efficient, equitable, or patient centered.

BACKGROUND

The core competencies provide a means of defining why care was not safe, timely, effective, efficient, equitable, or patient centered. The Institute of Medicine aims for improvement are also important because they are used to frame most publicly reported measures of quality. Few residents have an understanding of these public measures and how their futures will be affected by the growing trend toward quality report cards.

INTERVENTION

To help the residents understand the significance of public measures of quality, they learn to assess their patients as a "panel," looking at the care they provide for patients with coronary artery disease and diabetes mellitus. Residents use the health care matrix to analyze 1 of their patients, and then as a group they select a health care matrix for their improvement project. The way the health care matrix is formatted and the sequencing of the core competencies allow for the analysis of the cells to lead to the final question "What was learned and what needs to be improved?" The residents are then taught the tools and methods of quality improvement and complete their project. Some of these projects have had a significant influence on external measures of quality for this organization. The article describes the 8-week course that residents complete, the use of the health care matrix, the analysis of the patient panel, and finally an example of a completed project in which they improve the timeliness of antibiotics administration to patients with pneumonia (a public measure of quality).

Quality improvement and patient safety in the pediatric ambulatory setting: current knowledge and implications for residency training

The outpatient environment has been the leading edge of improvement work in pediatrics and it has similarly served as an effective locale for the training of pediatric residents in the science of improvement. This review summarizes what is known about the measurement of quality and patient safety in pediatric ambulatory settings. The current Accreditation Council for Graduate Medical Education (ACGME) requirements for resident training in improvement and their application in these settings are discussed. Some approaches and challenges to meeting these requirements are reviewed. Finally, some future directions that this work may follow are presented; the goal is to strengthen the effectiveness of improvement methods and their linkage to professional education.

Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review

PURPOSE

To evaluate published evidence that the Accreditation Council for Graduate Medical Education's six general competencies can each be measured in a valid and reliable way.

METHOD

In March 2008, the authors conducted searches of Medline and ERIC using combinations of search terms "ACGME," "Accreditation Council for Graduate Medical Education," "core competencies," "general competencies," and the specific competencies "systems-based practice" (SBP) and "practice based learning and improvement (PBLI)." Included were all publications presenting new qualitative or quantitative data about specific assessment modalities related to the general competencies since 1999; opinion pieces, review articles, and reports of consensus conferences were excluded. The search yielded 127 articles, of which 56 met inclusion criteria. Articles were subdivided into four categories: (1) quantitative/psychometric evaluations, (2) preliminary studies, (3) studies of SBP and PBLI, and (4) surveys.

RESULTS

Quantitative/psychometric studies of evaluation tools failed to develop measures reflecting the six competencies in a reliable or valid way. Few preliminary studies led to published quantitative data regarding reliability or validity. Only two published surveys met quality criteria. Studies of SBP and PBLI generally operationalized these competencies as properties of systems, not of individual trainees.

CONCLUSIONS

The peer-reviewed literature provides no evidence that current measurement tools can assess the competencies independently of one another. Because further efforts are unlikely to be successful, the authors recommend using the competencies to guide and coordinate specific evaluation efforts, rather than attempting to develop instruments to measure the competencies directly.

A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education

The Accreditation Council for Graduate Medical Education (ACGME) initiated its Outcome Project to better prepare physicians-in-training to practice in the rapidly changing medical environment and mandated assessment of competency in six outcomes, including Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP). Before the initiation of the Outcome Project, these competencies were not an explicit element of most graduate medical education training programs. Since 1999, directors of ACGME-accredited programs nationwide have been challenged to teach and assess these competencies. The authors describe an institution-wide curriculum intended to facilitate the teaching and assessment of PBLI and SBP competencies in the 115 ACGME-accredited residency and fellowship programs (serving 1,327 trainees) sponsored by Mayo School of Graduate Medical Education. Strategies to establish the curriculum in 2005 included development of a Quality Improvement (QI) curriculum Web site, one-on-one consultations with program directors, a three-hour program director workshop, and didactic sessions for residents and fellows on core topics. An interim program director self-assessment survey revealed a 13% increase in perceived ability to measure competency in SBP, no change in their perceived ability to measure competence in PBLI, a 15% increase in their ability to provide written documentation of competence in PBLI, and a 35% increase in their ability to provide written documentation of competence in SBP between 2005 and 2007. Nearly 70% of the programs had trainees participating in QI projects. Further research is needed to evaluate the cost-effectiveness of such a program and to measure its impact on learner knowledge, skills, and attitudes and, ultimately, on patient outcomes.

Utilizing Peer Academic Detailing to Improve Childhood Immunization Coverage Levels

Interventions that utilize academic detailing to improve childhood immunization have been implemented across the country. This study evaluates the effectiveness of an academic detailing intervention to increase childhood immunization rates in pediatric and family medicine practices in a major metropolitan area. Educational teams of one physician, nurse, and office manager delivered 83 peer education sessions at practices in the intervention group. Postintervention immunization rates for children 12-23 months of age increased 1% in the intervention group and decreased 3% in the control group. Postintervention coverage levels for children 12-23 months of age did not differ between the intervention and control groups. Results indicated this office-based intervention was not sufficient to effect measurable changes in immunization coverage levels after 1 year of participation. Future interventions need to provide initial feedback regarding practice immunization coverage levels prior to the educational interventions and include multiple encounters.

Teaching and evaluating group competency in systems-based practice in anesthesiology

BACKGROUND

Teaching and assessment of the systems-based practice competency has been problematic in hospital-based specialties such as anesthesiology. We developed a method to teach systems-based practice with collaborative team projects. The outcome was assessed with a tool that focused on group attributes.

METHODS

Resident teams chose projects that focused on the health care system. Projects included economic analyses, safety initiatives, process analyses, and policy revisions. Projects were presented by groups in poster discussion sessions. The educational program was evaluated using five criteria: implementation, awareness and acceptance in the organization, utility, sustainability, and diffusion to other programs.

RESULTS

The plan was implemented in 2005 and remains a required part of the resident curriculum. Key hospital and medical school leaders in our health care system participated in projects. Interdisciplinary collaboration occurred with multiple clinical departments. Nine projects performed economic analysis, 5 involved safety initiatives, 10 performed process analysis and recommended change, and 4 affected policy change in the institution. The program has been sustainable and has been effective in creating multidisciplinary institutional policy.

CONCLUSIONS

We developed an innovative method to teach systems-based practice through a team-based project initiative. The projects appear to have had a positive impact on our health care organization. Our assessment tool for the project evaluated team, rather than individual, performance, which is crucial in this competency.

Learning About Systems-Based Practice in the Informal Curriculum: A Case Study in an Academic Pediatric Continuity Clinic

OBJECTIVE

Pediatric residents learn about systems-based practice (SBP) explicitly in the formal curriculum and implicitly in the informal curriculum as they engage in practice alongside physician faculty. Recent studies describe innovative ways to address SBP in the formal curriculum for SBP, but the informal curriculum has not been explored. We examined what, and how, third-year pediatric residents learn about SBP in the informal curriculum at one continuity clinic, and to consider how this learning aligns with the formal curriculum.

METHODS

A case study involving 10 third-year pediatric residents and 10 continuity preceptors was conducted at one continuity clinic, housed in a community-based, pediatric primary care center. Data were derived from 5 months (100 hours) of direct observation in the precepting room at the case clinic, semistructured interviews with residents (before and after observation) and with preceptors (after observation). Interview transcripts and notes from observation were inductively coded and analyzed for major themes.

RESULTS

Two themes emerged in the informal curriculum. Residents perceived "our system," the academic health system in which they trained and practiced as separate and distinct from the "real system," the larger, societal context of health care. Residents also understood SBP as a commitment to helping individual patients and families navigate the complexities of "our system," dealing with issues that concerned them.

CONCLUSIONS

Residents learn important lessons about SBP in the informal curriculum in continuity clinic. These lessons may reinforce some elements of the competency-based formal curriculum for SBP, but challenge others.

Effectiveness of an Educational Intervention in Oral Health for Pediatric Residents

OBJECTIVE

To evaluate the effectiveness of an oral health educational intervention designed to increase proficiency of pediatric residents in oral health.

METHODS

Pediatric residents at the University of North Carolina at Chapel Hill (UNC) participated in oral health education that included didactic sessions, hands-on instruction by pediatric dentists and residents, preventive dentistry prompts, and change strategies to introduce oral health into practice. Pediatric residents at East Carolina University (ECU), who had a short practicum in oral health, and Wake Forest University (WFU), who had no specific oral health instruction, served as comparison groups. All residents completed questionnaires before and 12 months after instruction began at UNC. Effects were tested for each school separately by repeated-measure analysis of variance.

RESULTS

The mean percentage of UNC residents who answered 18 knowledge questions correctly and reported frequently performing 10 preventive dental practices increased by 17.7% and 65.1%, respectively, from baseline levels. Residents' confidence in performing 10 counseling and oral health screening activities improved by 17.9%. Opinions about including oral health in their clinical care improved only slightly. Changes at ECU and WFU were small, but low response rates in those schools preclude substantive conclusions from between schools comparisons.

CONCLUSIONS

Multifaceted instruction in oral health was effective in improving pediatric residents' knowledge about oral health, their confidence in providing oral health services, and the delivery of these services in their ambulatory care practices. Residents also adopted the use of fluoride varnish, an innovation in pediatrics. More studies are needed to define the most efficient and effective residency-based instruction.