Teaching systems-based practice to residents by using independent study projects

Putting residents in the office: an effective method to teach the systems-based practice competency

OBJECTIVES

Systems-based practice (SBP) was 1 of 6 core competencies established by the Accreditation Council for Graduate Medical Education and has proven to be one of the most difficult to effectively implement. This pilot study presents an immersion workshop as an effective tool to teach the SBP competency in a way that could easily be integrated into a residency curriculum.

DESIGN

In 2006, 16 surgical residents rotated through 3 stations for 30 minutes each: coding and billing, scheduling operations and return appointments, and patient check-in. Participants were administered a pretest and posttest questionnaire evaluating their knowledge of SBP, and were asked to evaluate the workshop.

SETTING

Outpatient clinic at MedStar Georgetown University Hospital, Washington, DC.

PARTICIPANTS

Residents in the general surgery residency training program at MedStar Georgetown University Hospital.

RESULTS

Most residents (62.5%) improved their score after the workshop, whereas 31.25% showed no change and 6.25% demonstrated a decrease in score. Overall within their training levels, all groups demonstrated an increase in mean test score. Postgraduate year-2 residents demonstrated the greatest change in mean score (20%), whereas postgraduate year-4 residents demonstrated the smallest change in mean score (3.3%).

CONCLUSIONS

An immersion workshop where general surgery residents gained direct exposure to SBP concepts in situ was an effective and practical method of integrating this core competency into the residency curriculum. Such a workshop could complement more formal didactic teaching and be easily incorporated into the curriculum. For example, this workshop could be integrated into the ambulatory care requirement that each resident must fulfill as part of their clinical training.

Where do we go from here? Moving from systems-based practice process measures to true competency via developmental milestones

For many educators it has been challenging to meet the Accreditation Council for Graduate Medical Education's requirements for teaching systems-based practice (SBP). An additional layer of complexity for educators is evaluating competency in SBP, despite milestones and entrustable professional activities (EPAs). In order to address this challenge, the authors present the results of a literature review for how SBP is currently being taught and a series of recommendations on how to achieve competency in SBP for graduate medical trainees with the use of milestones. The literature review included 29 articles and demonstrated that only 28% of the articles taught more than one of the six core principles of SBP in a meaningful way. Only 7% of the articles received the highest grade of A. The authors summarize four guiding principles for creating a competency-based curriculum that is in alignment with the Next Accreditation System (NAS): 1) the curriculum needs to include all of the core principles in that competency, 2) the objectives of the curriculum should be driven by clinical outcomes, 3) the teaching modalities need to be interactive and clinically relevant, and 4) the evaluation process should be able to measure competency and be directly reflective of pertinent milestones and/or EPAs. This literature review and the provided guiding principles can guide other residency educators in their development of competency-based curricula that meets the standards of the NAS.

Development and initial validation of a project-based rubric to assess the systems-based practice competency of residents in the clinical chemistry rotation of a pathology residency

CONTEXT

Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature.

OBJECTIVE

To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry.

DESIGN

Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects.

RESULTS

Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability.

CONCLUSIONS

We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

A pilot study of a practice management training module for medical residents

BACKGROUND

In 2005 a competency based curriculum was introduced in the Dutch postgraduate medical training programs. While the manager's role is one of the seven key competencies, there is still no formal management course in most postgraduate curricula. Based on a needs assessment we conducted, several themes were identified as important for a possible management training program. We present the results of the pilot training we performed to investigate two of these themes.

METHODS

The topics "knowledge of the healthcare system" and "time management" were developed from the list of suggested management training themes. Fourteen residents participated in the training and twenty-four residents served as control. The training consisted of two sessions of four hours with a homework assignment in between. 50 True/false-questions were given as pre- and post-test to both the test and control groups to assess the level of acquired knowledge among the test group as well as the impact of the intervention. We also performed a qualitative evaluation using evaluation forms and in-depth interviews.

RESULTS

All fourteen residents completed the training. Six residents in the control group were lost to follow up. The pre- and post-test showed improvement among the participating residents in comparison to the residents from the control group, but this improvement was not significant. The qualitative assessment showed that all residents evaluated the training positively and experienced it as a useful addition to their training in becoming a medical specialist.

CONCLUSION

Our training was evaluated positively and considered to be valuable. This study supports the need for mandatory medical management training as part of the postgraduate medical curriculum. Our training could be an example of how to teach two important themes in the broad area of medical management education.

Development and participant assessment of a practical quality improvement educational initiative for surgical residents

BACKGROUND

As patient-safety and quality efforts spread throughout health care, the need for physician involvement is critical, yet structured training programs during surgical residency are still uncommon. Our objective was to develop an extended quality-improvement curriculum for surgical residents that included formal didactics and structured practical experience.

METHODS

Surgical trainees completed an 8-hour didactic program in quality-improvement methodology at the start of PGY3. Small teams developed practical quality-improvement projects based on needs identified during clinical experience. With the assistance of the hospital's process-improvement team and surgical faculty, residents worked through their selected projects during the following year. Residents were anonymously surveyed after their participation to assess the experience.

RESULTS

During the first 3 years of the program, 17 residents participated, with 100% survey completion. Seven quality-improvement projects were developed, with 57% completing all DMAIC (Define, Measure, Analyze, Improve, Control) phases. Initial projects involved issues of clinical efficiency and later projects increasingly focused on clinical care questions. Residents found the experience educationally important (65%) and believed they were well equipped to lead similar initiatives in the future (70%). Based on feedback, the timeline was expanded from 12 to 24 months and changed to start in PGY2.

CONCLUSIONS

Developing an extended curriculum using both didactic sessions and applied projects to teach residents the theory and implementation of quality improvement is possible and effective. It addresses the ACGME competencies of practice-based improvement and learning and systems-based practice. Our iterative experience during the past 3 years can serve as a guide for other programs.

Bedside Teaching on Time to Disposition Improves Length of Stay for Critically-ill Emergency Departments Patients

INTRODUCTION

We tested the effect of a brief disposition process intervention on residents' time to disposition and emergency department (ED) length of stay (LOS) in high acuity ED patients.

METHODS

This was a quasi-experimental study design in a single teaching hospital where ED residents are responsible for administrative bed requests for patients. Enrollment was performed for intervention and control groups on an even-odd day schedule. Inclusion criteria were ED patients triaged as Emergency Severity Index (ESI) 1 and 2. In the intervention group, the attending physician prompted the resident to make the disposition immediately after the evaluation of resuscitation patients. In the control group, the attending physicians did not intervene in the disposition process unless more than 2 hours passed without a disposition. Main outcomes were time to disposition and total ED LOS.

RESULTS

A total of 104 patients were enrolled; 53 (51%) in the intervention group and 51 (49%) in the control group. After controlling for ESI and resident training year, mean disposition time was significantly shorter in the intervention group by 41.4 minutes (95% CI: 32.6-50.1). LOS was also shorter in the intervention group by 93.3 minutes (95% CI: 41.9-144.6).

CONCLUSION

Prompting residents to enter administrative disposition orders in high acuity patients is associated with significant reduction in both time to disposition and ED LOS.

Using lean methodology to teach quality improvement to internal medicine residents at a safety net hospital

The overall objective of this initiative was to develop a quality improvement (QI) curriculum using Lean methodology for internal medicine residents at Boston Medical Center, a safety net academic hospital. A total of 90 residents and 8 School of Public Health students participated in a series of four, 60- to 90-minute interactive and hands-on QI sessions. Seventeen QI project plans were created and conducted over a 4-month period. The curriculum facilitated internal medicine residents' learning about QI and development of positive attitudes toward QI (assessed using pre- and post-attitude surveys) and exposed them to an interprofessional team structure that duplicates future working relationships. This QI curriculum can be an educational model of how health care trainees can work collaboratively to improve health care quality.

A multidisciplinary approach for teaching systems-based practice to internal medicine residents

BACKGROUND

Rapid growth in the complexity of the health care environment (including monitoring systems for health care quality and patient safety) may result in graduating internists not being adequately prepared for the demands the system places on them. In response, the Residency Review Committee for Internal Medicine created the Educational Innovations Project (EIP) to encourage select residency training programs to develop new strategies and methods to meet changing demands in graduate medical education.

METHODS

As part of the EIP, our program created an innovative administrative internship. This multiyear curriculum provides systems-based practice training and consists of a series of rotations that take place during the 3 years of internal medicine residency. Each session involves close interaction with the nonphysician personnel who are instrumental in making our institution a functional and cohesive unit. To assess the potential impact of the rotations, we survey senior residents, recent graduates, and faculty educators. In conjunction with the Performance and Patient Experience departments of the hospital, we track several systems-based practice metrics for residents, including compliance with core health care measures, length of stay, and patient satisfaction.

RESULTS

Residents recognize the need to develop systems-based practice skills, to readily participate in structured curricula designed to enhance such skills, and to provide leadership in organizing and publishing quality improvement initiatives, and upon graduation, they may lament that they did not receive even more vigorous training in these areas.

CONCLUSION

Although internal medicine residencies continue to improve their training in systems-based practice, our experience suggests that an even greater emphasis on these skills may be warranted.

Abortion training at multiple sites: an unexpected curriculum for teaching systems-based practice

BACKGROUND

In 1999, the Accreditation Council for Graduate Medical Education endorsed systems-based practice as one of six general competencies.

PURPOSE

The objective is to explore the paradigm of teaching residents systems-based practice during a women's health rotation that included abortion training in multiple settings.

METHODS

During a routine women's health rotation, residents from two urban family medicine residency programs received early abortion training at a high-volume abortion clinic and their continuity clinic. Thirty-min semistructured interviews were conducted with all 26 residents who rotated between July 2005 and August 2006. Transcripts were analyzed using thematic codes.

RESULTS

Through exposure to different healthcare delivery systems, residents learned about systems-based practice, including understanding the failure of the larger system to meet patients' reproductive healthcare needs, differences between two systems, and potential systems barriers they might face as providers.

CONCLUSIONS

Abortion training in multiple settings may serve as a paradigm for teaching systems-based practice during other rotations that include training in multiple sites.

Addressing the systems-based practice requirement with health policy content and educational technology

Duke University Hospital Office of Graduate Medical Education and Duke University's Fuqua School of Business collaborated to offer a Health Policy lecture series to residents and fellows across the institution, addressing the "Systems-based Practice" competency.During the first year, content was offered in two formats: live lecture and web/podcast. Participants could elect the modality which was most convenient for them. In Year Two, the format was changed so that all content was web/podcast and a quarterly live panel discussion was led by module presenters or content experts. Lecture evaluations, qualitative focus group feedback, and post-test data were analyzed.A total of 77 residents and fellows from 8 (of 12) Duke Graduate Medical Education departments participated. In the first year, post-test results were the same for those who attended the live lectures and those who participated via web/podcast. A greater number of individuals participated in Year Two. Participants from both years expressed the need for health policy content in their training programs. Participants in both years valued a hybrid format for content delivery, recognizing a desire for live interaction with the convenience of accessing web/podcasts at times and locations convenient for them. A positive unintended consequence of the project was participant networking with residents and fellows from other specialties.

Quality education: a pilot quality improvement curriculum for psychiatry residents

BACKGROUND

A series of Institute of Medicine's reports have highlighted the need for greater quality improvement (QI) training in medical education; however, few formal QI curricula for medical trainees have been described in the literature.

AIM

The objective of this study was to develop a contextual QI curriculum involving a QI workshop and longitudinal QI projects (QIPs) for psychiatry trainees.

METHODS

We examined psychiatry residents' attitudes on QI training following their exposure to a physician-manager curriculum using focus group methodology. Focus group data were used to inform revisions to the QI curriculum. Following the curriculum revisions, we administered a resident questionnaire to elicit resident perceptions on the modified QI curriculum.

RESULTS

Focus group data from 40 psychiatry residents at the University of Toronto identified the following themes: challenges with QIP workload, difficulties of QI workshop integration into the curriculum, and value of the experiential component of the QIP. Of the 26 residents, 18 completed the resident questionnaire on the revised curriculum and reported an enhanced appreciation of QI in their current clinical practice.

CONCLUSION

The study results suggest that this experiential format warrants further exploration as a model for QI training in medicine.

Internal medicine residency graduates' perceptions of the systems-based practice and practice-based learning and improvement competencies

BACKGROUND

Resident education in Systems-Based Practice (SBP) and Practice-Based Learning and Improvement (PBLI) is required but underemphasized.

PURPOSES

The objectives are to identify SBP and PBLI knowledge and skills with the most relevance to our graduates' practices and to determine how well they were prepared during residency training to address these issues.

METHODS

A survey was drafted based on Accreditation Council for Graduate Medical Education competency definitions and published literature on SBP and PBLI. Respondents indicated the extent to which each item is relevant to their practice and the adequacy of instruction received on a 5-point Likert scale.

RESULTS

All topics had high perceived relevance to practice with most topics rated low for adequacy of training. Topics of practice management and health care economics contained the largest gaps between mean ratings of relevance and adequacy of training (p < .001). Few differences in ratings were seen based on graduate demographics.

CONCLUSIONS

This survey has allowed us to prioritize SBP and PBLI curricula to meet the needs of our graduates.

Residents' engagement in quality improvement: a systematic review of the literature

PURPOSE

Residents are being asked to participate in quality improvement (QI) initiatives in hospitals and clinics with increasing frequency; however, the effectiveness of improving patient care through residents' participation in QI initiatives is unknown.

METHOD

A thorough, systematic review of the English-language medical literature published between 1987 and October 2008 was performed to identify clinical QI initiatives in which there was active engagement of residents. Multiple search strategies were employed using PubMed, EMBASE, CINAHL, and ERIC. Articles were excluded in which residents played a passive or peripheral role in the QI initiative.

RESULTS

Twenty-eight articles were identified that documented residents' active leadership, development, or participation in a clinical QI initiative, such as curriculum change, clinical guideline implementation, or involvement with a clinical QI team. The role and participation of residents varied widely. Measures of patient health are described as outcomes in the QI initiatives of 5 of the 28 articles. Twenty-three articles described process improvements in patient care or residents' education as the outcome measure.

CONCLUSION

There are few articles that describe the clinical or educational effectiveness of residents' participation in QI efforts; the authors describe barriers that may be partly responsible. They conclude that there is a great need for additional research on the effectiveness of residents' participation in QI initiatives, particularly as they affect patient health outcomes.

Taking a unified approach to teaching and implementing quality improvements across multiple residency programs: the Atlantic Health experience

The Accreditation Council for Graduate Medical Education recently emphasized the importance of systems-based practice and systems-based learning; however, successful models of collaborative quality improvement (QI) initiatives in residency training curricula are not widely available. Atlantic Health successfully conceptualized and implemented a QI collaborative focused on medication safety across eight residency training programs representing 219 residents. During a six-month period, key faculty and resident leaders from 8 (of 10) Atlantic Health residency training programs participated in three half-day collaborative learning sessions focused on improving medication reconciliation. Each session included didactic presentations from a multidisciplinary team of clinical experts as well as the application of principles that identified challenges, barriers, and solutions to QI initiatives. The learning sessions emphasized the fundamental principles of medication reconciliation, its critical importance as a vital part of patient handoff in all health care settings, and the challenges of achieving successful medication reconciliation improvement in light of work hours restrictions and patient loads. Each residency program developed a detailed implementation and measurement plan for individual "action learning" projects, using the Plan-Do-Study-Act method of improvement. Each program then implemented its QI project, and expert faculty (e.g., physicians, nurses, pharmacists, QI staff) provided mentoring between learning sessions. Several projects resulted in permanent changes in medication reconciliation processes, which were then adopted by other programs. The structure, process, and outcomes of this effort are described in detail.

A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education

The Accreditation Council for Graduate Medical Education (ACGME) initiated its Outcome Project to better prepare physicians-in-training to practice in the rapidly changing medical environment and mandated assessment of competency in six outcomes, including Practice-Based Learning and Improvement (PBLI) and Systems-Based Practice (SBP). Before the initiation of the Outcome Project, these competencies were not an explicit element of most graduate medical education training programs. Since 1999, directors of ACGME-accredited programs nationwide have been challenged to teach and assess these competencies. The authors describe an institution-wide curriculum intended to facilitate the teaching and assessment of PBLI and SBP competencies in the 115 ACGME-accredited residency and fellowship programs (serving 1,327 trainees) sponsored by Mayo School of Graduate Medical Education. Strategies to establish the curriculum in 2005 included development of a Quality Improvement (QI) curriculum Web site, one-on-one consultations with program directors, a three-hour program director workshop, and didactic sessions for residents and fellows on core topics. An interim program director self-assessment survey revealed a 13% increase in perceived ability to measure competency in SBP, no change in their perceived ability to measure competence in PBLI, a 15% increase in their ability to provide written documentation of competence in PBLI, and a 35% increase in their ability to provide written documentation of competence in SBP between 2005 and 2007. Nearly 70% of the programs had trainees participating in QI projects. Further research is needed to evaluate the cost-effectiveness of such a program and to measure its impact on learner knowledge, skills, and attitudes and, ultimately, on patient outcomes.

Effecting the sixth core competency: a project-based curriculum

OBJECTIVE

In agreement with the Accreditation Council for Graduate Medical Education guidelines, a systems-based practice (SBP) curriculum was implemented to provide a hands-on, team-based experience, while providing an opportunity to contribute to quality improvement, and to develop a method to assess residents' understanding of SBP.

STUDY DESIGN

During departmental conferences, issues affecting our health care operations were identified. Seven teams were formed and the actionable items were investigated. The salient issues and potential solutions were presented to the department.

RESULTS

This project resulted in the development of tools designed to assess competency in SBP, communication skills, and professionalism. Completion of a pre- and posttest and annual oral examination questions revealed a significant improvement in the comprehension of SBP.

CONCLUSION

This project served as an effective educational forum for the competencies of SBP, communication skills, and professionalism. This initiative also offered improvement of patient care and/or educating peer providers, thereby fulfilling the goal of SBP in the process of education.

Self-directed Learning in Health Professions Education

Introduction: Self-directed learning has been recommended as a promising methodology for lifelong learning in medicine. However, the concept of self-directed learning continues to be elusive, with students and educators finding difficulty in defining it and agreeing on its worth. Methods: In this paper we review the literature of self-directed learning in health professions education and present a framework based on Malcolm Knowles’ key components of self-directed learning. Results: The key components of self-directed learning are: the educator as a facilitator, identification of learning needs, development of learning objectives, identification of appropriate resources, implementation of the process, commitment to a learning contract and evaluation of learning. Several but not all of these components are often described in the published literature. Conclusion: Although the presented framework provides some consistency for educators interested in applying SDL methods, future studies are needed to standardise self-directed learning curricula and to determine the effectiveness of these components on educational outcomes. Key words: Independent study, Medical education, Self-assessment, Self-directed learning, Selfeducation

A self-instructional model to teach systems-based practice and practice-based learning and improvement

BACKGROUND

When mandated as resident competencies in 1999, systems-based practice (SBP) and practice-based learning and improvement (PBLI) were new concepts to many.

OBJECTIVE

To describe and evaluate a 4-week clinical elective (Achieving Competence Today-ACT) to teach residents SBP and PBLI.

DESIGN

ACT consisted of a four-week active learning course and follow-up teaching experience, guided and supported by web-based materials. The curriculum included readings, scheduled activities, work products including an improvement project, and weekly meetings with a non-expert preceptor. The evaluation used a before-after cross-comparison of ACT residents and their peers.

PARTICIPANTS

Seventy-eight residents and 42 faculty in 18 US Internal Medicine residency programs participated between 2003 and 2005. RESULTS AND MAIN MEASUREMENTS: All residents and faculty preceptors responded to a knowledge test, survey of attitudes, and self-assessment of competency to do 15 tasks related to SBP/PBLI. All measures were normalized to a 100-point scale. Each program's principal investigator (PI) identified aspects of ACT that were most and least effective in enhancing resident learning. ACT residents' gains in knowledge (4.4 on a 100-point scale) and self-assessed competency (11.3) were greater than controls' (-1.9, -8.0), but changes in attitudes were not significantly different. Faculty preceptors' knowledge scores did not change, but their attitudes became more positive (15.8). PIs found a ready-to-use curriculum effective (rated 8.5 on a 10-point scale).

CONCLUSIONS

ACT increased residents' knowledge and self-assessment of their own competency and raised faculty's assessment of the importance of residents' learning SBP/PBLI. Faculty content expertise is not required for residents to learn SBP/PBLI.

A multidisciplinary systems-based practice learning experience and its impact on surgical residency education

OBJECTIVE

To design and implement a multidisciplinary systems-based practice learning experience that is focused on improving and standardizing the preoperative quality of care for general surgical patients.

DESIGN

Four parameters of preoperative care were designated as quality assessment variables, including bowel preparation, perioperative beta-blockade, prophylactic antibiotic use, and deep venous thrombosis prevention. Four groups of general surgery residents (PGY I-V), each led by 1 chief resident, were assigned a quality parameter, performed an evidence-based current literature review, and formulated a standardized management approach based on the level of evidence and recommendations available. Because preoperative preparation includes anesthetic care and operating room preparation, we presented our findings at the Department of Surgery Grand Rounds in a multidisciplinary format that included presentations by each resident group, the Department of Anesthesia, the Department of Medicine, and the Department of Nursing. The aim of the multidisciplinary quality assurance conference was to present the evidence-based literature findings in order to determine how standardization of preoperative care would alter anesthetic and nursing care, and to obtain feedback about management protocols. To determine the educational impact of this model of integrated systems-based practice quality assessment on the teaching experience, residents were queried regarding the value of this educational venue and responses were rated on a Likert scale.

RESULTS

Resident participation was excellent. The residents garnered valuable information by performing a literature review and evaluating the best preoperative preparation given each parameter. Furthermore, integration of their findings into systems-based practice including anesthesia and nursing care provided an appreciation of the complexities of care as well as the associated need for appropriate medical knowledge, communication, and professionalism. The derivation of treatment protocols included an opportunity to incorporate several competencies across multiple disciplines. The residents evaluated 5 questions and deemed the educational exercise an effective model to enrich surgical resident education while simultaneously improving patient care. The residents also strongly agreed that they would participate in similar projects in the future as well as recommend this educational exercise to other residents. A finalized preoperative order set was created and distributed to all residents for use in the preoperative care of general surgery patients.

CONCLUSIONS

Our multidisciplinary systems-based practice learning experience focused on improving and standardizing the preoperative quality of care for patients, and general surgery residents were pivotal participants in that process. This exercise had a positive impact on our general surgery residency education program and proved to be a valuable model of systems-based practice competency.

Validation of a method for assessing resident physicians' quality improvement proposals

BACKGROUND

Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking.

OBJECTIVE

We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability.

DESIGN

QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised.

PARTICIPANTS

Seven raters used the instrument to assess 45 resident QI proposals.

MEASUREMENTS

Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively.

RESULTS

QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high.

CONCLUSIONS

This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Patient safety knowledge and its determinants in medical trainees

BACKGROUND

Patient safety is a core educational topic for medical trainees.

OBJECTIVES

To determine the current level and determinants of patient safety knowledge in medical trainees.

DESIGN

Multi-institutional cross-sectional assessment of patient safety knowledge.

PARTICIPANTS

Residents and medical students from seven Harvard-affiliated residencies and two Harvard Medical School courses.

MEASUREMENTS

Participants were administered a 14-item validated test instrument developed based on the patient safety curriculum of the Risk Management Foundation (Cambridge, MA). The primary outcome measure was the amount of patient safety knowledge demonstrated by trainees on the validated test instrument. The secondary outcome measure was their subjective perceptions as to their baseline knowledge level in this domain.

RESULTS

Ninety-two percent (640/693) of residents and medical students completed the patient safety test. Participants correctly answered a mean 58.4% of test items (SD 15.5%). Univariate analyses show that patient safety knowledge levels varied significantly by year of training (p = 0.001), degree program (p < 0.001), specialty (p < 0.001), country of medical school (p = 0.006), age (p < 0.001), and gender (p = 0.050); all but the latter two determinants remained statistically significant in multivariate models. In addition, trainees were unable to assess their own knowledge deficiencies in this domain.

CONCLUSIONS

Patient safety knowledge is limited among medical trainees across a broad range of training levels, degrees, and specialties. Effective educational interventions that target deficiencies in patient safety knowledge are greatly needed.

Learning About Systems-Based Practice in the Informal Curriculum: A Case Study in an Academic Pediatric Continuity Clinic

OBJECTIVE

Pediatric residents learn about systems-based practice (SBP) explicitly in the formal curriculum and implicitly in the informal curriculum as they engage in practice alongside physician faculty. Recent studies describe innovative ways to address SBP in the formal curriculum for SBP, but the informal curriculum has not been explored. We examined what, and how, third-year pediatric residents learn about SBP in the informal curriculum at one continuity clinic, and to consider how this learning aligns with the formal curriculum.

METHODS

A case study involving 10 third-year pediatric residents and 10 continuity preceptors was conducted at one continuity clinic, housed in a community-based, pediatric primary care center. Data were derived from 5 months (100 hours) of direct observation in the precepting room at the case clinic, semistructured interviews with residents (before and after observation) and with preceptors (after observation). Interview transcripts and notes from observation were inductively coded and analyzed for major themes.

RESULTS

Two themes emerged in the informal curriculum. Residents perceived "our system," the academic health system in which they trained and practiced as separate and distinct from the "real system," the larger, societal context of health care. Residents also understood SBP as a commitment to helping individual patients and families navigate the complexities of "our system," dealing with issues that concerned them.

CONCLUSIONS

Residents learn important lessons about SBP in the informal curriculum in continuity clinic. These lessons may reinforce some elements of the competency-based formal curriculum for SBP, but challenge others.

Interdepartmental problem-solving as a method for teaching and learning systems-based practice

RATIONALE AND OBJECTIVES

Systems-based practice (SBP) has been the most difficult competency to implement in radiology residency programs, and methods for teaching and learning SBP concepts are needed. Because systems problems are usually multifactorial, a multidisciplinary approach is required. In our institution, survey data indicated patient dissatisfaction with emergency care. Prolonged wait times for radiology procedures were identified as a systems problem. When hospital administration asked the emergency medicine and radiology departments to work together to improve patient care, residents had a "real-world" opportunity to achieve the SBP competency.

MATERIALS AND METHODS

Systems problems were identified and categorized. Data on patient transport were collected. Accurate time logs documenting when studies were ordered, performed, and interpreted were maintained. Data were analyzed at interdepartmental meetings and three improvements were planned and implemented.

RESULTS

A direct line of communication was established between radiology and emergency medicine via a dedicated cellular telephone. A joint emergency medicine/radiology teaching conference was established. Additional transport personnel were employed. Residents in both departments increased their understanding of their role in the health care system, demonstrated an ability to identify systems problems and appropriately implement improvements, and enhanced their professional relationships.

CONCLUSION

This innovative method integrated educational goals with patient care goals, was grounded in "real-life" experience, and held residents accountable for results. Competence in SBP, patient care, professionalism, and interpersonal and communication skills were demonstrated. We recommend this approach as an efficient and effective way to integrate SBP into everyday clinical practice.