Development and implementation of a systems-based practice project requirement for radiology residents

A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts?

BACKGROUND

With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes.

METHODS

A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts.

RESULTS

18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes.

CONCLUSION

This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.

Meeting ACGME and ABMS Quality Improvement Requirements in a Sleep Medicine Fellowship Program

STUDY OBJECTIVES

To describe a sustainable program of teaching and implementing quality improvement (QI) in a 12-month sleep medicine fellowship.

METHODS

We created a QI curriculum based on Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialty (ABMS) Part IV Maintenance of Certification (MOC) requirements for QI. Two program faculty with prior QI training volunteered to mentor fellows. Our institution's central QI office houses QI experts who teach QI across the health system. One of these experts, referred to as the "QI consultant," helped us adapt QI teaching materials to include 4 online modules and 5 group sessions. Fellows worked in teams to complete 2 data-guided QI cycles.

RESULTS

The curriculum required 29 hours for fellows, 18 hours for faculty, and 55 hours for the QI consultant; now that teaching materials have been created, the QI consultant's involvement will decrease over time. Post-curriculum surveys showed that fellows' knowledge of QI concepts increased, as did their confidence performing QI activities. Fellows' QI projects objectively improved timeliness and quality of care for patients. Sleep medicine fellows and QI faculty mentors evaluated the curriculum positively. The curriculum met ACGME requirements for QI, and fellows and mentoring faculty received ABMS Part IV MOC credit upon completion of the curriculum.

CONCLUSIONS

A QI curriculum can successfully be implemented into a 12-month sleep medicine fellowship to increase sleep medicine fellows' QI knowledge and confidence, meet ACGME and MOC requirements, and contribute to care of patients with sleep disorders.

Practical Implications for an Effective Radiology Residency Quality Improvement Program for Milestone Assessment

Utilization of a radiology resident-specific quality improvement (QI) program and curriculum based on the Accreditation Council for Graduate Medical Education (ACGME) milestones can enable a program's assessment of the systems-based practice component and prepare residents for QI implementation post graduation. This article outlines the development process, curriculum, QI committee formation, and resident QI project requirements of one institution's designated radiology resident QI program. A method of mapping the curriculum to the ACGME milestones and assessment of resident competence by postgraduate year level is provided. Sample projects, challenges to success, and lessons learned are also described. Survey data of current trainees and alumni about the program reveal that the majority of residents and alumni responders valued the QI curriculum and felt comfortable with principles and understanding of QI. The most highly valued aspect of the program was the utilization of a resident education committee. The majority of alumni responders felt the residency quality curriculum improved understanding of QI, assisted with preparation for the American Board of Radiology examination, and prepared them for QI in their careers. In addition to the survey results, outcomes of resident project completion and resident scholarly activity in QI are evidence of the success of this program. It is hoped that this description of our experiences with a radiology resident QI program, in accordance with the ACGME milestones, may facilitate the development of successful QI programs in other diagnostic radiology residencies.

Evidence-based Practice of Radiology

Current health care reform in the United States is producing a shift in radiology practice from the traditional volume-based role of performing and interpreting a large number of examinations to providing a more affordable and higher-quality service centered on patient outcomes, which is described as a value-based approach to the provision of health care services. In the 1990 s, evidence-based medicine was defined as the integration of current best evidence with clinical expertise and patient values. When these methods are applied outside internal medicine, the process is called evidence-based practice (EBP). EBP facilitates understanding, interpretation, and application of the best current evidence into radiology practice, which optimizes patient care. It has been incorporated into "Practice-based Learning and Improvement" and "Systems-based Practice," which are two of the six core resident competencies of the Accreditation Council for Graduate Medical Education and two of the 12 American Board of Radiology milestones for diagnostic radiology. Noninterpretive skills, such as systems-based practice, are also formally assessed in the "Quality and Safety" section of the American Board of Radiology Core and Certifying examinations. This article describes (a) the EBP framework, with particular focus on its relevance to the American Board of Radiology certification and maintenance of certification curricula; (b) how EBP can be integrated into a residency program; and (c) the current value and likely place of EBP in the radiology information technology infrastructure. Online supplemental material is available for this article.

Quality and Safety in Orthopaedics: Learning and Teaching at the Same Time: AOA Critical Issues

Increasing attention has been placed on providing higher quality and safer patient care. This requires the development of a new set of competencies to better understand and navigate the system and lead the orthopaedic team. While still trying to learn and develop these competencies, the academic orthopaedist is also expected to model and teach them.The orthopaedic surgeon must understand what is being measured and why, both for purposes of providing better care and to eliminate unnecessary expense in the system. Metrics currently include hospital-acquired conditions, "never events," and thirty-day readmission rates. More will undoubtedly follow.Although commitment and excellence at the individual level are essential, the orthopaedist must think at the systems level to provide the highest value of care. A work culture characterized by respect and trust is essential to improved communication, teamwork, and confidential peer review. An increasing number of resources, both in print and electronic format, are available for us to understand what we can do now to improve quality and safety.Resident education in quality and safety is a fundamental component of the systems-based practice competency, the Next Accreditation System, and the Clinical Learning Environment Review. This needs to be longitudinally integrated into the curriculum and applied parallel to the development of resident knowledge and skill, and will be best learned if resident learning is experiential and taught within a genuine culture of quality and safety.

Putting residents in the office: an effective method to teach the systems-based practice competency

OBJECTIVES

Systems-based practice (SBP) was 1 of 6 core competencies established by the Accreditation Council for Graduate Medical Education and has proven to be one of the most difficult to effectively implement. This pilot study presents an immersion workshop as an effective tool to teach the SBP competency in a way that could easily be integrated into a residency curriculum.

DESIGN

In 2006, 16 surgical residents rotated through 3 stations for 30 minutes each: coding and billing, scheduling operations and return appointments, and patient check-in. Participants were administered a pretest and posttest questionnaire evaluating their knowledge of SBP, and were asked to evaluate the workshop.

SETTING

Outpatient clinic at MedStar Georgetown University Hospital, Washington, DC.

PARTICIPANTS

Residents in the general surgery residency training program at MedStar Georgetown University Hospital.

RESULTS

Most residents (62.5%) improved their score after the workshop, whereas 31.25% showed no change and 6.25% demonstrated a decrease in score. Overall within their training levels, all groups demonstrated an increase in mean test score. Postgraduate year-2 residents demonstrated the greatest change in mean score (20%), whereas postgraduate year-4 residents demonstrated the smallest change in mean score (3.3%).

CONCLUSIONS

An immersion workshop where general surgery residents gained direct exposure to SBP concepts in situ was an effective and practical method of integrating this core competency into the residency curriculum. Such a workshop could complement more formal didactic teaching and be easily incorporated into the curriculum. For example, this workshop could be integrated into the ambulatory care requirement that each resident must fulfill as part of their clinical training.

The unmet need for a national surgical quality improvement curriculum: a systematic review

INTRODUCTION

The Accreditation Council for Graduate Medical Education Next Accreditation System will require general surgery training programs to demonstrate outstanding clinical outcomes and education in quality improvement (QI). The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative reports the results of a systematic review of the literature investigating the availability of a QI curriculum.

METHODS

Using defined search terms, a systematic review was conducted in Embase, PubMed, and Google Scholar (January 2000-March 2013) to identify a surgical QI curriculum. Bibliographies from selected articles and other relevant materials were also hand searched. Curriculum was defined as an organized program of learning complete with content, instruction, and assessment for use in general surgical residency programs. Two independent observers graded surgical articles on quality of curriculum presented.

RESULTS

Overall, 50 of 1155 references had information regarding QI in graduate medical education. Most (n = 24, 48%) described QI education efforts in nonsurgical fields. A total of 31 curricular blueprints were identified; 6 (19.4%) were specific to surgery. Targeted learners were most often post graduate year-2 residents (29.0%); only 6 curricula (19.4%) outlined a course for all residents within their respective programs. Plan, Do, Study, Act (n = 10, 32.3%), and Root Cause Analysis (n = 5, 16.1%) were the most common QI content presented, the majority of instruction was via lecture/didactics (n = 26, 83.9%), and only 7 (22.6%) curricula used validated tool kits for assessment.

CONCLUSION

Elements of QI curriculum for surgical education exist; however, comprehensive content is lacking. The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative will build on the high-quality components identified in our review and develop data-centered QI content to generate a comprehensive national QI curriculum for use in graduate surgical education.

Practice quality improvement during residency: where do we stand and where can we improve?

RATIONALE AND OBJECTIVES

Completing a systems-based practice project, equivalent to a practice quality improvement project (PQI), is a residency requirement by the Accreditation Council for Graduate Medical Education and an American Board of Radiology milestone. The aim of this study was to assess the residents' perspectives on quality improvement projects in radiology.

MATERIALS AND METHODS

Survey data were collected from 154 trainee members of the Association of University Radiologists to evaluate the residents' views on PQI.

RESULTS

Most residents were aware of the requirement of completing a PQI project and had faculty mentors for their projects. Residents who thought it was difficult to find a mentor were more likely to start their project later in residency (P < .0001). Publication rates were low overall, and lack of time was considered the greatest obstacle. Having dedicated time for a PQI project was associated with increased likelihood of publishing or presenting the data (P = .0091). Residents who rated the five surveyed PQI steps (coming up with an idea, finding a mentor, designing a project, finding resources, and finding time) as difficult steps were more likely to not have initiated a PQI project (P < .0001 for the first four and P = .0046 for time).

CONCLUSION

We present five practical areas of improvement to make PQI a valuable learning experience: 1) Increasing awareness of PQI and providing ideas for projects, 2) encouraging faculty mentorship and publication, 3) educating residents about project design and implementation, 4) providing resources such as books and funds, and 5) allowing dedicated time.

Development and initial validation of a project-based rubric to assess the systems-based practice competency of residents in the clinical chemistry rotation of a pathology residency

CONTEXT

Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature.

OBJECTIVE

To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry.

DESIGN

Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects.

RESULTS

Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability.

CONCLUSIONS

We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Improvement in reporting skills of radiology residents with a structured reporting curriculum

RATIONALE AND OBJECTIVES

Radiology residents must acquire dictation and reporting skills to meet Accreditation Council for Graduate Medical Examination requirements and provide optimal patient care. Historically, these skills have been taught informally and vary between institutions and among radiologists. A structured curriculum improves resident report quality when using a quantitative grading scheme. This study describes the implementation of such a curriculum and evaluates its utility in tracking resident progress.

MATERIALS AND METHODS

We implemented a three-stage reporting curriculum in our diagnostic radiology residency program in 2009. Stages 1 and 2 involve instruction and formative feedback composed of suggestions for improvement in a 360° format from faculty, peers, and others within the resident's sphere of influence. The third stage involves individual, biannual, written feedback with scored reports specifically assessing four categories: succinctness, spelling/grammar, clarity, and responsible referral. Biannual scores were collected from 2009 to 2013, sorted by year of residency training (R1 to R4), and average training level scores were statistically compared.

RESULTS

Review of 1500 reports over a 4-year period yielded a total of 153 scores: 54, 36, 29, and 34 from R1, R2, R3, and R4 residents, respectively. The mean (standard deviation) scores for R1, R2, R3, and R4 residents were 10.20 (1.06), 10.25 (0.81), 10.5 (0.74), and 10.75 (0.69), respectively. Post hoc analysis identified significant differences between R1 and R4 residents (P = .012) and R2 and R4 residents (P = .009).

CONCLUSIONS

Residents' reporting scores showed significant improvement over the course of their residency training. This indicates that there may be a benefit in using an organized reporting curriculum to track resident progress in producing reports that may improve patient care.

Bedside Teaching on Time to Disposition Improves Length of Stay for Critically-ill Emergency Departments Patients

INTRODUCTION

We tested the effect of a brief disposition process intervention on residents' time to disposition and emergency department (ED) length of stay (LOS) in high acuity ED patients.

METHODS

This was a quasi-experimental study design in a single teaching hospital where ED residents are responsible for administrative bed requests for patients. Enrollment was performed for intervention and control groups on an even-odd day schedule. Inclusion criteria were ED patients triaged as Emergency Severity Index (ESI) 1 and 2. In the intervention group, the attending physician prompted the resident to make the disposition immediately after the evaluation of resuscitation patients. In the control group, the attending physicians did not intervene in the disposition process unless more than 2 hours passed without a disposition. Main outcomes were time to disposition and total ED LOS.

RESULTS

A total of 104 patients were enrolled; 53 (51%) in the intervention group and 51 (49%) in the control group. After controlling for ESI and resident training year, mean disposition time was significantly shorter in the intervention group by 41.4 minutes (95% CI: 32.6-50.1). LOS was also shorter in the intervention group by 93.3 minutes (95% CI: 41.9-144.6).

CONCLUSION

Prompting residents to enter administrative disposition orders in high acuity patients is associated with significant reduction in both time to disposition and ED LOS.

Teaching medical management and operations engineering for systems-based practice to radiology residents

RATIONALE AND OBJECTIVES

To better prepare radiology residents for providing care within the context of the larger health care system, this study evaluated the feasibility and impact of a curriculum to enhance radiology residents' understanding and ability to apply concepts from medical management and industrial and operational engineering to systems-based practice problems in radiology practice.

MATERIALS AND METHODS

A multiprofessional team including radiology, medical education, and industrial and operational engineering professionals collaborated in developing a seven-module curriculum, including didactic lectures, interactive large-group analysis, and small-group discussions with case-based radiology examples, which illustrated real-life management issues and the roles physicians held. Residents and faculty participated in topic selection. Pre- and post-instruction formative assessments were administered, and results were shared with residents during teaching sessions.

RESULTS

Attendance and participation in case-based scenario resolutions indicate the feasibility and impact of the interactive curriculum on residents' interest and ability to apply curricular concepts to systems-based practice in radiology. Paired t test analyses (P < .05) and effect sizes showed residents significantly increased their knowledge and ability to apply concepts to systems-based practice issues in radiology.

CONCLUSIONS

Our iterative curriculum development and implementation process demonstrated need and support for a multiprofessional team approach to teach management and operational engineering concepts. Curriculum topics are congruent with Accreditation Council for Graduate Medical Education requirements for systems-based practice. The case-based curriculum using a mixed educational format of didactic lectures and small-group discussion and problem analysis could be adopted for other radiology programs, for both residents and continuing medical education applications.

Reshaping orthopaedic resident education in systems-based practice

BACKGROUND

Despite advances in understanding the "systems-based practice" competency in resident education, this topic has remained difficult to teach, assess, and document. The goal of this study was to perform a needs assessment and an analysis of the current state of systems-based practice education in orthopaedic residency programs across the U.S. and within our own institution.

METHODS

A sample of orthopaedic educators and residents from across the U.S. who were attending the 2010 American Orthopaedic Association (AOA) Effective Orthopaedic Educator Course, AOA Resident Leadership Forum, and AOA Council of Residency Directors meeting were surveyed to determine (1) which aspects of systems-based practice, if any, were being taught; (2) how systems-based practice is being taught; and (3) how residency programs are assessing systems-based practice. In addition, an in-depth case study of these issues was performed by means of seven semi-structured focus group sessions with diverse stakeholders who participated in the care of musculoskeletal patients at the authors' institution. A quantitative approach was used to analyze the survey data. The focus group data were analyzed with procedures associated with grounded theory, relying on a constant comparative method to develop salient themes arising from the discussion.

RESULTS

"Clinical observation" (33%) and "didactic case-based learning" (23%) were reported by the survey respondents as the most commonly used teaching methods, but specific topics were taught inconsistently. Competency assessment was reported to occur infrequently, and 36% of respondents reported that systems-based practice areas were not being assessed by any methods. The focus group discussions emphasized the need for standardized experiential learning that was closely linked to the patient's perspective. Orthopaedic faculty members were uncomfortable with their knowledge of this competency and their ability to teach and assess it.

CONCLUSIONS

Teaching the systems-based practice competency occurs inconsistently, and formal assessment occurs infrequently. In addition to formal teaching, learning systems-based practice will be best achieved experientially and from the patient's perspective.

Business of radiology 101: the state of radiology business practice and health care policy curricula at US radiology residency programs

PURPOSE

The aim of this study was to assess the prevalence, content, and perceptions of curricula focused on radiology business practice and health care policy at US radiology residency training programs.

METHODS

The desired survey population was trainees and faculty members of radiology residency programs in the United States. Three anonymous survey instruments were distributed, including an e-mail survey to the membership of the ACR RFS, a paper survey to ACR RFS delegates attending the 2010 AMCLC, and an e-mail survey to the membership of the Association of Program Directors in Radiology (APDR).

RESULTS

Response rates for the surveys were 12%, 25%, and 21%, respectively. Members of the APDR and RFS agreed that understanding and competency in business practice and health care policy topics are important to the future careers of residents (total favorable sentiment >86% for APDR members and >96% for RFS members). Most survey respondents' home institutions offer some form of a noninterpretive curriculum (91% of APDR respondents, 74% of RFS respondents), but the breadth of topics addressed and educational time devoted to these curricula were quite variable. Subjective effectiveness of curricula was infrequently rated as very effective by 12% of APDR respondents and 6% of RFS respondents.

CONCLUSIONS

Despite the perceived importance of radiology business practice and health care policy education, and residency training requirements in competencies related to these subjects that have been in place for more than a decade, curricula addressing these items still seem to be in a stage of acceptance and development. Further commitment to and innovation within these curricula are requisite in educating our future radiologists.

Teaching systems-based competency in anesthesiology residency: development of an education and assessment tool

BACKGROUND

The Accreditation Council for Graduate Medical Education requires programs to educate and evaluate residents in 6 competencies, including systems-based practice. We designed a survey and assessment tool to address the competency as it pertains to anesthetic drug costs in an academic center.

METHODS

Residents, certified registered nurse anesthetists, and faculty were asked to complete an anesthetic drug-cost survey without relying on reference materials. After a combination of compulsory in-class didactic sessions and web-based education, the participants were asked to design an anesthetic, give example cases, and determine costs. The initial task was repeated 1 year later.

RESULTS

Our preintervention survey revealed that most practitioners knew very little about anesthetic drug costs, regardless of level of training or degree. All residents completed the mandatory online education tool, more than 80% attended the departmental grand rounds program, and 100% met the goal of designing an anesthetic for all cases within the preset price limit. A repeat of the cost estimate produced an improvement in cost estimates with reduction in variability (P < .05, Student unpaired t test), although estimates of volatile anesthetic and reversal agent costs did not achieve significance at the .05 level for any of the 3 cases.

CONCLUSION

Introducing a formalized teaching and assessment tool has improved our residents' understanding of anesthetic drug costs, and improved our ability to teach and assess the systems-based practice competency.