Quality improvement in medical education: current state and future directions

Is the delivery of a quality improvement education programme in obstetrics and gynaecology for final year medical students feasible and still effective in a shortened time frame?

BACKGROUND

Teaching clinical audit skills to nascent health professionals is one strategy to improve frontline care. The undergraduate medical curriculum at the University of Auckland provides improvement science theory and skills in Year 5 teaching, and the opportunity to put this into practice during an Obstetrics and Gynaecology (O&G) clinical attachment in Year 6. In 2015, a revised medical school curriculum at the university resulted in a planned reduction of the O&G attachment from five weeks to four, necessitating revision of the Year 6 Quality Improvement (QI) project. The aim of this study was to evaluate if the revised programme provided an important experiential learning opportunity for medical students without imposing an unsustainable burden on clinical services.

METHODS

Based on a CIPP (Context/Input/Process/Product) evaluation model, the study was conducted in several stages to get a sense of the context as the new programme was being planned (Context evaluation), the feasibility of an alternative approach to meet the educational need (Input evaluation), the implementation of the revised programme (Process evaluation) and finally, the programme outcomes (Product evaluation). We used multiple data sources (supervisors, students, academic administrators, and hospital staff) and data collection methods (questionnaires, focus groups, individual interviews, consultative workshops, student reports and oral presentations).

RESULTS

The context evaluation revealed the Year 6 QI programme to be valuable and contributed to O&G service improvements, however, the following concerns were identified: time to complete the project, timely topic selection and access to data, recognition of student achievement, and staff workload. The evaluation of the revised QI project indicated improvement in student perceptions of their QI knowledge and skills, and most areas previously identified as challenging, despite the concurrent reduction in the duration of the O&G attachment.

CONCLUSIONS

Applying the CIPP model for evaluation to our revised QI programme enabled streamlining of procedures to achieve greater efficiency without compromising the quality of the learning experience, or increasing pressure on staff. A four week clinical rotation is adequate for medical educators to consider opportunities for including QI projects as part of student experiential learning.

Engaging Learners in Health System Quality Improvement Efforts

In 1999, an Institute of Medicine report spurred health care organizations to implement systems-based quality improve ment efforts and tackle patient safety. Simultaneously, the Accreditation Council for Graduate Medical Education asked residency programs to address Practice-Based Learning and Systems-Based Practice competencies. Medical educators now advocate incorporation of these competencies in undergraduate medical education.The authors examine the success of these efforts both from the health care delivery and systems perspective as well as from the perspective of educators as they aspire to engage medical students and residents in these domains. The authors argue that the missing element that prevents health care systems from the full realization of the promise of quality improvement is bidirectional alignment. Included are examples from the literature to demonstrate how medical educators are moving toward alignment of learners with health system quality improvement and safety needs. Finally, the authors explore business and information technology governance literature in support of the hypothesis that bidirectional alignment should be the next step in moving from reactive to proactive systems of care.

Validity and Reliability of a Tool to Assess Quality Improvement Knowledge and Skills in Pediatrics Residents

BACKGROUND

Residency programs are expected to educate residents in quality improvement (QI). Effective assessments are needed to ensure residents gain QI knowledge and skills. Limitations of current tools include poor interrater reliability and requirement for scorer training.

OBJECTIVE

To provide evidence for the validity of the Assessment of Quality Improvement Knowledge and Skills (AQIKS), which is a new tool that provides a summative assessment of pediatrics residents' ability to recall QI concepts and apply them to a clinical scenario.

METHODS

We conducted a quasi-experimental study to measure the AQIKS performance in 2 groups of pediatrics residents: postgraduate year (PGY) 2 residents who participated in a 1-year longitudinal QI curriculum, and a concurrent control group of PGY-1 residents who received no formal QI training. The curriculum included 20 hours of didactics and participation in a resident-led QI project. Three faculty members with clinical QI experience, who were not involved in the curriculum and received no additional training, scored the AQIKS.

RESULTS

Complete data were obtained for 30 of 37 residents (81%) in the intervention group, and 36 of 40 residents (90%) in the control group. After completing a QI curriculum, the intervention group's mean score was 40% higher than at baseline (P < .001), while the control group showed no improvement (P = .29). Interrater reliability was substantial (κ = 0.74).

CONCLUSIONS

The AQIKS detects an increase in QI knowledge and skills among pediatrics residents who participated in a QI curriculum, with better interrater reliability than currently available assessment tools.

Strategies for Developing and Recognizing Faculty Working in Quality Improvement and Patient Safety

Academic clinical departments have the opportunity and responsibility to improve the quality and value of care and patient safety by supporting effective quality improvement activities. The pressure to provide high-value care while further developing academic programs has increased the complexity of decision making and change management in academic health systems. Overcoming these challenges will require faculty engagement and leadership; however, most academic departments do not have a sufficient number of individuals with expertise and experience in quality improvement and patient safety (QI/PS). Accordingly, the authors of this article advocate for a targeted and proactive approach to developing faculty working in QI/PS. They propose a strategy predicated on the identification of QI/PS as a strategic priority for academic departments, the creation of enabling resources in QI/PS, and the expansion of rigorous training programs in change management and in improvement and implementation sciences. Professional organizations, health systems, medical schools, and academic departments should recognize successful QI/PS work with awards and promotions. Individual faculty members should expand their collaborative networks, consider the generalizability and scholarly impact of their efforts when designing QI/PS initiatives, and benchmark the outcomes of their performance. Appointments and promotions committees should work proactively with department and QI/PS leaders to ensure that outstanding achievement in QI/PS is defined and recognized. As with the development of physician-investigators and clinician-educators, departments and health systems need a comprehensive approach to support and recognize the contributions of faculty working in QI/PS to meet the considerable needs and opportunities in health care.

Useful resources for members serving on a curriculum committee in schools and colleges of pharmacy

The curriculum committee has an important role in the design and delivery of a Doctor of Pharmacy program. The primary purpose of this article is to identify relevant resources for members to utilize to be active participants in a school or college of pharmacy curriculum committee. The resources presented are focused around the following seven key curricular management concepts: orientation to curriculum, syllabus review, teaching methods, curriculum review, interprofessional education, student workload, and policy development, as these are common agenda items for a committee meeting. Several curricular resources used by other health care disciplines were included to promote collaboration with interprofessional education activities. Awareness of such resources may benefit members to achieve optimal educational outcomes for the program.

Mapping the dark matter of context: a conceptual scoping review

CONTEXT

Like dark matter, the contexts for medical education are largely invisible to those within them, although context can have profound influences on teaching, learning and practice. For something that is so intrinsic to the field of medical education, the concept of context remains troubling to scholars and those running medical education programmes. This paper reports on a critical and conceptual review of the concept of context within the medical education literature and beyond.

METHODS

A review was undertaken drawing on two sources: concepts of context in the medical education literature, and concepts of context across multiple academic disciplines. This body of material was iteratively, discursively and inductively synthesised.

RESULTS

Few of the articles from the medical education literature described or defined context directly, tending instead to focus on describing specific elements of context, such as clinical disciplines, physical settings and political pressures, that could or did influence learning outcomes. The results were framed in terms of what context 'is', how context works (in terms of context-mechanism-outcome), and how context can be represented using patterns. The authors propose a definition of context in medical education, along with the means to model, contrast and compare different contexts based on recurring patterns.

CONCLUSIONS

Context matters in medical education and it can, despite many challenges, be considered systematically and objectively. The findings from this study both represent a catalyst and challenge medical education researchers to look at context afresh.

Programmatic Assessment of a Comprehensive Quality Improvement Curriculum in an Otolaryngology Residency

To date, there is minimal literature discussing quality improvement curricula in otolaryngology. Herein, we present our program, including didactic, web-based, and experiential learning, developed in the setting of a preexisting institutional quality and safety policy. Nine otolaryngology residents were evaluated with assessments focused on learner satisfaction, learner attitudes, and knowledge acquisition according to the Kirkpatrick framework. Wilcoxon signed-rank test was used to compare results. While the total score increased across all assessments, it was significant for only the Quality Improvement Knowledge Application Tool Revised ( P < .05). We find our initial learning outcomes encouraging and hope that our comprehensive curriculum can serve as a resource to other programs, which can be adapted to fit within the context of variable training environments. Furthermore, it is imperative to consider continuous assessment and refinement of any educational program, using the same quality improvement principles that we endeavor to teach.

"Making a difference" - Medical students' opportunities for transformational change in health care and learning through quality improvement projects

BACKGROUND

Quality improvement is increasingly becoming an essential aspect of the medical curriculum, with the intention of improving the health care system to provide better health care. The aim of this study was to explore undergraduate medical students' experiences of their involvement in quality improvement projects during a district health rotation.

METHODS

Student group reports from rotations in learning centres of the University of Pretoria in Mpumalanga Province, South Africa were analysed for the period 2012 to 2015. Interviews were conducted with health care providers at four learning centres in 2013.

RESULTS

Three main themes were identified: (1) 'Situated learning', describing students' exposure to the discrepancies between ideal and reality in a real-life situation and how they learned to deal with complex situations, individually and as student group; (2) 'Facing dilemmas', describing how students were challenged about the non-ideal reality; (3) 'Making a difference', describing the impact of the students' projects, with greater understanding of themselves and others through working in teams but also making a change in the health care system.

CONCLUSION

Quality improvement projects can provide an opportunity for both the transformation of health care and for transformative learning, with individual and 'collective' self-authorship.

Osteoporosis knowledge assessment and osteoporosis education recommendations in the health professions

A previous systematic review on osteoporosis knowledge published showed that only several studies investigated osteoporosis knowledge in health professionals, and it found that their knowledge was not as adequate and sufficient as it should be. Since then, studies published on osteoporosis knowledge among health professionals have also assessed and found that they still do not have adequate and sufficient osteoporosis knowledge. To increase and improve osteoporosis knowledge among health professionals, recommendations in osteoporosis education in the health professions, including the application of the cognitive load theory, online learning, problem-based learning, practical learning, simulation-based learning, interactive learning, and feedback are covered in order to ensure health professionals can have adequate and sufficient osteoporosis knowledge to best prevent and treat individuals with the disease.

ABCs of Safety and Quality for the Pediatric Resident and Fellow

The role of resident and fellow trainees in patient-centered improvement processes is critical to a health care system's success. There is a growing impetus to incorporate patient safety and quality improvement into the educational framework of physicians in training. As part of the Next Accreditation System, practice-based learning and improvement and systems-based practice domains mandate that residents and fellows be assessed on their ability to enhance the quality of care and advocate for patient safety. Best practices for incorporating quality improvement and patient safety into the curriculum of residents and fellows remains an area of interest for educators.

Patient Safety Curriculum for Anatomic Pathology Trainees: Recommendations Based on Institutional Experience

Because of the unique systems and skills involved in patient care by the pathologist, it is challenging to design and implement relevant training in patient safety for pathology trainees. We propose a patient safety curriculum for anatomic pathology (AP) residents based on our institutional experience. The Hospital of the University of the Pennsylvania employs a self-reporting safety database. The occurrences from July 2013 to June 2015 recorded in this system that involved the division of AP were reviewed and cataloged as preanalytic, analytic, and postanalytic. The distribution of these occurrences was then used to create a framework for curriculum development in AP. We identified areas in which trainees are involved in the identification and prevention of common patient safety errors that occur in our AP department. Using these data-proven target areas, and employing current Accreditation Council for Graduate Medical Education recommendations and patient safety literature, a strategy for delivering relevant patient safety training is proposed. Teaching patient safety to pathology trainees is a challenging, yet necessary, component of AP training programs. By analyzing the patient safety errors that occur in the AP department, relevant and actionable training can be developed. This provides quality professional development and improves overall performance as trainees are integrated into laboratory systems.

Pediatrics Residents' Confidence and Performance Following a Longitudinal Quality Improvement Curriculum

BACKGROUND

Quality improvement (QI) training is an integral part of residents' education. Understanding the educational value of a QI curriculum facilitates understanding of its impact.

OBJECTIVE

The purpose of this study was to evaluate the effects of a longitudinal QI curriculum on pediatrics residents' confidence and competence in the acquisition and application of QI knowledge and skills.

METHODS

Three successive cohorts of pediatrics residents (N = 36) participated in a longitudinal curriculum designed to increase resident confidence in QI knowledge and skills. Key components were a succession of progressive experiential projects, QI coaching, and resident team membership culminating in leadership of the project. Residents completed precurricular and postcurricular surveys and demonstrated QI competence by performance on the pediatric QI assessment scenario.

RESULTS

Residents participating in the Center for Advancing Pediatric Excellence QI curriculum showed significant increases in pre-post measures of confidence in QI knowledge and skills. Coaching and team leadership were ranked by resident participants as having the most educational value among curriculum components. A pediatric QI assessment scenario, which correlated with resident-perceived confidence in acquisition of QI skills but not QI knowledge, is a tool available to test pediatrics residents' QI knowledge.

CONCLUSIONS

A 3-year longitudinal, multimodal, experiential QI curriculum increased pediatrics residents' confidence in QI knowledge and skills, was feasible with faculty support, and was well-accepted by residents.

Integrating Quality Improvement and Continuing Professional Development at an Academic Medical Center: A Partnership Between Practice Plan, Hospital, and Medical School

INTRODUCTION

While quality improvement (QI) initiatives can be a highly effective means for improving health care delivery in academic medical centers (AMCs), many health care professionals are not formally trained in basic QI methodology, engaging clinicians in QI activities can be challenging, and there is often a lack of integration and coordination among QI functions (eg, Departments of Quality and Safety, Continuing Professional Development). In our AMC, we undertook a collaborative approach to achieve better vertical and horizontal integration of our QI education efforts. This article provides a case example describing our organizational context, what was done, and with what effect and makes our example and lessons learned available to others.

METHODS

We developed a new educational QI program that was jointly planned and implemented by a group comprising major QI stakeholders. This project was intended to create horizontal organizational linkages between continuing professional development, clinicians, the hospital, and QI department and produce QI activities that aligned with the strategic objectives of senior management.

RESULTS

The group developed and implemented a curriculum based on Lean methodology and concepts from the Institute for Health Care Improvement Model for Improvement. Two cohorts (27 teams) completed the training and planned and implemented QI projects. All projects were aligned with organizational quality, safety, and patient experience goals. The majority of projects met their aim statements.

DISCUSSION

This case description provides an example of successful horizontal integration of an AMCs' QI functions to disseminate knowledge and implement meaningful QI aligned with strategic objectives (vertical integration).

Setting the standard: Medical Education's first 50 years

CONTEXT

By understanding its history, the medical education community gains insight into why it thinks and acts as it does. This piece provides a Foucauldian archaeological critical discourse analysis (CDA) of the journal Medical Education on the publication of its 50th Volume. This analysis draws upon critical social science perspectives to allow the examination of unstated assumptions that underpin and shape educational tools and practices.

METHODS

A Foucauldian form of CDA was utilised to examine the journal over its first half-century. This approach emphasises the importance of language, and the ways in which words used affect and are affected by educational practices and priorities. An iterative methodology was used to organise the very large dataset (12,000 articles). A distilled dataset, within which particular focus was placed on the editorial pieces in the journal, was analysed.

RESULTS

A major finding was the diversity of the journal as a site that has permitted multiple - and sometimes contradictory - discursive trends to emerge. One particularly dominant discursive tension across the time span of the journal is that between a persistent drive for standardisation and a continued questioning of the desirability of standardisation. This tension was traced across three prominent areas of focus in the journal: objectivity and the nature of medical education knowledge; universality and local contexts, and the place of medical education between academia and the community.

CONCLUSIONS

The journal has provided the medical education community with a place in which to both discuss practical pedagogical concerns and ponder conceptual and social issues affecting the medical education community. This dual nature of the journal brings together educators and researchers; it also gives particular focus to a major and rarely cited tension in medical education between the quest for objective standards and the limitations of standard measures.

GME Concentrations: A Collaborative Interdisciplinary Approach to Learner-Driven Education

BACKGROUND

Graduate medical education (GME) programs may struggle to provide the knowledge, skills, and experiences necessary to meet trainee career interests and goals beyond a clinical focus. Sponsoring institutions can partner with programs to deliver content not included in typical clinical experiences of GME programs.

OBJECTIVE

To perform a needs assessment and to develop, implement, and measure acceptability and feasibility of an institution-wide GME Concentrations curriculum.

METHODS

In response to the needs assessment, GME leadership developed 4 concentrations: (1) Resident-as-Teacher; (2) Patient Safety and Quality Improvement; (3) Law, Ethics, and Health Policy; and (4) Leaders in Medicine. We formed advisory committees that developed curricula for each concentration, including didactics, experiential learning, and individual project mentoring. Participants completed pre- and postassessments. We assessed feasibility and evaluated participant presentations and final projects.

RESULTS

Over the course of 3 years, 91 trainees (of approximately 1000 trainees each year) from 36 GME programs (of 82 accredited programs) have participated in the program. The number of participants has increased each year, and 22 participants have completed the program overall. Cost for each participant is estimated at $500. Participant projects addressed a variety of education and health care areas, including curriculum development, quality improvement, and national needs assessments. Participants reported that their GME Concentrations experience enhanced their training and fostered career interests.

CONCLUSIONS

The GME Concentrations program provides a feasible, institutionally based approach for educating trainees in additional interest areas. Institutional resources are leveraged to provide and customize content important to participants' career interests beyond their specialty.

Do quality improvement collaboratives' educational components match the dominant learning style preferences of the participants?

Background

Quality improvement collaboratives are used to improve healthcare by various organizations. Despite their popularity literature shows mixed results on their effectiveness. A quality improvement collaborative can be seen as a temporary learning organization in which knowledge about improvement themes and methods is exchanged. In this research we studied: Does the learning approach of a quality improvement collaborative match the learning styles preferences of the individual participants and how does that affect the learning process of participants?

Methods

This research used a mixed methods design combining a validated learning style questionnaire with data collected in the tradition of action research methodology to study two Dutch quality improvement collaboratives. The questionnaire is based on the learning style model of Ruijters and Simons, distinguishing five learning style preferences: Acquisition of knowledge, Apperception from others, Discovery of new insights, Exercising in fictitious situations and Participation with others.

Results

The most preferred learning styles of the participants were Discovery and Participation. The learning style Acquisition was moderately preferred and Apperception and Exercising were least preferred. The educational components of the quality improvement collaboratives studied (national conferences, half-day learning sessions, faculty site visits and use of an online tool) were predominantly associated with the learning styles Acquisition and Apperception. We observed a decrease in attendance to the learning activities and non-conformance with the standardized set goals and approaches.

Conclusions

We conclude that the participants’ satisfaction with the offered learning approach changed over time. The lacking match between these learning style preferences and the learning approach in the educational components of the quality improvement collaboratives studied might be the reason why the participants felt they did not gain new insights and therefore ceased their participation in the collaborative. This study provides guidance for future organisers and participants of quality improvement collaboratives about which learning approaches will best suit the participants and enhance improvement work.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0915-z) contains supplementary material, which is available to authorized users.

Medical education and the quality improvement spiral: A case study from Mpumalanga, South Africa

Background: The short timeframe of medical students’ rotations is not always conducive to successful, in-depth quality-improvement projects requiring a more longitudinal approach.

Aim: To describe the process of inducting students into a longitudinal quality-improvement project, using the topic of the Mother- and Baby-Friendly Initiative as a case study; and to explore the possible contribution of a quality-improvement project to the development of student competencies.

Setting: Mpumalanga clinical learning centres, where University of Pretoria medical students did their district health rotations.

Method: Consecutive student groups had to engage with a hospital's compliance with specific steps of the Ten Steps to Successful Breastfeeding that form the standards for the Mother- and Baby-Friendly Initiative. Primary data sources included an on-site PowerPoint group presentation (n = 42), a written group report (n = 42) and notes of individual interviews in an end-of-rotation objectively structured clinical examination station (n = 139).

Results: Activities in each rotation varied according to the needs identified through the application of the quality-improvement cycle in consultation with the local health team. The development of student competencies is described according to the roles of a medical expert in the CanMEDS framework: collaborator, health advocate, scholar, communicator, manager and professional. The exposure to the real-life situation in South African public hospitals had a great influence on many students, who also acted as catalysts for transforming practice.

Conclusion: Service learning and quality-improvement projects can be successfully integrated in one rotation and can contribute to the development of the different roles of a medical expert. More studies could provide insight into the potential of this approach in transforming institutions and student learning.

The outcomes of recent patient safety education interventions for trainee physicians and medical students: a systematic review

OBJECTIVE

To systematically review the latest evidence for patient safety education for physicians in training and medical students, updating, extending and improving on a previous systematic review on this topic.

DESIGN

A systematic review.

DATA SOURCES

Embase, Ovid Medline and PsycINFO databases.

STUDY SELECTION

Studies including an evaluation of patient safety training interventions delivered to trainees/residents and medical students published between January 2009 and May 2014.

DATA EXTRACTION

The review was performed using a structured data capture tool. Thematic analysis also identified factors influencing successful implementation of interventions.

RESULTS

We identified 26 studies reporting patient safety interventions: 11 involving students and 15 involving trainees/residents. Common educational content included a general overview of patient safety, root cause/systems-based analysis, communication and teamwork skills, and quality improvement principles and methodologies. The majority of courses were well received by learners, and improved patient safety knowledge, skills and attitudes. Moreover, some interventions were shown to result in positive behaviours, notably subsequent engagement in quality improvement projects. No studies demonstrated patient benefit. Availability of expert faculty, competing curricular/service demands and institutional culture were important factors affecting implementation.

CONCLUSIONS

There is an increasing trend for developing educational interventions in patient safety delivered to trainees/residents and medical students. However, significant methodological shortcomings remain and additional evidence of impact on patient outcomes is needed. While there is some evidence of enhanced efforts to promote sustainability of such interventions, further work is needed to encourage their wider adoption and spread.

Republished: Key characteristics of successful quality improvement curricula in physician education: a realist review

PURPOSE

Quality improvement (QI) is a common competency that must be taught in all physician training programmes, yet, there is no clear best approach to teach this content in clinical settings. We conducted a realist systematic review of the existing literature in QI curricula within the clinical setting, highlighting examples of trainees learning QI by doing QI.

METHOD

Candidate theories describing successful QI curricula were articulated a priori. We searched MEDLINE (1 January 2000 to 12 March 2013), the Cochrane Library (2013) and Web of Science (15 March 2013) and reviewed references of prior systematic reviews. Inclusion criteria included study design, setting, population, interventions, clinical and educational outcomes. The data abstraction tool included categories for setting, population, intervention, outcomes and qualitative comments. Themes were iteratively developed and synthesised using realist review methodology. A methodological quality tool assessed the biases, confounders, secular trends, reporting and study quality.

RESULTS

Among 39 studies, most were before-after design with resident physicians as the primary population. Twenty-one described clinical interventions and 18 described educational interventions with a mean intervention length of 6.58 (SD=9.16) months. Twenty-eight reported successful clinical improvements; no studies reported clinical outcomes that worsened. Characteristics of successful clinical QI curricula include attention to the interface of educational and clinical systems, careful choice of QI work for the trainees and appropriately trained local faculty.

CONCLUSIONS

This realist review identified success characteristics to guide training programmes, medical schools, faculty, trainees, accrediting organisations and funders to further develop educational and improvement resources in QI educational programmes.

Quality improvement, patient safety, and continuing education: a qualitative study of the current boundaries and opportunities for collaboration between these domains

PURPOSE

Quality improvement/patient safety (QI/PS) and continuing education (CE) efforts have a common aim to improve health care outcomes. Yet, minimal collaboration occurs between them. This lack of integration can be problematic given the finite resources available and the potential value of approaching health care challenges from different perspectives. The authors conducted an exploratory study to understand Canadian leaders' perceptions and experiences with both their own and the other domain, with the aim of increasing their understanding of the boundaries and opportunities for collaborative approaches to improving health care.

METHOD

The authors conducted this study in 2011-2012 using a qualitative interpretivist framework to guide the collection and analysis of data from semistructured interviews. They used criterion-based, maximum variation, and snowball sampling to select 15 leaders from the domains of QI/PS and CE to interview. They transcribed verbatim the interviews and coded the transcripts using a directed content analysis approach.

RESULTS

Participants described the relationship between QI/PS and CE in four ways: (1) the separation of QI/PS and CE as distinct interventions, (2) (re)positioning CE in QI/PS activities, (3) (re)positioning QI/PS in CE activities, and (4) further integrating QI/PS and CE.

CONCLUSIONS

These findings have important implications for how leaders in QI/PS and CE should mindfully and strategically negotiate their relationship to ensure the relevance and effectiveness of their domain's activities.

Developing Enterprise Skills in Undergraduate Medical Students: A Mixed-methods Evaluation

INTRODUCTION

Support for the development of enterprise skills in medical education exists from the perspectives of educators, researchers, and healthcare leaders. However, literature is limited evaluating the understanding of medical students about these skills. This study aimed to determine whether medical students valued gaining enterprise skills within the course and those skills that they identified and recognized contributed to enterprise practice in subsequent training.

METHODOLOGY

Quantitative and qualitative evaluations were undertaken for over three years. Students completed end-of-course evaluations (n = 895) in 2011 and 2012, responding to closed questions utilizing a Likert scale. Subsequent qualitative reflections were collected by interviews one year later with nine students and eight supervisors.

RESULTS

Immediately after course completion, students gave positive feedback, identifying the development of independent learning, creativity, and reflection, as these enterprise skills were most valued. However, in subsequent reflection one year later, they were unable to transfer the acquired knowledge and identify the examples of enterprise around them in their later experiences and had mixed beliefs about its value in medicine.

CONCLUSION

Enterprise skills need to be revisited explicitly throughout the medical curriculum, with authentic real-life examples, to sustain students' understanding about the role of enterprise in medicine.

Postacute rehabilitation quality of care: toward a shared conceptual framework

There is substantial interest in mechanisms for measuring, reporting, and improving the quality of health care, including postacute care (PAC) and rehabilitation. Unfortunately, current activities generally are either too narrow or too poorly specified to reflect PAC rehabilitation quality of care. In part, this is caused by a lack of a shared conceptual understanding of what construes quality of care in PAC rehabilitation. This article presents the PAC-rehab quality framework: an evidence-based conceptual framework articulating elements specifically pertaining to PAC rehabilitation quality of care. The widely recognized Donabedian structure, process, and outcomes (SPO) model furnished the underlying structure for the PAC-rehab quality framework, and the International Classification of Functioning, Disability and Health (ICF) framed the functional outcomes. A comprehensive literature review provided the evidence base to specify elements within the SPO model and ICF-derived framework. A set of macrolevel-outcomes (functional performance, quality of life of patient and caregivers, consumers' experience, place of discharge, health care utilization) were defined for PAC rehabilitation and then related to their (1) immediate and intermediate outcomes, (2) underpinning care processes, (3) supportive team functioning and improvement processes, and (4) underlying care structures. The role of environmental factors and centrality of patients in the framework are explicated as well. Finally, we discuss why outcomes may best measure and reflect the quality of PAC rehabilitation. The PAC-rehab quality framework provides a conceptually sound, evidence-based framework appropriate for quality of care activities across the PAC rehabilitation continuum.

The impact of resident involvement in minimally-invasive urologic oncology procedures

INTRODUCTION

Robotic and laparoscopic surgical training is an integral part of resident education in urology, yet the effect of resident involvement on outcomes of minimally-invasive urologic procedures remains largely unknown. We assess the impact of resident participation on surgical outcomes using a large multi-institutional prospective database.

METHODS

Relying on the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) Participant User Files (2005-2011), we abstracted the 3 most frequently performed minimally-invasive urologic oncology procedures. These included radical prostatectomy, radical nephrectomy and partial nephrectomy. Multivariable logistic regression models were constructed to assess the impact of trainee involvement (PGY 1-2: junior, PGY 3-4: senior, PGY ≥5: chief) versus attending-only on operative time, length-of-stay, 30-day complication, reoperation and readmission rates.

RESULTS

A total of 5459 minimally-invasive radical prostatectomies, 1740 minimally-invasive radical nephrectomies and 786 minimally-invasive partial nephrectomies were performed during the study period, for which data on resident surgeon involvement was available. In multivariable analyses, resident involvement was not associated with increased odds of overall complications, reoperation, or readmission rates for minimally-invasive prostatectomy, radical and partial nephrectomy. However, operative time was prolonged when residents were involved irrespective of the type of procedure. Length-of-stay was decreased with senior resident involvement in minimally-invasive partial nephrectomies (odds ratio [OR] 0.49, p = 0.04) and prostatectomies (OR 0.68, p = 0.01). The major limitations of this study include its retrospective observational design, inability to adjust for the case complexity and surgeon/hospital characteristics, and the lack of information regarding the minimally-invasive approach utilized (whether robotic or laparoscopic).

CONCLUSIONS

Resident involvement is associated with increased operative time in minimally-invasive urologic oncology procedures. However, it does not adversely affect the complication, reoperation or readmission rates, as well as length-of-stay.

Clinical Quality Improvement Curriculum for Faculty in an Academic Medical Center

Interested faculty enrolled in this 6-month-long quality improvement (QI) course to facilitate independent QI project work. The course included monthly 1.5-hour sessions: 20-minute presentations covering key QI concepts, then small group activities to facilitate project work. Faculty were required to identify, construct, and implement an independent QI project. They met individually with mentors twice during the course, with additional guidance offered virtually via phone or e-mail, and completed pretests and posttests of QI knowledge (maximum score = 15) and self-assessed confidence. A statistically significant difference in knowledge (pre-course mean = 7.75, standard deviation [SD] = 3.06; post-course mean = 11.75, SD = 3.28; P = .02) and self-assessed confidence (pre mean = 3.08, SD = 0.65; post mean = 4.5, SD = 0.68; P < .0001) was found. Of 8 faculty, 5 were able to conduct small tests of change; 3 studied the current processes and planned to run tests of change. Positive responses to this course helped obtain buy-in from leadership to develop a leadership program in QI.

Trainee and program director perceptions of quality improvement and patient safety education: preparing for the next accreditation system

OBJECTIVE

To assess the current state of quality improvement and patient safety (QIPS) education at a large teaching hospital.

METHODS

We surveyed 429 trainees (138 residents, 291 clinical fellows) and 38 program directors (PDs; 2 were PDs of >1 program) from 39 Accreditation Council for Graduate Medical Education-accredited training programs.

RESULTS

Twenty-nine PDs (76.3%) and 259 trainees (60.3%) responded. Most trainees (68.8%) reported participation in projects culminating in scholarly products (39.9%) or clinical innovations (44%). Most PDs reported that teaching (88.9%) and project supervision (83.3%) are performed by expert faculty. Nearly half of the PDs (45.8%) and trainees (49.6%) perceived project-based learning to be of equal value to formal curricula. Compared with trainees, a greater proportion of PDs reported needs for funding for projects, teaching faculty to provide mentorship, and faculty development (P < .05).

CONCLUSIONS

Providing additional financial, administrative, and operational support could enhance the value of curricula and projects. Developing expert teaching faculty is paramount.

The Quality Improvement Knowledge Application Tool Revised (QIKAT-R)

PURPOSE

Quality improvement (QI) has been part of medical education for over a decade. Assessment of QI learning remains challenging. The Quality Improvement Knowledge Application Tool (QIKAT), developed a decade ago, is widely used despite its subjective nature and inconsistent reliability. From 2009 to 2012, the authors developed and assessed the validation of a revised QIKAT, the "QIKAT-R."

METHOD

Phase 1: Using an iterative, consensus-building process, a national group of QI educators developed a scoring rubric with defined language and elements. Phase 2: Five scorers pilot tested the QIKAT-R to assess validity and inter- and intrarater reliability using responses to four scenarios, each with three different levels of response quality: "excellent," "fair," and "poor." Phase 3: Eighteen scorers from three countries used the QIKAT-R to assess the same sets of student responses.

RESULTS

Phase 1: The QI educators developed a nine-point scale that uses dichotomous answers (yes/no) for each of three QIKAT-R subsections: Aim, Measure, and Change. Phase 2: The QIKAT-R showed strong discrimination between "poor" and "excellent" responses, and the intra- and interrater reliability were strong. Phase 3: The discriminative validity of the instrument remained strong between excellent and poor responses. The intraclass correlation was 0.66 for the total nine-point scale.

CONCLUSIONS

The QIKAT-R is a user-friendly instrument that maintains the content and construct validity of the original QIKAT but provides greatly improved interrater reliability. The clarity within the key subsections aligns the assessment closely with QI knowledge application for students and residents.

A novel program to improve patient safety by integrating peer review into the emergency medicine residency curriculum

BACKGROUND

Evaluating the quality of care as part of a quality improvement process is required in many clinical environments by accrediting bodies. It produces metrics used to evaluate department and individual provider performance, provides outcomes-based feedback to clinicians, and identifies ways to reduce error.

DISCUSSION

To improve patient safety and train our residents to perform peer review, we expanded our quality assurance program from a narrow, administrative process carried out by a small number of attendings to an educationally focused activity of much greater scope incorporating all residents on a monthly basis. We developed an explicit system by which residents analyze sets of high-risk cases and record their impressions onto structured databases, which are reviewed by faculty. At monthly meetings, results from the month's case reviews are presented, learning points discussed, and corrective actions are proposed.

CONCLUSION

By integrating Clinical Quality Review (CQR) as a core, continuous component of the residency curriculum, we increased the number of cases reviewed more than 10-fold and implemented a variety of clinical process improvements. An anonymous survey conducted after 2 years of resident-led CQR indicated that residents value their exposure to the peer review process and feel it benefits them as clinicians, but also that the program requires a significant investment of time that can be burdensome.

Effects of My Child's Safety Web-Based Program for Caregivers of Children with Cancer in South Korea

Objectives

The purposes of this study were to develop a Web-based education program, My Child's Safety, which includes patient safety education and information on the diagnosis, treatment, and management for caregivers of children with cancer, and to examine the efficacy of the My Child's Safety program in promoting the caregivers' awareness of patient safety.

Methods

A one-group pre- and post-test design was adopted. The participants were the caregivers of children with cancer and were recruited from one pediatric hemato-oncology unit of a tertiary university hospital in a large metropolitan city of South Korea. They were asked to review the Web-based program for patient safety and then complete questionnaires developed to measure the awareness of patient safety among the caregivers.

Results

In the study, the total score of the caregivers' awareness of patient safety had increased significantly after Web-based self-learning patient safety education. Also caregivers' awareness of their right to ask and know about procedures and treatments during hospitalization had increased after the program was used.

Conclusions

The Web-based patient safety education program effectively improved the awareness of patient safety and the awareness of the right to know and ask about procedures and treatments during hospitalization among the caregivers. Family caregivers were less likely to ask healthcare professionals questions related to safety.

Supporting service change in palliative care: a framework approach

BACKGROUND

Palliative care services are increasingly identifying areas for improvement, then trying to create appropriate changes in response. Nurses in particular are often expected to take leading roles in quality improvement (QI) but are not necessarily trained or supported in these processes.

METHODS

A framework approach to change was developed to guide services through a change cycle and delivered via workshops by representatives of three Australian national projects. Participants were predominantly nurses (80%), with the majority (63.7%) over the age of 50.

FINDINGS

The workshops and the framework were positively evaluated, with participants feeling confident in a number of QI-related activities following workshop training.

CONCLUSION

Recognising and addressing problems in clinical practice and service delivery is an important way for nurses to ensure quality care for patients; however, they need support in developing the skills and knowledge that are essential to successful QI activities.

Patient safety training in pediatric emergency medicine: a national survey of program directors

OBJECTIVES

The Accreditation Council for Graduate Medical Education requires training in patient safety and medical errors but does not provide specification for content or methods. Pediatric emergency medicine (EM) fellowship directors were surveyed to characterize current training of pediatric EM fellows in patient safety and to determine the need for additional training.

METHODS

From June 2013 to August 2013, pediatric EM fellowship directors were surveyed via e-mail.

RESULTS

Of the 71 eligible survey respondents, 57 (80.3%) completed surveys. A formal curriculum was present in 24.6% of programs, with a median of 6 hours (range = 1 to 18 hours) dedicated to the curriculum. One program evaluated the efficacy of the curriculum. Nearly 91% of respondents without formal programs identified lack of local faculty expertise or interest as the primary barrier to implementing patient safety curricula. Of programs without formal curricula, 93.6% included at least one component of patient safety training in their fellowship programs. The majority of respondents would implement a standardized patient safety curriculum for pediatric EM if one was available.

CONCLUSIONS

Despite the importance of patient safety training and requirements to train pediatric EM fellows in patient safety and medical errors, there is a lack of formal curriculum and local faculty expertise. The majority of programs have introduced components of patient safety training and desire a standardized curriculum.

The unmet need for a national surgical quality improvement curriculum: a systematic review

INTRODUCTION

The Accreditation Council for Graduate Medical Education Next Accreditation System will require general surgery training programs to demonstrate outstanding clinical outcomes and education in quality improvement (QI). The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative reports the results of a systematic review of the literature investigating the availability of a QI curriculum.

METHODS

Using defined search terms, a systematic review was conducted in Embase, PubMed, and Google Scholar (January 2000-March 2013) to identify a surgical QI curriculum. Bibliographies from selected articles and other relevant materials were also hand searched. Curriculum was defined as an organized program of learning complete with content, instruction, and assessment for use in general surgical residency programs. Two independent observers graded surgical articles on quality of curriculum presented.

RESULTS

Overall, 50 of 1155 references had information regarding QI in graduate medical education. Most (n = 24, 48%) described QI education efforts in nonsurgical fields. A total of 31 curricular blueprints were identified; 6 (19.4%) were specific to surgery. Targeted learners were most often post graduate year-2 residents (29.0%); only 6 curricula (19.4%) outlined a course for all residents within their respective programs. Plan, Do, Study, Act (n = 10, 32.3%), and Root Cause Analysis (n = 5, 16.1%) were the most common QI content presented, the majority of instruction was via lecture/didactics (n = 26, 83.9%), and only 7 (22.6%) curricula used validated tool kits for assessment.

CONCLUSION

Elements of QI curriculum for surgical education exist; however, comprehensive content is lacking. The American College of Surgeons-National Surgical Quality Improvement Project Quality In-Training Initiative will build on the high-quality components identified in our review and develop data-centered QI content to generate a comprehensive national QI curriculum for use in graduate surgical education.

Use of the Quality Improvement (QI) Knowledge Application Tool in Assessing Pediatric Resident QI Education

BACKGROUND

Assessing the effectiveness of quality improvement curricula is important to improving this area of resident education.

OBJECTIVE

To assess the ability of the Quality Improvement Knowledge Application Tool (QIKAT) to differentiate between residents who were provided instruction in QI and those who were not, when scored by individuals not involved in designing the QIKAT, its scoring rubric, or QI curriculum instruction.

METHODS

The QIKAT and a 9-item self-assessment of QI proficiency were administered to an intervention and a control group. The intervention was a longitudinal curriculum consisting of 8 hours of didactic QI training and 6 workshops providing just-in-time training for resident QI projects. Two uninvolved faculty scored the QIKAT.

RESULTS

A total of 33 residents in the intervention group and 27 in the control group completed the baseline and postcurriculum QIKAT and self-assessment. QIKAT mean intervention group scores were significantly higher than mean control group scores postcurriculum (P < .001). Absolute QIKAT differences were small (of 15 points, intervention group improved from a mean score of 12.8 to 13.2). Interrater agreement as measured by kappa test was low (0.09). Baseline self-assessment showed no differences, and after instruction, the intervention group felt more proficient in QI knowledge than controls in 4 of 9 domains tested.

CONCLUSIONS

The QIKAT detected a statistically significant improvement postintervention, but the absolute differences were small. Self-reported gain in QI knowledge and proficiency agreed with the results of the QIKAT. However, QIKAT limitations include poor interrater agreement and a scoring rubric that lacks specificity. Programs considering using QIKAT to assess curricula should understand these limitations.

Integrating patient safety into health professionals' curricula: a qualitative study of medical, nursing and pharmacy faculty perspectives

BACKGROUND

As efforts to integrate patient safety into health professional curricula increase, there is growing recognition that the rate of curricular change is very slow, and there is a shortage of research that addresses critical perspectives of faculty who are on the 'front-lines' of curricular innovation. This study reports on medical, nursing and pharmacy teaching faculty perspectives about factors that influence curricular integration and the preparation of safe practitioners.

METHODS

Qualitative methods were used to collect data from 20 faculty members (n=6 medical from three universities; n=6 pharmacy from two universities; n=8 nursing from four universities) engaged in medical, nursing and pharmacy education. Thematic analysis generated a comprehensive account of faculty perspectives.

RESULTS

Faculty perspectives on key challenges to safe practice vary across the three disciplines, and these different perspectives lead to different priorities for curricular innovation. Additionally, accreditation and regulatory requirements are driving curricular change in medicine and pharmacy. Key challenges exist for health professional students in clinical teaching environments where the culture of patient safety may thwart the preparation of safe practitioners.

CONCLUSIONS

Patient safety curricular innovation depends on the interests of individual faculty members and the leveraging of accreditation and regulatory requirements. Building on existing curricular frameworks, opportunities now need to be created for faculty members to act as champions of curricular change, and patient safety educational opportunities need to be harmonises across all health professional training programmes. Faculty champions and practice setting leaders can collaborate to improve the culture of patient safety in clinical teaching and learning settings.

Developing a quality and safety curriculum for fellows: lessons learned from a neonatology fellowship program

Formal training in health care quality and safety has become an important component of medical education at all levels, and quality and safety are core concepts within the practice-based learning and system-based practice medical education competencies. Residency and fellowship programs are rapidly attempting to incorporate quality and safety curriculum into their training programs but have encountered numerous challenges and barriers. Many program directors have questioned the feasibility and utility of quality and safety education during this stage of training. In 2010, we adopted a quality and safety educational module in our neonatal fellowship program that sought to provide a robust and practical introduction to quality improvement and patient safety through a combination of didactic and experiential activities. Our module has been successfully integrated into the fellowship program's curriculum and has been beneficial to trainees, faculty, and our clinical services, and our experience suggests that fellowship may be particularly well suited to incorporation of quality and safety training. We describe our module and share tools and lessons learned during our experience; we believe these resources will be useful to other fellowship programs seeking to improve the quality and safety education of their trainees.

Documenting quality improvement and patient safety efforts: the quality portfolio. A statement from the academic hospitalist taskforce

Physicians increasingly investigate, work, and teach to improve the quality of care and safety of care delivery. The Society of General Internal Medicine Academic Hospitalist Task Force sought to develop a practical tool, the quality portfolio, to systematically document quality and safety achievements. The quality portfolio was vetted with internal and external stakeholders including national leaders in academic medicine. The portfolio was refined for implementation to include an outlined framework, detailed instructions for use and an example to guide users. The portfolio has eight categories including: (1) a faculty narrative, (2) leadership and administrative activities, (3) project activities, (4) education and curricula, (5) research and scholarship, (6) honors, awards, and recognition, (7) training and certification, and (8) an appendix. The authors offer this comprehensive, yet practical tool as a method to document quality and safety activities. It is relevant for physicians across disciplines and institutions and may be useful as a standalone document or as an adjunct to traditional promotion documents. As the Next Accreditation System is implemented, academic medical centers will require faculty who can teach and implement the systems-based practice requirements. The quality portfolio is a method to document quality improvement and safety activities.

Laparoscopic radical prostatectomy demonstrates less morbidity than open radical prostatectomy: an analysis of the American College of Surgeons-National Surgical Quality Improvement Program database with a focus on surgical trainee involvement

INTRODUCTION

Complication rates of open radical prostatectomies (ORPs) and laparoscopic radical prostatectomies (LRPs) performed by highly experienced surgeons in centers of excellence are well known. Using a standardized, national, risk-adjusted surgical database, we compared 30-day outcomes following ORP and LRP and analyzed how trainee involvement influenced outcomes.

METHODS

The American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) is a risk-adjusted data collection analyzing preoperative risk factors, demographics, and 30-day postoperative outcomes. From 2005 to 2011, we identified 10,669 total prostatectomies. Of these, 2278 were ORP and 8391 were LRP. Data on trainee involvement were available on 63% of cases.

RESULTS

Comparison of all 10,669 prostatectomies showed a decreased incidence of overall morbidity, serious morbidity, surgical site infections, mortality, wound disruption, urinary tract infection, bleeding, and sepsis or septic shock (p<0.05) for LRP compared with ORP. Trainee involvement was associated with a higher incidence of bleeding, overall and serious morbidity (p<0.001). This difference is isolated to postgraduate year (PGY) 6-10 trainees performing ORP (p<0.001). Overall and serious morbidity was equivalent between PGY groups 1-10 versus attending without trainee performing LRP and PGY groups 1-5 versus attending without trainee performing ORP. Operative times were shorter for ORP versus LRP by an average of 38 minutes (p<0.05), and in cases involving trainees, operative times decreased with trainee experience for both procedures. The length of stay was shorter for LRP compared with ORP (3.2 vs. 1.8 days, p<0.001).

CONCLUSIONS

The large sample size, standardized data definitions, and quality control measures of the ACS-NSQIP database allow for in-depth analysis of subtle, but significant differences in outcomes between groups. Trainee involvement in LRP appears safe to patients. However, the increased morbidity in ORP involving trainees may be mitigated by awareness, simulation laboratories, and standardized competency assessment.

A comparison of student performance on discipline-specific versus integrated exams in a medical school course

Curricular reform is a widespread trend among medical schools. Assessing the impact that pedagogical changes have on students is a vital step in review process. This study examined how a shift from discipline-focused instruction and assessment to integrated instruction and assessment affected student performance in a second-year medical school pathology course. We investigated this by comparing pathology exam scores between students exposed to traditional discipline-specific instruction and exams (DSE) versus integrated instruction and exams (IE). Exam content was controlled, and individual questions were evaluated using a modified version of Bloom's taxonomy. Additionally, we compared United States Medical Licensing Examination (USMLE) step 1 scores between DSE and IE groups. Our findings indicate that DSE students performed better than IE students on complete pathology exams. However, when exam content was controlled, exam scores were equivalent between groups. We also discovered that the integrated exams were composed of a significantly greater proportion of questions classified on the higher levels of Bloom's taxonomy and that IE students performed better on these questions overall. USMLE step 1 exam scores were similar between groups. The finding of a significant difference in content complexity between discipline-specific and integrated exams adds to recent literature indicating that there are a number of potential biases related to curricular comparison studies that must be considered. Future investigation involving larger sample sizes and multiple disciplines should be performed to explore this matter further.

Building capacity for quality: a pilot co-learning curriculum in quality improvement for faculty and resident learners

BACKGROUND

Despite a mandate to teach quality improvement (QI) to residents, many training programs lack faculty capacity to deliver a QI curriculum.

OBJECTIVE

We piloted a co-learning curriculum in QI to train residents while simultaneously developing QI teachers. We evaluated the curriculum's acceptability and feasibility and its effect on faculty engagement in doing and teaching QI.

METHODS

The curriculum involved 2 half-day, interactive sessions, a team-based QI project, and end-of-year project presentations. Key curriculum design principles included (1) residents and faculty co-attend all interactive sessions, (2) residents and faculty work together on team-based QI projects, and (3) QI projects align with divisional QI priorities. Using the Kirkpatrick framework for learner outcomes, we focused our program evaluation on Level 1 (satisfaction) and Level 2 (knowledge and skills acquisition) outcomes using year-end curriculum evaluations.

RESULTS

Our study included 14 residents (70%) and 6 faculty members (30%). With respect to satisfaction (Kirkpatrick Level 1 outcome), 93% (13 of 14) of residents and 100% (6 of 6) of faculty participants rated the overall curriculum as "above average" or "outstanding." Regarding faculty knowledge and skills acquisition (Kirkpatrick Level 2 outcomes), faculty self-rated their QI knowledge and interest in QI higher than their intent to incorporate QI into future practice and their comfort in teaching or supervising QI projects. All 5 faculty respondents (100%) rated the co-learning model for faculty development in QI as "above average" or "outstanding."

CONCLUSIONS

Teaching QI to residents and faculty as co-learners is feasible and acceptable and offers a promising model for programs to teach QI to residents while concurrently building faculty capacity.