How Theory Can Inform Our Understanding of Experiential Learning in Quality Improvement Education

Problem-Based Learning in Medical Education: A Global Research Landscape of the Last Ten Years (2013-2022)

Background

Problem-based learning (PBL) constructs a curriculum that merges theory and practice by employing clinical scenarios or real-world problems. Originally designed for the pre-clinical phase of undergraduate medicine, PBL has since been integrated into diverse aspects of medical education. Therefore, this study aims to map the global scientific landscape related to PBL in medical education in the last ten years.

Methods

We combined bibliometrics and network analysis to analyze the metadata of related research articles published between 2013 and 2022 and indexed in the Web of Science Core Collection.

Results

Our results show an annual publication rate of 9.42%. The two main journals disseminating research on this subject are BMC Medical Education and Medical Teacher. Education & Educational Research and Health Care Sciences & Services are the two most frequent research areas, and also the two most central nodes of the related network. The USA and China are the most publishing countries, while the Netherlands and Canada are the most collaborative. The Maastricht University holds the position of most publishing and collaborative research organization. The University of California ranks second in publications, while the University of Toronto is the second most central research organization.

Conclusions

Our study provides an overview of the last ten years of publications related to PBL and medical education, and we hope it can be of interest to educators, researchers, and students involved with this subject.

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-024-02003-1.

Effects of an Experiential Learning Curriculum on Systems-Thinking

Introduction

Though health systems science (HSS) is referred to as the third pillar of medical education along with the pillars of basic science and clinical care, the effects of learning theories used to teach components of HSS including quality improvement/ patient safety (QI/PS), are poorly understood. Experiential learning theory is often referenced in QI/ PS education but its effects on QI/PS education are not well-described.

Objective

To examine the effects of teaching QI using experiential learning theory on resident systems-thinking.

Methods

Data was gathered from 30 resident participants in a 3-h QI workshop designed using experiential learning theory. Using a mixed-methods design, aspects of learner systems-thinking were analyzed both before and after the workshop. Learners were asked about their confidence in systems-thinking behaviors, and their QI plans were evaluated qualitatively for themes as well as quantitatively via the QIKAT-R.

Results

There was a significant increase in self-reported confidence in systems-thinking behaviors post-workshop. Odds ratio of the QI aim statement focusing on the systems-level of the problem after the workshop was 41.4 with a 95% CI of 8.9142 to 192.2721, p-value .0001. Thematic analysis of QI plans revealed a change in residents' thinking about healthcare problems. They shifted from attributing problems to individual actors to thinking about the problem as lying in the interaction between systems.

Conclusion

Experiential learning theory can be an effective framework for QI to transform learners into systems-thinkers.

A Qualitative Study to Understand the Cultural Factors That Influence Clinical Data Use for Continuing Professional Development

INTRODUCTION

The use of data to inform lifelong learning has become increasingly important in continuing professional development (CPD) practice. Despite the potential benefits of data-driven learning, our understanding of how physicians engage in data-informed learning activities, particularly for CPD, remains unclear and warrants further study. The purpose of this study was to explore how physicians perceive cultural factors (individual, organizational, and systemic) that influence the use of clinical data to inform lifelong learning and self-initiated CPD activities.

METHODS

This qualitative study is part of an explanatory sequential mixed-methods study examining data-informed learning. Participants were psychiatrists and general surgeons from Canada and the United States. Recruitment occurred between April 2019 and November 2019, and the authors conducted semistructured telephone interviews between May 2019 and November 2019. The authors performed thematic analysis using an iterative, inductive method of constant comparative analysis.

RESULTS

The authors interviewed 28 physicians: 17 psychiatrists (61%) and 11 general surgeons (39%). Three major themes emerged from the continuous, iterative analysis of interview transcripts: (1) a strong relationship between data and trust, (2) a team-based approach to data-informed learning for practice improvement, and (3) a need for organizational support and advocacy to put data into practice.

CONCLUSION

Building trust, taking a team-based approach, and engaging multiple stakeholders, such as data specialists and organizational leadership, may significantly improve the use of data-informed learning. The results are situated in the existing literature, and opportunities for future research are summarized.

Bridging medical education goals and health system outcomes: An instrumental case study of pre-clerkship students' improvement projects

INTRODUCTION

Many medical schools engage students in health system improvement (HSI) efforts. Evaluation of these efforts often focuses on students' learning outcomes and rarely considers the impact on health systems, despite the significant commitment health systems make to these efforts. Our study identified and evaluated system-level outcomes of pre-clerkship medical students' engagement in HSI efforts.

METHODS

We used an instrumental case study approach to examine the effects of pre-clerkship medical students' engagement in HSI projects as part of a 15-month experiential curriculum. We extracted data from 53 project summaries and posters completed during the 2017-18 academic year and follow-up survey data collected in May 2019 from physician coaches and health system professionals who mentored students, contributed to these projects, and worked in the clinical microsystems where the projects occurred.

RESULTS

We identified three categories and ten indicators of health system outcomes relevant to medical student engagement in HSI. Using these indicators, our evaluation found multiple benefits to the microsystems in which projects occurred. These included achievement of project aims, perceived immediate and sustained project impact on the health system, and development and implementation of projects with aims that aligned with national and health system priorities.

CONCLUSION

Evaluation of HSI curricula needs to include effects on health systems so that program design can optimize the experience for all involved. Our study offers a framework others can use to evaluate system-level effects of project-based HSI curricula and shows several ways in which students' engagement can add value to health systems.

A Tale of Four Programs: How Residents Learn About Quality Improvement during Postgraduate Medical Education at the University of Calgary

Phenomenon: Quality Improvement (QI) has become increasingly incorporated into competency frameworks for physician learners over the past two decades. As contemporary medical education adapts competency-based models of training, learners will be required to demonstrate competency in QI. There is a need to explore how various residency programs are currently teaching QI, including how residents might participate in experiential QI activities, and the various outcomes of these strategies. Approach: A collective case study examined how residents in four postgraduate programs at a single academic institution have learned about QI since the formal integration of QI as a cross-cutting competency. Data from surveys, interviews, observations, and archival records were collected in order to develop a comprehensive understanding of each case in its real-life context and explore current and historical trends and patterns within and across the four programs. Findings: Teaching and resident involvement in QI projects increased across all four programs since its formal integration into the national physician competency framework. Two programs had a longitudinal, hybrid QI curriculum involving didactic and experiential components. Two programs had didactic-alone QI curricula, with minimal resident engagement in applied QI activities. Between-program differences were quantified with regards to learning climate, safety climate, QI knowledge, skills, and attitudes, attitudes toward research during residency, and quality of mentorship for scholarly activities. Residents in programs with experiential learning reported higher knowledge, skills, and attitudes toward QI were motivated to lead improvement efforts in their future practice. Residents in programs with didactic-only QI teaching perceived that the historical operationalization of the scholarly project as research was a barrier to their involvement in QI, as it was not valued or legitimized in their academic and clinical contexts. Common barriers and facilitators to engagement with QI across all programs included time, mentorship, motivation, and competing demands such as feeling pressure to conduct research in order to obtain competitive fellowship positions or employment. Common across all programs was the perception of the residency scholarly project requirement as a "checkbox." Associations were quantified between the constructs of learning climate with safety culture, and safety culture with QI knowledge, skills, and attitudes. Insights: While hybrid QI curricula with experiential learning remains an effective curriculum strategy, tensions between research and QI may be a critical barrier to learner engagement in experiential activities. In addition to providing learners with support, time, mentorship, and explicitly communicating the value QI by the program, the local safety culture may impact QI learning beyond the core curriculum. Reconceptualization of the scholarly project requirements to normalize QI activities and recognizing the potential influence of the local organizational culture on QI learning and how trainees can positively or negatively shape these cultures warrants consideration.

Trends and challenges of medical education

The aim of this article is to reflect on the contextual factors that influence medical education, in order to provide some recommendations about the challenges that physicians face. A literature review was conducted to write this reflection paper, and it was found that the education of health professionals has undergone significant changes that are based on the development of biomedical sciences, technology, and pedagogy, and are influenced by the needs of the general population, the requirements of the health systems of each country, and scientific progress. These changes have given rise to challenges inherent to the special characteristics of each country and related to the quantity, quality, and relevance of the training of physicians. In this regard, it has been suggested that, in the face of such changes, it is necessary to design curricula that integrate all aspects of health care, consider the supply and demand of medical services, emphasize professionalism, take into account the pedagogical training of health sciences teachers, comply with quality standards, assist students in selecting undergraduate programs adequately (in this case Medicine), and ensure relevant clinical practice scenarios. It should also be noted that medical education has not adapted adequately to the aforementioned changes in many countries and, therefore, the development of teaching and learning strategies has lagged behind and physicians in such countries do not have the capacity to provide the health care required by the general population in a proper manner.

Experiential Learning in Project-Based Quality Improvement Education: Questioning Assumptions and Identifying Future Directions

PURPOSE

Project-based experiential learning is a defining element of quality improvement (QI) education despite ongoing challenges and uncertainties. The authors examined stakeholders' perceptions and experiences of QI project-based learning to increase understanding of factors that influence learning and project experiences.

METHOD

The authors used a case study approach to examine QI project-based learning in 3 advanced longitudinal QI programs, 2 at the University of Toronto and 1 at an academic tertiary-care hospital. From March 2016 to June 2017, they undertook 135 hours of education program observation and 58 interviews with learners, program directors, project coaches, and institutional leaders and reviewed relevant documents. They analyzed data using a conventional and directed data analysis approach.

RESULTS

The findings provide insight into 5 key factors that influenced participants' project-based learning experiences and outcomes: (1) variable emphasis on learning versus project objectives and resulting benefits, tensions, and consequences; (2) challenges integrating the QI project into the curriculum timeline; (3) project coaching factors (e.g., ability, capacity, role clarity); (4) participants' differing access to resources and ability to direct a QI project given their professional roles; and (5) workplace environment influence on project success.

CONCLUSIONS

The findings contribute to an empirical basis toward more effective experiential learning in QI by identifying factors to target and optimize. Expanding conceptualizations of project-based learning for QI education beyond learner-initiated, time-bound projects, which are at the core of many QI educational initiatives, may be necessary to improve learning and project outcomes.

Development of a tool for identifying and addressing prioritised determinants of quality improvement initiatives led by healthcare professionals: a mixed-methods study

Background

Several frameworks have been developed to identify essential determinants for healthcare improvement. These frameworks aim to be comprehensive, leading to the creation of long lists of determinants that are not prioritised based on being experienced as most important. Furthermore, most existing frameworks do not describe the methods or actions used to identify and address the determinants, limiting their practical value. The aim of this study is to describe the development of a tool with prioritised facilitators and barriers supplemented with methods to identify and address each determinant. The tool can be used by those performing quality improvement initiatives in healthcare practice.

Methods

A mixed-methods study design was used to develop the tool. First, an online survey was used to ask healthcare professionals about the determinants they experienced as most facilitating and most hindering during the performance of their quality improvement initiative. A priority score was calculated for every named determinant, and those with a priority score ≥ 20 were incorporated into the tool. Semi-structured interviews with implementation experts were performed to gain insight on how to analyse and address the determinants in our tool.

Results

The 25 healthcare professionals in this study experienced 64 facilitators and 66 barriers when performing their improvement initiatives. Of these, 12 facilitators and nine barriers were incorporated into the tool. Sufficient support from management of the department was identified as the most important facilitator, while having limited time to perform the initiative was considered the most important barrier. The interviews with 16 experts in implementation science led to various inputs for identifying and addressing each determinant. Important themes included maintaining adequate communication with stakeholders, keeping the initiative at a manageable size, learning by doing and being able to influence determinants.

Conclusions

This paper describes the development of a tool with prioritised determinants for performing quality improvement initiatives with suggestions for analysing and addressing these determinants. The tool is developed for those engaged in quality improvement initiatives in practice, so in this way it helps in bridging the research to practice gap of determinants frameworks. More research is needed to validate and develop the tool further.

A realist synthesis of quality improvement curricula in undergraduate and postgraduate medical education: what works, for whom, and in what contexts?

BACKGROUND

With the integration of quality improvement (QI) into competency-based models of physician training, there is an increasing requirement for medical students and residents to demonstrate competence in QI. There may be factors that commonly facilitate or inhibit the desired outcomes of QI curricula in undergraduate and postgraduate medical education. The purpose of this review was to synthesise attributes of QI curricula in undergraduate and postgraduate medical education associated with curricular outcomes.

METHODS

A realist synthesis of peer-reviewed and grey literature was conducted to identify the common contexts, mechanisms, and outcomes of QI curricula in undergraduate and postgraduate medical education in order to develop a programme theory to articulate what works, for whom, and in what contexts.

RESULTS

18854 records underwent title and abstract screening, full texts of 609 records were appraised for eligibility, data were extracted from 358 studies, and 218 studies were included in the development and refinement of the final programme theory. Contexts included curricular strategies, levels of training, clinical settings, and organisational culture. Mechanisms were identified within the overall QI curricula itself (eg, clear expectations and deliverables, and protected time), in the didactic components (ie, content delivery strategies), and within the experiential components (eg, topic selection strategies, working with others, and mentorship). Mechanisms were often associated with certain contexts to promote educational and clinical outcomes.

CONCLUSION

This research describes the various pedagogical strategies for teaching QI to medical learners and highlights the contexts and mechanisms that could potentially account for differences in educational and clinical outcomes of QI curricula. Educators may benefit from considering these contexts and mechanisms in the design and implementation of QI curricula to optimise the outcomes of training in this competency area.

Assessing the current state of quality improvement training in urology in the UK: Findings from the General Medical Council 2018 trainee survey

Objective:

The General Medical Council (GMC) of the UK has identified the need to support doctors through education in safety and quality improvement (QI) methods. This study reports findings from the GMC annual survey of 2018 from urology trainees regarding the state of QI training and their training needs.

Material and methods:

We designed a set of four questions to assess how QI methods are being taught nationally, and inserted them in the 2018 annual GMC trainee survey for urology. This is a cross-sectional study assessing the current state of QI training and mentoring received by trainees, and their self-assessed ability and confidence in completing a QI project as part of training requirements. Data were statistically analysed in Stata 15 stratified by Local Education Training Boards (LETBs)/Deanery and by specialty trainee level (ST3–7).

Results:

In total, 270 responses were received from urology trainees. Data showed significant variation across the country. Responses from ST3–7 trainees ranged from 5–20% on completing more than three QI projects, while 7–58% replied that they had done none. Across all ST grades, 40% of trainees stated they had not undertaken QI, whereas 0–27% reported they had not received any mentoring on QI to date. There was significant variation across training regions too: 11–74% of trainees answered that they have received training in QI methods, and 58–100% responded that they were confident in undertaking QI projects. Across all LETBs, 1–3% responded that they uploaded projects on national websites for dissemination; finally, a range of 0–18% stated they had completed more than three projects.

Conclusion:

This is the first national snapshot of QI training for the entire urology specialty in the UK. The study demonstrates wide variation in QI training and activity undertaken by trainees, and shows a lack of systematic implementation of QI education across training regions.

Level of evidence:

2c

Does a Long-Term Quality and Safety Curriculum for Health Care Professionals Improve Clinical Practice? An Evaluation of Quality Improvement Projects

INTRODUCTION

Little is known about the effects on clinical practice of continuing education quality and safety curricula. The aim of this study is to gain insight into learning outcomes on the fourth level of the Kirkpatrick evaluation model for systematically deployed quality improvement projects performed by health care professionals during a Masters in Healthcare Quality and Safety in the Netherlands.

METHODS

The researchers reviewed 35 projects led by health care professionals in 16 different hospitals to determine their scopes and effects. Afterward, professionals took an online survey to determine the extent of their project's sustainability and spread.

RESULTS

Improving health care safety was the most prevalent quality dimension (n = 11, 31%). A positive change was measured by professionals for 64% (n = 35) of the primary outcomes. Statistical significance was measured in 19 (35%) of the outcomes, of which nine (47%) were found to have a statistically significant effect. A minority of professionals (17%) judged their project as sustained by the department, while some stated that the intervention (37%) or the results of the project (11%) had spread.

DISCUSSION

Although most projects indicated an improvement in their primary outcomes, only a few resulted in statistically significant changes. Teaching professionals in using evaluation methods that take into account the complex context where these projects are performed and teaching them leadership skills is needed to reduce the likelihood of unmeasured outcomes. Analyzing learning experiences of professionals in performing the project is important to see what they learned from performing quality improvement projects, providing experiences that may lead to sustainable effects in future projects.

A National Survey of Integrated Vascular Surgery Residents' Experiences With and Attitudes About Quality Improvement During Residency

BACKGROUND

Integrated vascular surgery residency, or "0+5," programs provide education in the Accreditation Council for Graduate Medical Education (ACGME) competencies of Systems-Based Practice (SBP) and Practice-Based Learning and Improvement (PBLI), which include milestones related to quality improvement (QI). It is unclear what QI curricula are in place in 0+5 programs nationally or how 0+5 residents perceive the importance of QI.

OBJECTIVE

The purpose of this study is to assess current 0+5 residents' knowledge, experiences with, and attitudes about QI.

DESIGN

A survey was developed using the ACGME Common Program Requirements and Milestones pertaining to QI. All 0+5 residents from 2017 to 2018 academic year were emailed an electronic link to the survey. Descriptive statistics and cross-tabulations were calculated using Stata/MP version 13.1.

SETTING

All 0+5 vascular surgery residency programs in the United State (n = 52).

PARTICIPANTS

The survey was completed by 35% (n = 90/257) of 0+5 residents, representing 75% of 0+5 programs in the United States (n = 39/52).

RESULTS

Forty-one percent of respondents felt that applying QI methods is very important and 33% felt that QI education is very important for their future work, however, just 13% felt very prepared to lead a QI initiative. Residents' perceptions of preparedness to lead QI projects and the importance they attached to QI education were significantly influenced by their participation in a QI project (p = 0.003 and p = 0.038 respectively). Finally, just 8% (n = 6) of residents responded correctly to all 13 knowledge-based questions and these residents felt better prepared to lead a QI initiative compared to those who answered incorrectly (p = 0.002).

CONCLUSIONS

Most 0+5 residents report participation in a QI project during residency, however, few feel prepared to lead a QI initiative in practice. Furthermore, only half of PGY5 0+5 residents report achieving specific ACGME targets for graduation pertaining to QI. Current QI curricula in 0+5 programs may be inadequate in teaching fundamental QI concepts and achieving ACGME competency targets for graduation.