Using a Logic Model to Design and Evaluate a Quality Improvement Leadership Course

Empowering the future: improving community wellbeing and health literacy through outreach and service-learning

Institutions training future healthcare professionals in healthcare and community engagement play a crucial role beyond traditional classroom settings. Recognizing their potential to support under-represented groups and minorities, institutions increasingly encourage engagement with schools and community organizations. However, work remains to advance meaningful and impactful educational outreach and service-learning programs. This manuscript synthesizes the perspectives of a group of medical school educators to discuss developing sustainable programs to engage youth in Science, Technology, Engineering, Math, and Medicine (STEMM) education with a focus on biomedical science. Through near-peer education and service-learning, healthcare students can impart knowledge, provide mentorship, promote enthusiasm for STEMM fields, and nurture health-related self-efficacy within individuals and communities. Collaborative efforts through student-as-teacher approaches bridge health-related disparities and cultivate healthier, more empowered futures for all. We advocate for community outreach strategies that target future health professionals early in their education and support the scholarship of teaching and learning and program evaluation. Successful long-term programs must ensure that results are systematically assessed, measured, and perpetuated. This perspective aims to highlight the role of service learning and community outreach in increasing individual health literacy and fostering an enduring interest in STEMM careers, thereby empowering the next generation of elementary and secondary school students.

Evaluation of an Educational Scholarship Fellowship Program for Health Professions Educators

Introduction

Historically, the requirement to produce scholarship for advancement has challenged health professions educators heavily engaged in teaching. As biomedical scientists or healthcare practitioners, few are trained in educational scholarship, and related faculty development varies in scope and quality across institutions. Currently, there is a need for faculty development and mentoring programs to support the development of these skills.

Methods

The International Association of Medical Science Educators (IAMSE) established the Medical Educator Fellowship (MEF) Program to foster health professions educational scholarship. MEF addresses the following: curriculum design, teaching methods and strategies, assessment, educational scholarship, and leadership. Participants receive mentorship and faculty development, and complete an educational scholarship project. Using a logic model, we conducted a retrospective program evaluation with data from Program records, database searches, graduate surveys, and focus groups.

Results

Over 14 years, MEF graduated 61 participants with diverse terminal degrees from five continents and six academic program areas. Graduate survey responses indicated enhanced post-Program skills in all focus areas, that the majority would recommend MEF to a colleague, and that mentorship, networking, and professional development were strengths. Focus group outcomes indicated professional growth, increased confidence, and increased sense of community.

Conclusion

MEF addresses health professions educators' need for faculty development and mentorship in educational scholarship. Evaluation outcomes suggest that MEF effectively enhanced perceived skills across focus areas. Similar programs are essential to support faculty who dedicate significant time to teaching. Organizations like IAMSE can demonstrate the value of educational scholarship and positively impact health professions educator careers by supporting such programs.

A Decision-oriented Approach to Evaluating a Leadership Curriculum in Fellowship

Background

Graduate Medical Education (GME) emphasizes the role of the annual program evaluation to identify opportunities, create action plans, and track improvements longitudinally. There is a lack of a systematic approach to the evaluation of educational curricula. Comprehensive curriculum evaluation can inform the educators about specific modifications to achieve high standards, desired outcomes, and the anticipated objectives.

Objective

To evaluate a leadership in quality improvement program in a pulmonary/critical care fellowship training program using the context, input, process, product (CIPP) model. The CIPP model, given its focus on evaluating different aspects of a program, provides concrete and targeted feedback to guide improvement decisions.

Methods

Evaluation questions addressing the four focused areas were created, pilot tested, and revised. The questions were framed toward optimization of alignment (e.g., program activities with stated objectives, program goals with theoretical perspective, program curriculum with trainee needs) and gaining information about the efficacy of the program in achieving the desired outcomes. To enhance the validity of the results, we triangulated the data-gathering approach by administering surveys and conducting interviews and focus groups by random selection from the eligible participants. Qualitative data were transcribed, coded, and categorized into themes aligning with the four aspects of the CIPP model.

Results

We interviewed 9 participants and conducted three focus groups with 20 participants. The surveys provided vital quantitative information that was cross-verified with the qualitative data; 23 of the 25 (92%) participants completed the survey. The results of qualitative thematic analysis were organized in the CIPP format. The context evaluation of the program revealed that the fellows and faculty were unfamiliar with the guiding principles of the course. The input evaluation highlighted the competing interests that hampered the engagement of the fellows during the evening weekly report-outs. The process evaluation revealed clustering of didactic sessions at the start of the course. The product evaluation stressed the difficulty in completing the quality improvement projects in the allotted timeframe.

Conclusion

Conducting a robust evaluation of an educational curriculum provides insights into gaps in the various stages of the program. Time and resources needed for conducting evaluation by using the CIPP model should be considered.

The Evaluation of an Interprofessional QI Program: A Qualitative Study

Background: Quality Improvement (QI) is the key for every healthcare organization. QI programs may help healthcare professionals to develop the needed skills for interprofessional collaboration through interprofessional education. Furthermore, the role of diversity in QI teams is not yet fully understood. This evaluation study aimed to obtain in-depth insights into the expectations and experiences of different stakeholders of a hospital-wide interprofessional QI program. Methods: This qualitative study builds upon 20 semi-structured interviews with participants and two focus groups with the coaches and program advisory board members of this QI program. Data were coded and analyzed using thematic analysis. Results: Three themes emerged from the analysis: "interprofessional education", "networking" and "motivation: presence with pitfalls". Working within interprofessional project groups was valuable, because participants with different experiences and skills helped to move the QI project forward. It was simultaneously challenging because IPE was new and revealed problems with hierarchy, communication and planning. Networking was also deemed valuable, but a shared space to keep in contact after finalizing the program was missing. The participants were highly motivated to finish their QI project, but they underestimated the challenges. Conclusions: A hospital-wide QI program must explicitly pay attention to interprofessional collaboration and networking. Leaders of the QI program must cherish the motivation of the participants and make sure that the QI projects are realistic.

First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs

Policy Points Realist evaluation (RE) is an emerging and promising research approach for evaluating integrated care, addressing what works, how, for whom, and in what circumstances. The rich philosophical foundation of RE, critical realism, can help to systematically unravel an integrated care program's initial theory prior to implementation, as a first step within RE. RE can be considered a robust methodological asset in integrated care research by facilitating a deeper level of insight into program functioning than traditional forms of evaluation do and by shaping a realist-informed monitoring and evaluation process.

CONTEXT

The complexity of integrated care and the need for transferable evaluation insights ask for a suitable evaluation paradigm. Realist evaluation (RE), underpinned by the philosophy of critical realism, is a theory-driven approach that addresses what works, how, for whom, and in what circumstances. The current study illustrates the process needed for RE's first step: eliciting the initial program theory (IPT). The TARGET program, a Dutch primary care initiative to facilitate more integrated care for chronically ill patients, i.e., care that is efficient, tailored, and holistic,  was taken as a real-world case.

METHODS

An RE approach informed the phased IPT elicitation: (1) identifying an abstract theory framework; (2) formulating the preliminary IPT, building on the abstract theory and informed by previous scientific studies that underpin TARGET; and (3) refining the preliminary IPT, informed by RE expert interviews (n = 7). An RE heuristic tool, specifying the interplay between intervention-context-actors-mechanisms-outcomes (ICAMO) and retroductive reasoning, was applied to synthesize the underlying theory of individual TARGET components into TARGET's IPT.

FINDINGS

Separate but related IPTs were identified for the two main types of actors involved in TARGET: primary care professionals (PCPs) and patients. For both actors, two sorts of mechanisms are assumed to be activated by TARGET, which-via instrumental outcomes-contribute to long-term quadruple aim targets. The first is confidence to enhance PCPs' person-centered conversational skills and to increase patients' active engagement in TARGET. The second is mutual trust, between PCPs and patients and between PCPs and their network partners. A supportive context is assumed crucial for activating these mechanisms-for example, sufficient resources to invest in integrated care.

CONCLUSIONS

Although the IPT elicitation process is time intensive and requires a mind shift, it facilitates a deeper insight into program functioning than accommodated by the prevailing experimental designs in integrated care. Furthermore, the design of a realist-informed evaluation process can be informed by the IPT.

[The Logic Model - a rationale for an encompassing evaluation framework of the German Competence Centers for Postgraduate Medical Education in general practice]

BACKGROUND

The Competence Centers for Postgraduate Medical Education (KW) established throughout Germany represent complex programs to increase the efficiency and quality of postgraduate medical education in general practice. So-called Logic Models serve as the framework for evaluations and quality management processes of complex programs in many areas. The aim of this article is to develop such a model, using the example of the Competence Center in Baden-Württemberg (KWBW) in order to structure its complex program logic and use it as a framework for future evaluations and quality management processes.

METHODS

The adaptation of the Logic Model to the KWBW took place in an informal, nominal group process with employees and participants of the program.

RESULTS

We identified 76 core elements of the KWBW. These core elements were classified in one of the five pillars of the Logic Model (input, activity, output, outcome and impact) and categorized according to fields of action and target groups.

DISCUSSION

The Logic Model, which was developed using the KWBW as an example, identifies and structures important core elements of a complex postgraduate medical training program for the first time. It seems to be well suited for visualizing the internal logic of this complex program and the interaction of the various elements within the KWBW. It can thus be used as the basis for a comprehensive and systematic evaluation concept and for quality assurance.

CONCLUSION

The model is also a prerequisite for comparative research questions of other university programs for postgraduate medical education and thus offers an opportunity for cooperative development efforts. This article therefore contributes to promoting quality in continuing medical training.

Accelerating learning healthcare system development through embedded research: Career trajectories, training needs, and strategies for managing and supporting embedded researchers

BACKGROUND

Health systems and organizations seeking to achieve learning healthcare system principles are increasingly relying on embedded research teams to optimize delivery of evidence-based, high-quality care that improves patient and staff experience alike. However, building organizational capacity to conduct and benefit from embedded research may be challenging in the absence of clearer guidance on career pathways and training, as well as strategies for managing and supporting this unique workforce.

METHODS

In February 2018, 115 attendees from multiple agencies, institutions and professional societies participated in a conference to accelerate development of learning healthcare systems through embedded research. Workgroups engaged in structured brainstorming discussions of key domains; 21 diverse members focused on strengthening the embedded research community through more explicit development and support of multilevel career trajectories.

RESULTS

Emphasis emerged on the need for training that goes beyond traditional curricula in rigorous scientific methods to include leadership, communication, and other organizational and business skills rarely offered in research training programs. These skills are required for effective engagement of multilevel stakeholders supporting evidence-based changes in routine care. Improving readiness of other stakeholders to effectively act on evidence was noted as equally crucial, as was creation of mid-career development opportunities for researchers and implementers.

CONCLUSIONS

Further development and support of the embedded research workforce will require explicit attention to novel training programs and support of researchers and the stakeholders in the systems they aim to improve.

IMPLICATIONS

Strategies for improving career entry and mastery of skills that foster effective multilevel stakeholder engagement hold promise for strengthening the embedded research community and their contributions to systematic improvements in health and health care.

Evolution and Evaluation of a Quality and Patient Safety Fellowship for Practicing Health Professionals

Developing clinical quality champions is an important strategy for improving health care quality. The NorthShore Quality and Patient Safety Fellowship was a yearlong program for practicing physicians devoting 4 hours/wk to a didactic curriculum and quality practicum. Thirty-seven clinicians completed the Fellowship from 2011 to 2018. Sixty percent of graduates reported a significant impact on their quality-related career trajectory, with 44% of early graduates and 64% of recent graduates reporting a new quality role or responsibility as a result of the Fellowship. Fifty-four percent of practicum projects were adopted or adapted by the organization. The Fellowship has been an effective framework to identify and train future quality champions and has led to further quality leadership opportunities for many graduates. Evolution of the Fellowship aligned practicum projects with organizational quality priorities. This curricular framework may be useful for other organizations that seek to develop quality champions among practicing physicians.

Developing Leaders and Scholars in Health Care Improvement: The VA Quality Scholars Program Competencies

Despite the need for leaders in health care improvement across health professions, there are no standards for the knowledge and skills that should be achieved through advanced interprofessional health care improvement training. Existing health care improvement training competencies focus on foundational knowledge expected of all trainees or for specific career pathways. Health care improvement leaders fill multiple roles within organizations and promote interprofessional improvement practice. The diverse skill set required of modern health care improvement leaders necessitates the development of training competencies specifically for fellowships in applied health care improvement. The authors describe the development of the revised national Veterans Affairs Quality Scholars (VAQS) Program competencies. The VAQS Program is an interprofessional, postdoctoral training program whose mission is to develop leaders and scholars to improve health care. An interprofessional committee of VAQS faculty reviewed and revised the competencies over 4 months beginning in fall 2018. The first draft was developed using 111 competencies submitted by 11 VAQS training sites and a review of published competencies. The final version included 22 competencies spanning 5 domains: interprofessional collaboration and teamwork, improvement and implementation science, organization and system leadership, methodological skills and analytic techniques for improvement and research, and teaching and coaching. Once attained, the VAQS competencies will guide the skill development that interprofessional health care improvement leaders need to participate in and lead health care improvement scholarship and implementation. These broad competencies are relevant to advanced training programs that develop health care improvement leaders and scholars and may be used by employers to understand the knowledge and skills expected of individuals who complete advanced fellowships in applied health care improvement.

Development of a Hybrid, Interprofessional, Interactive Quality Improvement Curriculum as a Model for Continuing Professional Development

Over the past 20 years, there has been an increased focus on quality improvement (QI) in health care, which is critical in achieving care that is patient-centered, safer, timelier, and more effective, efficient, and equitable. At the center of this movement is QI education, which is known to lead to learning, behavior change, and improved outcomes. However, there is a need for the development and provision of long-duration, interactive, interprofessional training in QI, to allow for in-depth learning and application of learned skills. To this end, we designed a curriculum for an established interprofessional, interactive, web-based QI fellowship for doctorally prepared clinicians. Curricular content is delivered virtually to geographically dispersed learners over a 2-year time span. The didactic curriculum and experiential learning opportunities provide learners with the foundational knowledge and practical skills to engage in-and eventually, lead-QI initiatives around the country. Evaluation of learner satisfaction and cognitive, affective, and skills-based learning has found that this model is an effective method to train geographically distributed learners. A hybrid training structure is used, where learners interact with the material through 3 distinct delivery modes: (1) virtual instruction in QI topics; (2) face-to-face training, mentorship, and the opportunity for practical application of applied knowledge and skills through the completion of QI projects; and (3) opportunities for other types of training, tailored to each learner's Individual Development Plan. This training program model holds value for QI learning in various health care settings, which are interprofessional by nature. These foundational concepts of hybrid learning to distributed learners-wherein an instructor delivers curriculum in small, face-to-face batches, interprofessional learning is supplemented in a virtual, longitudinal manner, and learners are allowed the opportunity to put skills into action for real-world problems in interdisciplinary clinical teams-can be applied in a multitude of settings, with comparatively lower time and cost expenditure than traditional training programs.